2024-05-10T04:16:39.121 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-10T04:16:39.122 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} --archive /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547 --owner scheduled_pdonnell@teuthology --name pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/orig.config.yaml 2024-05-10T04:16:39.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T04:16:39.277 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547 branch: wip-pdonnell-testing-20240509.171335-reef-debug description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} email: pdonnell@redhat.com first_in_suite: false job_id: '7700547' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240509.171335-reef-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 50174015a7e79f951b988949abe236791f52d4da subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 50174015a7e79f951b988949abe236791f52d4da kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240509.171335-reef-debug sha1: 50174015a7e79f951b988949abe236791f52d4da owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4933 sha1: 50174015a7e79f951b988949abe236791f52d4da sleep_before_teardown: 0 subset: 420/512 suite: fs suite_branch: wip-pdonnell-testing-20240509.171335-reef-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50174015a7e79f951b988949abe236791f52d4da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMeyq81OFeL8xLJpN2UlcrXpnJE3ymfT8IGZH4EMNsH845sStFYc+vshogNZTAdXvc5J05ZdvUKRPeepl30c9E= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLj3lagIhnKgZspcz98Rk9t2AAAEIt4E4Bm8zAQ7ovuVMxIM5fXcTPEuMAxvXyYOzG1V7I3Jt3BDM16DzDP4Kv0= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJDo/iGAI7KdpfoIkfazaI+qow3VUGHxgXqZIdLatnBv55gJfP90zgyrSCn280Dk4H5GCui4YGbT2VCynloWrvk= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-10_02:33:39 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-10T04:16:39.277 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa; will attempt to use it 2024-05-10T04:16:39.278 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50174015a7e79f951b988949abe236791f52d4da/qa/tasks 2024-05-10T04:16:39.278 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-10T04:16:39.280 INFO:teuthology.task.internal:Checking packages... 2024-05-10T04:16:39.305 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '50174015a7e79f951b988949abe236791f52d4da' 2024-05-10T04:16:39.305 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-10T04:16:39.305 INFO:teuthology.packaging:ref: None 2024-05-10T04:16:39.305 INFO:teuthology.packaging:tag: None 2024-05-10T04:16:39.305 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T04:16:39.305 INFO:teuthology.packaging:sha1: 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:16:39.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T04:16:39.526 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1330.g50174015 2024-05-10T04:16:39.528 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-10T04:16:39.535 INFO:teuthology.task.internal:no buildpackages task found 2024-05-10T04:16:39.535 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-10T04:16:39.548 INFO:teuthology.task.internal:Saving configuration 2024-05-10T04:16:39.568 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-10T04:16:39.576 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-10T04:16:39.603 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 04:11:21.550402', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMeyq81OFeL8xLJpN2UlcrXpnJE3ymfT8IGZH4EMNsH845sStFYc+vshogNZTAdXvc5J05ZdvUKRPeepl30c9E='} 2024-05-10T04:16:39.622 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 04:11:21.549280', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLj3lagIhnKgZspcz98Rk9t2AAAEIt4E4Bm8zAQ7ovuVMxIM5fXcTPEuMAxvXyYOzG1V7I3Jt3BDM16DzDP4Kv0='} 2024-05-10T04:16:39.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-10 04:11:21.547960', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJDo/iGAI7KdpfoIkfazaI+qow3VUGHxgXqZIdLatnBv55gJfP90zgyrSCn280Dk4H5GCui4YGbT2VCynloWrvk='} 2024-05-10T04:16:39.648 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-10T04:16:39.655 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-10T04:16:39.655 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-10T04:16:39.655 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-10T04:16:39.655 INFO:teuthology.run_tasks:Running task console_log... 2024-05-10T04:16:39.769 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc8ffe590d0>, signals=[15]) 2024-05-10T04:16:39.770 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-10T04:16:39.780 INFO:teuthology.task.internal:Opening connections... 2024-05-10T04:16:39.780 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:16:39.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T04:16:39.858 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-05-10T04:16:39.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T04:16:39.975 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-05-10T04:16:39.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T04:16:40.052 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-10T04:16:40.064 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-05-10T04:16:40.087 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-05-10T04:16:40.087 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:ID="rhel" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="fedora" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8.6" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T04:16:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:16:40.146 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T04:16:40.146 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T04:16:40.146 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T04:16:40.146 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T04:16:40.152 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-05-10T04:16:40.187 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-05-10T04:16:40.209 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-05-10T04:16:40.209 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-05-10T04:16:40.269 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T04:16:40.269 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T04:16:40.269 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2024-05-10T04:16:40.269 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2024-05-10T04:16:40.269 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T04:16:40.270 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T04:16:40.271 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-05-10T04:16:40.304 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-05-10T04:16:40.325 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-05-10T04:16:40.325 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-05-10T04:16:40.385 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2024-05-10T04:16:40.385 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.6 (Ootpa)" 2024-05-10T04:16:40.385 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2024-05-10T04:16:40.385 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2024-05-10T04:16:40.385 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.6" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-10T04:16:40.386 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-10T04:16:40.387 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-05-10T04:16:40.403 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-10T04:16:40.416 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-10T04:16:40.431 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-10T04:16:40.431 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-05-10T04:16:40.433 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-05-10T04:16:40.435 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-05-10T04:16:40.454 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-10T04:16:40.466 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-10T04:16:40.466 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-05-10T04:16:40.492 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-05-10T04:16:40.495 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-05-10T04:16:40.524 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-10T04:16:40.607 INFO:teuthology.run_tasks:Running task kernel... 2024-05-10T04:16:40.626 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-10T04:16:40.626 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-10T04:16:40.626 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-10T04:16:40.627 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T04:16:40.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:16:40.646 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-05-10T04:16:40.704 INFO:teuthology.orchestra.run.smithi150.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-10T04:16:40.704 INFO:teuthology.task.kernel:Running kernel on smithi150: 4.18.0-513.9.1.el8_9.x86_64 2024-05-10T04:16:40.704 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2024-05-10T04:16:42.214 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T04:16:42.214 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2024-05-10T04:16:42.517 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:16:42.517 INFO:teuthology.orchestra.run.smithi150.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-10T04:16:42.517 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:16:44.280 INFO:teuthology.orchestra.run.smithi150.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-05-10T04:16:45.300 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 14 MB/s | 14 MB 00:00 2024-05-10T04:16:48.617 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 411 kB/s | 24 kB 00:00 2024-05-10T04:16:49.370 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-10T04:16:49.370 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-10T04:16:49.445 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T04:16:49.446 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-05-10T04:16:49.446 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T04:16:49.591 DEBUG:teuthology.orchestra.run.smithi150:> echo no | sudo yum reinstall kernel || true 2024-05-10T04:16:49.926 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T04:16:49.927 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2024-05-10T04:16:50.162 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:16:50.162 INFO:teuthology.orchestra.run.smithi150.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-10T04:16:50.162 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:16:50.475 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:02 ago on Fri 10 May 2024 04:16:48 AM UTC. 2024-05-10T04:16:50.556 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-10T04:16:50.556 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-10T04:16:50.594 INFO:teuthology.orchestra.run.smithi150.stderr:Error: No packages marked for reinstall. 2024-05-10T04:16:50.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum reinstall -y kernel || true 2024-05-10T04:16:50.984 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T04:16:50.984 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2024-05-10T04:16:51.301 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:16:51.301 INFO:teuthology.orchestra.run.smithi150.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-10T04:16:51.301 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:16:51.604 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:03 ago on Fri 10 May 2024 04:16:48 AM UTC. 2024-05-10T04:16:51.673 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-10T04:16:51.673 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-10T04:16:51.699 INFO:teuthology.orchestra.run.smithi150.stderr:Error: No packages marked for reinstall. 2024-05-10T04:16:51.751 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-05-10T04:16:51.917 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-10T04:16:51.918 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-10T04:16:51.918 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-10T04:16:51.918 DEBUG:teuthology.parallel:result is None 2024-05-10T04:16:51.918 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-10T04:16:51.925 INFO:teuthology.task.internal:Creating test directory... 2024-05-10T04:16:51.925 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T04:16:51.928 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T04:16:51.930 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-10T04:16:51.951 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-10T04:16:51.960 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-10T04:16:51.969 INFO:teuthology.task.internal:Creating archive directory... 2024-05-10T04:16:51.969 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T04:16:51.985 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T04:16:51.991 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-10T04:16:52.028 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-10T04:16:52.037 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-10T04:16:52.037 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T04:16:52.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:16:52.078 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-10T04:16:52.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:16:52.096 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T04:16:52.098 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T04:16:52.121 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-10T04:16:52.130 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T04:16:52.151 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T04:16:52.163 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T04:16:52.183 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T04:16:52.183 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T04:16:52.197 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-10T04:16:52.199 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-10T04:16:52.208 INFO:teuthology.task.internal:Configuring sudo... 2024-05-10T04:16:52.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T04:16:52.210 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T04:16:52.212 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-10T04:16:52.241 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-10T04:16:52.252 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-10T04:16:52.252 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T04:16:52.280 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T04:16:52.283 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-10T04:16:52.300 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T04:16:52.379 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T04:16:52.464 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T04:16:52.508 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T04:16:52.549 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:16:52.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T04:16:52.616 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T04:16:52.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T04:16:52.716 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T04:16:52.800 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T04:16:52.845 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:16:52.845 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T04:16:52.914 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T04:16:52.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-10T04:16:53.021 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T04:16:53.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-10T04:16:53.154 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:16:53.154 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-10T04:16:53.224 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-05-10T04:16:53.226 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-05-10T04:16:53.229 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-05-10T04:16:53.270 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:16:53.295 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:16:53.337 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:16:53.675 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-10T04:16:53.687 INFO:teuthology.task.internal:Starting timer... 2024-05-10T04:16:53.687 INFO:teuthology.run_tasks:Running task pcp... 2024-05-10T04:16:53.705 INFO:teuthology.run_tasks:Running task selinux... 2024-05-10T04:16:53.721 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-05-10T04:16:53.722 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-05-10T04:16:53.801 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-05-10T04:16:53.803 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-05-10T04:16:53.888 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-05-10T04:16:53.890 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-05-10T04:16:53.947 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-05-10T04:16:53.949 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-10T04:16:53.949 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-05-10T04:16:53.977 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-05-10T04:16:53.977 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-05-10T04:16:54.000 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-05-10T04:16:54.001 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-05-10T04:16:54.029 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-05-10T04:16:54.029 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2024-05-10T04:16:54.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:16:54.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:16:54.059 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:16:54.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:16:54.088 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:16:54.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:16:54.115 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-10T04:16:54.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-05-10T04:16:54.142 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-05-10T04:16:54.179 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-05-10T04:16:54.224 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-10T04:16:54.234 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-10T04:16:54.251 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-10T04:16:54.252 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi150.front.sepia.ceph.com,smithi161.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-10T04:27:29.163 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2024-05-10T04:27:29.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-10T04:27:29.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T04:27:29.242 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-05-10T04:27:29.269 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-10T04:27:29.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-10T04:27:29.270 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T04:27:29.346 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-05-10T04:27:29.413 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-05-10T04:27:29.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-05-10T04:27:29.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-10T04:27:29.494 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-05-10T04:27:29.564 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-05-10T04:27:29.564 INFO:teuthology.run_tasks:Running task clock... 2024-05-10T04:27:29.576 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-10T04:27:29.576 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T04:27:29.576 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:27:29.579 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T04:27:29.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:27:29.581 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-10T04:27:29.581 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:27:29.610 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T04:27:29.613 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T04:27:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T04:27:29.629 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T04:27:29.635 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-10T04:27:29.650 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-10T04:27:29.693 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-05-10T04:27:29.704 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-05-10T04:27:29.710 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-05-10T04:27:29.720 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T04:27:29.721 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-05-10T04:27:29.722 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-05-10T04:27:29.733 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-05-10T04:27:29.734 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T04:27:29.738 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T04:27:29.748 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-10T04:27:29.752 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T04:27:29.762 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-10T04:27:29.811 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2024-05-10T04:27:29.814 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:27:29.814 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-10T04:27:29.814 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.814 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.815 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.815 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.831 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2024-05-10T04:27:29.835 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:27:29.835 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-05-10T04:27:29.835 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.836 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.836 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.836 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.841 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2024-05-10T04:27:29.844 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:27:29.844 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-05-10T04:27:29.844 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.845 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.845 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.845 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:27:29.845 INFO:teuthology.run_tasks:Running task pexec... 2024-05-10T04:27:29.855 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-10T04:27:29.855 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:27:29.856 DEBUG:teuthology.orchestra.run.smithi150:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T04:27:29.856 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi161.front.sepia.ceph.com 2024-05-10T04:27:29.856 DEBUG:teuthology.orchestra.run.smithi161:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T04:27:29.856 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi181.front.sepia.ceph.com 2024-05-10T04:27:29.857 DEBUG:teuthology.orchestra.run.smithi181:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-10T04:27:30.236 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T04:27:30.255 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T04:27:30.264 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T04:27:32.105 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-05-10T04:27:32.163 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-05-10T04:27:32.304 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 kB/s | 4.1 kB 00:00 2024-05-10T04:27:32.569 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-05-10T04:27:32.689 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-05-10T04:27:32.731 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-05-10T04:27:35.644 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-10T04:27:35.646 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-10T04:27:35.646 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-05-10T04:27:35.646 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-10T04:27:35.646 INFO:teuthology.orchestra.run.smithi161.stdout:Resetting modules: 2024-05-10T04:27:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools 2024-05-10T04:27:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:27:35.647 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-10T04:27:35.647 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-05-10T04:27:35.647 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:27:35.766 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout:Resetting modules: 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-05-10T04:27:35.768 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:27:35.817 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout:Resetting modules: 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-10T04:27:35.818 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-05-10T04:27:35.819 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:27:48.415 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-10T04:27:48.892 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T04:27:50.163 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-10T04:27:50.275 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T04:27:50.633 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T04:27:50.739 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T04:27:50.957 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-05-10T04:27:51.380 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-05-10T04:27:52.825 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-05-10T04:27:52.911 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-05-10T04:27:53.276 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-05-10T04:27:53.469 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 kB/s | 4.1 kB 00:00 2024-05-10T04:27:54.403 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-10T04:27:54.566 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================================== 2024-05-10T04:27:54.566 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================================== 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout:Installing group/module packages: 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T04:27:54.567 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout:Installing module profiles: 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools/common 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools rhel8 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================================== 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout:Install 13 Packages 2024-05-10T04:27:54.568 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:27:54.569 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 26 M 2024-05-10T04:27:54.569 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 87 M 2024-05-10T04:27:54.569 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-05-10T04:27:54.698 INFO:teuthology.orchestra.run.smithi181.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 192 kB/s | 25 kB 00:00 2024-05-10T04:27:54.730 INFO:teuthology.orchestra.run.smithi181.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 245 kB/s | 39 kB 00:00 2024-05-10T04:27:54.831 INFO:teuthology.orchestra.run.smithi181.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 360 kB/s | 94 kB 00:00 2024-05-10T04:27:54.847 INFO:teuthology.orchestra.run.smithi181.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 275 kB/s | 41 kB 00:00 2024-05-10T04:27:54.890 INFO:teuthology.orchestra.run.smithi181.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.3 MB/s | 1.0 MB 00:00 2024-05-10T04:27:54.956 INFO:teuthology.orchestra.run.smithi181.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 6.7 MB/s | 739 kB 00:00 2024-05-10T04:27:55.007 INFO:teuthology.orchestra.run.smithi181.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.0 MB/s | 239 kB 00:00 2024-05-10T04:27:55.115 INFO:teuthology.orchestra.run.smithi181.stdout:(8/13): netavark-1.7.0-2.module+el8.9.0+21697+6 23 MB/s | 3.7 MB 00:00 2024-05-10T04:27:55.166 INFO:teuthology.orchestra.run.smithi181.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 968 kB/s | 154 kB 00:00 2024-05-10T04:27:55.299 INFO:teuthology.orchestra.run.smithi181.stdout:(10/13): buildah-1.31.5-1.module+el8.9.0+21697+ 19 MB/s | 8.8 MB 00:00 2024-05-10T04:27:55.391 INFO:teuthology.orchestra.run.smithi181.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 11 MB/s | 2.5 MB 00:00 2024-05-10T04:27:55.416 INFO:teuthology.orchestra.run.smithi181.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 424 kB/s | 49 kB 00:00 2024-05-10T04:27:55.533 INFO:teuthology.orchestra.run.smithi181.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-10T04:27:55.534 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-05-10T04:27:55.534 INFO:teuthology.orchestra.run.smithi181.stdout:Total 26 MB/s | 26 MB 00:00 2024-05-10T04:27:55.815 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-05-10T04:27:55.836 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-05-10T04:27:55.836 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-05-10T04:27:56.066 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-05-10T04:27:56.074 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-05-10T04:27:56.306 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-10T04:27:56.319 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T04:27:56.471 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-05-10T04:27:56.471 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-05-10T04:27:56.471 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-05-10T04:27:56.471 INFO:teuthology.orchestra.run.smithi161.stdout:Installing group/module packages: 2024-05-10T04:27:56.471 INFO:teuthology.orchestra.run.smithi161.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T04:27:56.471 INFO:teuthology.orchestra.run.smithi161.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T04:27:56.471 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout:Installing module profiles: 2024-05-10T04:27:56.472 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools/common 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools rhel8 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout:Install 13 Packages 2024-05-10T04:27:56.473 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:27:56.474 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 26 M 2024-05-10T04:27:56.474 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 87 M 2024-05-10T04:27:56.474 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-05-10T04:27:56.487 INFO:teuthology.orchestra.run.smithi150.stdout:========================================================================================================== 2024-05-10T04:27:56.487 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-05-10T04:27:56.487 INFO:teuthology.orchestra.run.smithi150.stdout:========================================================================================================== 2024-05-10T04:27:56.487 INFO:teuthology.orchestra.run.smithi150.stdout:Installing group/module packages: 2024-05-10T04:27:56.487 INFO:teuthology.orchestra.run.smithi150.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-10T04:27:56.487 INFO:teuthology.orchestra.run.smithi150.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-10T04:27:56.487 INFO:teuthology.orchestra.run.smithi150.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-10T04:27:56.488 INFO:teuthology.orchestra.run.smithi150.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-10T04:27:56.489 INFO:teuthology.orchestra.run.smithi150.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-10T04:27:56.490 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-05-10T04:27:56.490 INFO:teuthology.orchestra.run.smithi150.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T04:27:56.490 INFO:teuthology.orchestra.run.smithi150.stdout:Installing module profiles: 2024-05-10T04:27:56.490 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools/common 2024-05-10T04:27:56.490 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:27:56.492 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling module streams: 2024-05-10T04:27:56.492 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools rhel8 2024-05-10T04:27:56.492 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:27:56.492 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-10T04:27:56.492 INFO:teuthology.orchestra.run.smithi150.stdout:========================================================================================================== 2024-05-10T04:27:56.492 INFO:teuthology.orchestra.run.smithi150.stdout:Install 13 Packages 2024-05-10T04:27:56.492 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:27:56.493 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 26 M 2024-05-10T04:27:56.493 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 87 M 2024-05-10T04:27:56.493 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-05-10T04:27:56.651 INFO:teuthology.orchestra.run.smithi161.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 141 kB/s | 25 kB 00:00 2024-05-10T04:27:56.667 INFO:teuthology.orchestra.run.smithi161.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 204 kB/s | 39 kB 00:00 2024-05-10T04:27:56.685 INFO:teuthology.orchestra.run.smithi150.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 127 kB/s | 25 kB 00:00 2024-05-10T04:27:56.695 INFO:teuthology.orchestra.run.smithi161.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 430 kB/s | 94 kB 00:00 2024-05-10T04:27:56.719 INFO:teuthology.orchestra.run.smithi150.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 410 kB/s | 94 kB 00:00 2024-05-10T04:27:56.735 INFO:teuthology.orchestra.run.smithi161.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 489 kB/s | 41 kB 00:00 2024-05-10T04:27:56.802 INFO:teuthology.orchestra.run.smithi161.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 7.5 MB/s | 1.0 MB 00:00 2024-05-10T04:27:56.836 INFO:teuthology.orchestra.run.smithi150.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 114 kB/s | 39 kB 00:00 2024-05-10T04:27:56.853 INFO:teuthology.orchestra.run.smithi150.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 245 kB/s | 41 kB 00:00 2024-05-10T04:27:56.860 INFO:teuthology.orchestra.run.smithi161.stdout:(6/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 4.0 MB/s | 239 kB 00:00 2024-05-10T04:27:56.861 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-05-10T04:27:56.928 INFO:teuthology.orchestra.run.smithi150.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 4.8 MB/s | 1.0 MB 00:00 2024-05-10T04:27:56.961 INFO:teuthology.orchestra.run.smithi161.stdout:(7/13): cockpit-podman-75-1.module+el8.9.0+2169 3.2 MB/s | 739 kB 00:00 2024-05-10T04:27:57.003 INFO:teuthology.orchestra.run.smithi150.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.8 MB/s | 739 kB 00:00 2024-05-10T04:27:57.078 INFO:teuthology.orchestra.run.smithi150.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.6 MB/s | 239 kB 00:00 2024-05-10T04:27:57.211 INFO:teuthology.orchestra.run.smithi161.stdout:(8/13): netavark-1.7.0-2.module+el8.9.0+21697+6 11 MB/s | 3.7 MB 00:00 2024-05-10T04:27:57.228 INFO:teuthology.orchestra.run.smithi161.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 576 kB/s | 154 kB 00:00 2024-05-10T04:27:57.245 INFO:teuthology.orchestra.run.smithi150.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 22 MB/s | 8.8 MB 00:00 2024-05-10T04:27:57.271 INFO:teuthology.orchestra.run.smithi150.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 800 kB/s | 154 kB 00:00 2024-05-10T04:27:57.328 INFO:teuthology.orchestra.run.smithi161.stdout:(10/13): buildah-1.31.5-1.module+el8.9.0+21697+ 14 MB/s | 8.8 MB 00:00 2024-05-10T04:27:57.379 INFO:teuthology.orchestra.run.smithi150.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 9.9 MB/s | 3.7 MB 00:00 2024-05-10T04:27:57.405 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T04:27:57.470 INFO:teuthology.orchestra.run.smithi161.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 10 MB/s | 2.5 MB 00:00 2024-05-10T04:27:57.495 INFO:teuthology.orchestra.run.smithi161.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 296 kB/s | 49 kB 00:00 2024-05-10T04:27:57.505 INFO:teuthology.orchestra.run.smithi150.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 11 MB/s | 2.5 MB 00:00 2024-05-10T04:27:57.530 INFO:teuthology.orchestra.run.smithi150.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 329 kB/s | 49 kB 00:00 2024-05-10T04:27:57.570 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T04:27:57.587 INFO:teuthology.orchestra.run.smithi161.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 22 MB/s | 8.1 MB 00:00 2024-05-10T04:27:57.588 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-05-10T04:27:57.588 INFO:teuthology.orchestra.run.smithi161.stdout:Total 23 MB/s | 26 MB 00:01 2024-05-10T04:27:57.622 INFO:teuthology.orchestra.run.smithi150.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 22 MB/s | 8.1 MB 00:00 2024-05-10T04:27:57.623 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-10T04:27:57.623 INFO:teuthology.orchestra.run.smithi150.stdout:Total 23 MB/s | 26 MB 00:01 2024-05-10T04:27:57.754 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T04:27:57.869 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-05-10T04:27:57.891 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-05-10T04:27:57.891 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-05-10T04:27:57.903 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-05-10T04:27:57.922 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T04:27:57.924 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-05-10T04:27:57.925 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-05-10T04:27:58.100 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T04:27:58.120 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-05-10T04:27:58.129 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-05-10T04:27:58.153 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-05-10T04:27:58.162 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-05-10T04:27:58.326 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T04:27:58.570 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T04:27:58.858 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-05-10T04:27:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-05-10T04:27:59.343 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T04:27:59.434 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T04:27:59.452 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-10T04:27:59.540 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T04:27:59.642 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T04:27:59.644 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-10T04:27:59.682 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T04:27:59.785 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-10T04:27:59.825 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T04:27:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-10T04:27:59.995 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T04:28:00.046 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T04:28:00.148 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-10T04:28:00.196 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T04:28:00.457 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T04:28:00.633 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-10T04:28:00.885 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-10T04:28:00.912 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T04:28:01.132 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T04:28:01.321 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T04:28:01.349 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T04:28:01.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T04:28:01.755 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T04:28:01.756 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T04:28:01.756 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T04:28:01.756 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T04:28:01.756 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T04:28:01.758 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T04:28:01.899 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T04:28:02.024 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T04:28:02.410 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-10T04:28:02.765 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T04:28:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T04:28:03.144 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T04:28:03.293 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-10T04:28:03.522 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-10T04:28:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T04:28:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T04:28:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T04:28:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T04:28:03.559 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T04:28:03.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T04:28:03.705 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T04:28:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-10T04:28:04.113 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-10T04:28:04.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-10T04:28:13.273 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T04:28:13.273 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2024-05-10T04:28:13.849 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:28:13.849 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-05-10T04:28:13.849 INFO:teuthology.orchestra.run.smithi181.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:13.849 INFO:teuthology.orchestra.run.smithi181.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:13.850 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:13.850 INFO:teuthology.orchestra.run.smithi181.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T04:28:13.850 INFO:teuthology.orchestra.run.smithi181.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:13.850 INFO:teuthology.orchestra.run.smithi181.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:13.850 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:13.850 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T04:28:13.851 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T04:28:13.851 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:13.851 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:13.851 INFO:teuthology.orchestra.run.smithi181.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:13.851 INFO:teuthology.orchestra.run.smithi181.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T04:28:13.851 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:28:13.851 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-10T04:28:14.013 DEBUG:teuthology.parallel:result is None 2024-05-10T04:28:16.806 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T04:28:16.806 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2024-05-10T04:28:17.352 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:28:17.352 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:28:17.353 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-10T04:28:17.517 DEBUG:teuthology.parallel:result is None 2024-05-10T04:28:18.753 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-10T04:28:18.754 INFO:teuthology.orchestra.run.smithi150.stdout:Installed products updated. 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:19.328 INFO:teuthology.orchestra.run.smithi150.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:28:19.329 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T04:28:19.503 DEBUG:teuthology.parallel:result is None 2024-05-10T04:28:19.504 INFO:teuthology.run_tasks:Running task install... 2024-05-10T04:28:19.512 DEBUG:teuthology.task.install:project ceph 2024-05-10T04:28:19.513 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'}} 2024-05-10T04:28:19.513 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50174015a7e79f951b988949abe236791f52d4da'} 2024-05-10T04:28:19.513 INFO:teuthology.task.install:Using flavor: default 2024-05-10T04:28:19.521 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-10T04:28:19.521 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-10T04:28:19.521 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T04:28:19.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:28:19.524 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T04:28:19.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:28:19.525 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'tag': None, 'wait_for_package': False} 2024-05-10T04:28:19.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:28:19.674 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T04:28:19.674 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T04:28:19.748 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T04:28:19.748 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T04:28:19.752 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/ 2024-05-10T04:28:19.752 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1330.g50174015 2024-05-10T04:28:19.947 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T04:28:19.947 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:28:19.947 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T04:28:19.981 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T04:28:19.981 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T04:28:20.019 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T04:28:20.020 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:28:20.020 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T04:28:20.029 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-10T04:28:20.029 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:28:20.029 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-10T04:28:20.049 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T04:28:20.049 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T04:28:20.053 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T04:28:20.063 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-10T04:28:20.063 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50174015a7e79f951b988949abe236791f52d4da/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-10T04:28:20.118 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T04:28:20.134 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T04:28:20.136 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-10T04:28:20.196 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T04:28:20.202 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-05-10T04:28:20.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-05-10T04:28:20.218 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-10T04:28:20.262 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-05-10T04:28:20.264 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-05-10T04:28:20.287 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-05-10T04:28:20.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-05-10T04:28:20.584 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T04:28:20.641 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T04:28:20.672 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T04:28:22.234 INFO:teuthology.orchestra.run.smithi181.stdout:61 files removed 2024-05-10T04:28:22.261 INFO:teuthology.orchestra.run.smithi161.stdout:61 files removed 2024-05-10T04:28:22.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T04:28:22.308 INFO:teuthology.orchestra.run.smithi150.stdout:61 files removed 2024-05-10T04:28:22.319 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T04:28:22.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-10T04:28:22.624 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2024-05-10T04:28:22.656 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-05-10T04:28:22.697 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2024-05-10T04:28:24.839 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 140 kB/s | 80 kB 00:00 2024-05-10T04:28:24.986 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 113 kB/s | 80 kB 00:00 2024-05-10T04:28:25.125 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 108 kB/s | 80 kB 00:00 2024-05-10T04:28:25.415 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 21 kB/s | 11 kB 00:00 2024-05-10T04:28:25.579 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 20 kB/s | 11 kB 00:00 2024-05-10T04:28:25.793 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-05-10T04:28:26.091 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 2.3 kB/s | 1.5 kB 00:00 2024-05-10T04:28:26.188 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2024-05-10T04:28:26.241 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-05-10T04:28:26.380 INFO:teuthology.orchestra.run.smithi150.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-05-10T04:28:26.451 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 2.4 kB/s | 1.5 kB 00:00 2024-05-10T04:28:26.627 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2024-05-10T04:28:27.097 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 20 MB/s | 14 MB 00:00 2024-05-10T04:28:27.513 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 16 MB/s | 14 MB 00:00 2024-05-10T04:28:27.742 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 14 MB 00:01 2024-05-10T04:28:30.506 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 386 kB/s | 24 kB 00:00 2024-05-10T04:28:30.836 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 343 kB/s | 24 kB 00:00 2024-05-10T04:28:31.035 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 343 kB/s | 24 kB 00:00 2024-05-10T04:28:31.191 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-10T04:28:32.330 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 67 MB 00:01 2024-05-10T04:28:32.478 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 67 MB 00:01 2024-05-10T04:28:34.126 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 61 MB 00:01 2024-05-10T04:28:42.547 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-10T04:28:42.720 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-10T04:28:45.390 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 61 MB 00:01 2024-05-10T04:28:45.813 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 61 MB 00:01 2024-05-10T04:28:45.869 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 67 MB 00:01 2024-05-10T04:29:00.778 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T04:29:00.779 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T04:29:00.798 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T04:29:00.799 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T04:29:00.801 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T04:29:00.802 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T04:29:00.805 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T04:29:00.809 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T04:29:00.811 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T04:29:00.963 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T04:29:00.964 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T04:29:00.983 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T04:29:00.984 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T04:29:00.986 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T04:29:00.987 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T04:29:00.990 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T04:29:00.995 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T04:29:00.996 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T04:29:01.034 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-10T04:29:01.048 INFO:teuthology.orchestra.run.smithi150.stdout:=================================================================================================================================== 2024-05-10T04:29:01.048 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-05-10T04:29:01.048 INFO:teuthology.orchestra.run.smithi150.stdout:=================================================================================================================================== 2024-05-10T04:29:01.048 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T04:29:01.049 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T04:29:01.050 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T04:29:01.051 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T04:29:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T04:29:01.053 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T04:29:01.054 INFO:teuthology.orchestra.run.smithi150.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T04:29:01.055 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T04:29:01.056 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T04:29:01.057 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T04:29:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T04:29:01.059 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T04:29:01.060 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling module streams: 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql 10 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout:=================================================================================================================================== 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout:Install 130 Packages 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 16 Packages 2024-05-10T04:29:01.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:29:01.062 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 221 M 2024-05-10T04:29:01.062 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-05-10T04:29:01.218 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-05-10T04:29:01.232 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T04:29:01.233 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T04:29:01.234 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T04:29:01.235 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T04:29:01.236 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T04:29:01.237 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T04:29:01.238 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T04:29:01.239 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T04:29:01.240 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T04:29:01.241 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T04:29:01.242 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T04:29:01.243 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql 10 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout:Install 130 Packages 2024-05-10T04:29:01.244 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 16 Packages 2024-05-10T04:29:01.245 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:29:01.245 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 221 M 2024-05-10T04:29:01.245 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-05-10T04:29:01.417 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T04:29:01.418 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-10T04:29:01.437 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-10T04:29:01.438 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-10T04:29:01.439 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-10T04:29:01.441 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-10T04:29:01.443 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-10T04:29:01.447 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-10T04:29:01.449 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-10T04:29:01.627 INFO:teuthology.orchestra.run.smithi150.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2024-05-10T04:29:01.670 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout:=================================================================================================================================== 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout:=================================================================================================================================== 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.2.2-1330.g50174015.el8 ceph 6.4 k 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.7 M 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 829 k 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1330.g50174015.el8 ceph 157 k 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.2.2-1330.g50174015.el8 ceph 1.5 M 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 138 k 2024-05-10T04:29:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 3.5 M 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 7.4 M 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 49 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 14 M 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.2.2-1330.g50174015.el8 ceph 49 M 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 263 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 173 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 246 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 18 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 31 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 756 k 2024-05-10T04:29:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.2.2-1330.g50174015.el8 ceph 197 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.2.2-1330.g50174015.el8 ceph 371 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 364 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.2.2-1330.g50174015.el8 ceph 107 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.2.2-1330.g50174015.el8 ceph 91 k 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.3 M 2024-05-10T04:29:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.2.2-1330.g50174015.el8 ceph 179 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 3.5 M 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-10T04:29:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 23 M 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 24 k 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.2.2-1330.g50174015.el8 ceph 2.2 M 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 246 k 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.2.2-1330.g50174015.el8 ceph 5.1 M 2024-05-10T04:29:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.2.2-1330.g50174015.el8 ceph 19 M 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1330.g50174015.el8 ceph-noarch 16 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.2.2-1330.g50174015.el8 ceph 24 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.2.2-1330.g50174015.el8 ceph 139 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-10T04:29:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.2.2-1330.g50174015.el8 ceph 461 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.2.2-1330.g50174015.el8 ceph 7.2 M 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-10T04:29:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.2.2-1330.g50174015.el8 ceph 45 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.2.2-1330.g50174015.el8 ceph 129 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-10T04:29:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-10T04:29:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-10T04:29:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-10T04:29:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql 10 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout:=================================================================================================================================== 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout:Install 130 Packages 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 16 Packages 2024-05-10T04:29:01.695 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:29:01.696 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 221 M 2024-05-10T04:29:01.696 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-05-10T04:29:01.714 INFO:teuthology.orchestra.run.smithi161.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2024-05-10T04:29:02.088 INFO:teuthology.orchestra.run.smithi181.stdout:(1/146): ceph-18.2.2-1330.g50174015.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2024-05-10T04:29:02.844 INFO:teuthology.orchestra.run.smithi150.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 682 kB/s | 829 kB 00:01 2024-05-10T04:29:02.955 INFO:teuthology.orchestra.run.smithi181.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 957 kB/s | 829 kB 00:00 2024-05-10T04:29:03.102 INFO:teuthology.orchestra.run.smithi150.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 608 kB/s | 157 kB 00:00 2024-05-10T04:29:03.105 INFO:teuthology.orchestra.run.smithi181.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 1.0 MB/s | 157 kB 00:00 2024-05-10T04:29:03.398 INFO:teuthology.orchestra.run.smithi161.stdout:(2/146): ceph-fuse-18.2.2-1330.g50174015.el8.x8 492 kB/s | 829 kB 00:01 2024-05-10T04:29:03.645 INFO:teuthology.orchestra.run.smithi150.stdout:(4/146): ceph-base-18.2.2-1330.g50174015.el8.x8 2.4 MB/s | 5.7 MB 00:02 2024-05-10T04:29:03.656 INFO:teuthology.orchestra.run.smithi161.stdout:(3/146): ceph-immutable-object-cache-18.2.2-133 608 kB/s | 157 kB 00:00 2024-05-10T04:29:04.147 INFO:teuthology.orchestra.run.smithi181.stdout:(4/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 2.1 MB/s | 2.2 MB 00:01 2024-05-10T04:29:04.178 INFO:teuthology.orchestra.run.smithi150.stdout:(5/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 2.8 MB/s | 1.5 MB 00:00 2024-05-10T04:29:04.564 INFO:teuthology.orchestra.run.smithi181.stdout:(5/146): ceph-base-18.2.2-1330.g50174015.el8.x8 2.0 MB/s | 5.7 MB 00:02 2024-05-10T04:29:04.789 INFO:teuthology.orchestra.run.smithi181.stdout:(6/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 2.3 MB/s | 1.5 MB 00:00 2024-05-10T04:29:05.505 INFO:teuthology.orchestra.run.smithi150.stdout:(6/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 3.9 MB/s | 5.1 MB 00:01 2024-05-10T04:29:05.620 INFO:teuthology.orchestra.run.smithi150.stdout:(7/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 879 kB/s | 2.2 MB 00:02 2024-05-10T04:29:06.231 INFO:teuthology.orchestra.run.smithi161.stdout:(4/146): ceph-mds-18.2.2-1330.g50174015.el8.x86 860 kB/s | 2.2 MB 00:02 2024-05-10T04:29:06.381 INFO:teuthology.orchestra.run.smithi181.stdout:(7/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 2.8 MB/s | 5.1 MB 00:01 2024-05-10T04:29:06.673 INFO:teuthology.orchestra.run.smithi161.stdout:(5/146): ceph-base-18.2.2-1330.g50174015.el8.x8 1.1 MB/s | 5.7 MB 00:05 2024-05-10T04:29:07.540 INFO:teuthology.orchestra.run.smithi161.stdout:(6/146): ceph-mgr-18.2.2-1330.g50174015.el8.x86 1.1 MB/s | 1.5 MB 00:01 2024-05-10T04:29:09.640 INFO:teuthology.orchestra.run.smithi161.stdout:(7/146): ceph-mon-18.2.2-1330.g50174015.el8.x86 1.7 MB/s | 5.1 MB 00:02 2024-05-10T04:29:09.878 INFO:teuthology.orchestra.run.smithi150.stdout:(8/146): ceph-osd-18.2.2-1330.g50174015.el8.x86 4.3 MB/s | 19 MB 00:04 2024-05-10T04:29:09.987 INFO:teuthology.orchestra.run.smithi150.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 225 kB/s | 24 kB 00:00 2024-05-10T04:29:10.447 INFO:teuthology.orchestra.run.smithi181.stdout:(8/146): ceph-osd-18.2.2-1330.g50174015.el8.x86 3.3 MB/s | 19 MB 00:05 2024-05-10T04:29:10.555 INFO:teuthology.orchestra.run.smithi181.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 225 kB/s | 24 kB 00:00 2024-05-10T04:29:11.254 INFO:teuthology.orchestra.run.smithi150.stdout:(10/146): ceph-common-18.2.2-1330.g50174015.el8 2.3 MB/s | 23 MB 00:10 2024-05-10T04:29:11.370 INFO:teuthology.orchestra.run.smithi150.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 2.1 MB/s | 246 kB 00:00 2024-05-10T04:29:11.429 INFO:teuthology.orchestra.run.smithi150.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 531 kB/s | 31 kB 00:00 2024-05-10T04:29:11.464 INFO:teuthology.orchestra.run.smithi181.stdout:(10/146): ceph-radosgw-18.2.2-1330.g50174015.el 2.8 MB/s | 14 MB 00:05 2024-05-10T04:29:11.614 INFO:teuthology.orchestra.run.smithi181.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 1.6 MB/s | 246 kB 00:00 2024-05-10T04:29:11.654 INFO:teuthology.orchestra.run.smithi150.stdout:(13/146): libcephfs2-18.2.2-1330.g50174015.el8. 3.3 MB/s | 756 kB 00:00 2024-05-10T04:29:11.681 INFO:teuthology.orchestra.run.smithi181.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 465 kB/s | 31 kB 00:00 2024-05-10T04:29:11.738 INFO:teuthology.orchestra.run.smithi150.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 1.6 MB/s | 139 kB 00:00 2024-05-10T04:29:11.867 INFO:teuthology.orchestra.run.smithi150.stdout:(15/146): librados-devel-18.2.2-1330.g50174015. 1.1 MB/s | 139 kB 00:00 2024-05-10T04:29:12.080 INFO:teuthology.orchestra.run.smithi150.stdout:(16/146): libradosstriper1-18.2.2-1330.g5017401 2.1 MB/s | 461 kB 00:00 2024-05-10T04:29:12.123 INFO:teuthology.orchestra.run.smithi181.stdout:(13/146): libcephfs2-18.2.2-1330.g50174015.el8. 1.7 MB/s | 756 kB 00:00 2024-05-10T04:29:12.290 INFO:teuthology.orchestra.run.smithi181.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 835 kB/s | 139 kB 00:00 2024-05-10T04:29:12.482 INFO:teuthology.orchestra.run.smithi181.stdout:(15/146): librados-devel-18.2.2-1330.g50174015. 725 kB/s | 139 kB 00:00 2024-05-10T04:29:12.774 INFO:teuthology.orchestra.run.smithi181.stdout:(16/146): libradosstriper1-18.2.2-1330.g5017401 1.5 MB/s | 461 kB 00:00 2024-05-10T04:29:14.265 INFO:teuthology.orchestra.run.smithi161.stdout:(8/146): ceph-radosgw-18.2.2-1330.g50174015.el8 3.1 MB/s | 14 MB 00:04 2024-05-10T04:29:14.305 INFO:teuthology.orchestra.run.smithi150.stdout:(17/146): librgw2-18.2.2-1330.g50174015.el8.x86 3.3 MB/s | 7.2 MB 00:02 2024-05-10T04:29:14.315 INFO:teuthology.orchestra.run.smithi161.stdout:(9/146): ceph-selinux-18.2.2-1330.g50174015.el8 491 kB/s | 24 kB 00:00 2024-05-10T04:29:14.472 INFO:teuthology.orchestra.run.smithi150.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 268 kB/s | 45 kB 00:00 2024-05-10T04:29:14.490 INFO:teuthology.orchestra.run.smithi161.stdout:(10/146): ceph-osd-18.2.2-1330.g50174015.el8.x8 2.7 MB/s | 19 MB 00:06 2024-05-10T04:29:14.522 INFO:teuthology.orchestra.run.smithi150.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 2.5 MB/s | 129 kB 00:00 2024-05-10T04:29:14.566 INFO:teuthology.orchestra.run.smithi161.stdout:(11/146): cephfs-mirror-18.2.2-1330.g50174015.e 3.2 MB/s | 246 kB 00:00 2024-05-10T04:29:14.606 INFO:teuthology.orchestra.run.smithi150.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 2.3 MB/s | 197 kB 00:00 2024-05-10T04:29:14.632 INFO:teuthology.orchestra.run.smithi161.stdout:(12/146): libcephfs-devel-18.2.2-1330.g50174015 465 kB/s | 31 kB 00:00 2024-05-10T04:29:14.765 INFO:teuthology.orchestra.run.smithi150.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 2.3 MB/s | 371 kB 00:00 2024-05-10T04:29:14.791 INFO:teuthology.orchestra.run.smithi161.stdout:(13/146): libcephfs2-18.2.2-1330.g50174015.el8. 4.7 MB/s | 756 kB 00:00 2024-05-10T04:29:14.890 INFO:teuthology.orchestra.run.smithi150.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 2.9 MB/s | 364 kB 00:00 2024-05-10T04:29:14.917 INFO:teuthology.orchestra.run.smithi161.stdout:(14/146): libcephsqlite-18.2.2-1330.g50174015.e 1.1 MB/s | 139 kB 00:00 2024-05-10T04:29:14.942 INFO:teuthology.orchestra.run.smithi150.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 2.1 MB/s | 107 kB 00:00 2024-05-10T04:29:14.975 INFO:teuthology.orchestra.run.smithi161.stdout:(15/146): librados-devel-18.2.2-1330.g50174015. 2.3 MB/s | 139 kB 00:00 2024-05-10T04:29:15.032 INFO:teuthology.orchestra.run.smithi150.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 1.0 MB/s | 91 kB 00:00 2024-05-10T04:29:15.100 INFO:teuthology.orchestra.run.smithi161.stdout:(16/146): libradosstriper1-18.2.2-1330.g5017401 3.6 MB/s | 461 kB 00:00 2024-05-10T04:29:15.583 INFO:teuthology.orchestra.run.smithi181.stdout:(17/146): librgw2-18.2.2-1330.g50174015.el8.x86 2.6 MB/s | 7.2 MB 00:02 2024-05-10T04:29:15.691 INFO:teuthology.orchestra.run.smithi181.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 413 kB/s | 45 kB 00:00 2024-05-10T04:29:15.758 INFO:teuthology.orchestra.run.smithi150.stdout:(25/146): rbd-mirror-18.2.2-1330.g50174015.el8. 4.6 MB/s | 3.3 MB 00:00 2024-05-10T04:29:15.800 INFO:teuthology.orchestra.run.smithi181.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 1.2 MB/s | 129 kB 00:00 2024-05-10T04:29:15.858 INFO:teuthology.orchestra.run.smithi150.stdout:(26/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 1.8 MB/s | 179 kB 00:00 2024-05-10T04:29:15.966 INFO:teuthology.orchestra.run.smithi150.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1330.g 224 kB/s | 24 kB 00:00 2024-05-10T04:29:15.983 INFO:teuthology.orchestra.run.smithi181.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 1.1 MB/s | 197 kB 00:00 2024-05-10T04:29:16.058 INFO:teuthology.orchestra.run.smithi150.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 1.5 MB/s | 138 kB 00:00 2024-05-10T04:29:16.200 INFO:teuthology.orchestra.run.smithi181.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 1.7 MB/s | 371 kB 00:00 2024-05-10T04:29:16.201 INFO:teuthology.orchestra.run.smithi161.stdout:(17/146): librgw2-18.2.2-1330.g50174015.el8.x86 6.6 MB/s | 7.2 MB 00:01 2024-05-10T04:29:16.201 INFO:teuthology.orchestra.run.smithi150.stdout:(29/146): ceph-radosgw-18.2.2-1330.g50174015.el 1.4 MB/s | 14 MB 00:10 2024-05-10T04:29:16.309 INFO:teuthology.orchestra.run.smithi161.stdout:(18/146): python3-ceph-argparse-18.2.2-1330.g50 413 kB/s | 45 kB 00:00 2024-05-10T04:29:16.386 INFO:teuthology.orchestra.run.smithi181.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 1.9 MB/s | 364 kB 00:00 2024-05-10T04:29:16.409 INFO:teuthology.orchestra.run.smithi161.stdout:(19/146): python3-ceph-common-18.2.2-1330.g5017 1.3 MB/s | 129 kB 00:00 2024-05-10T04:29:16.476 INFO:teuthology.orchestra.run.smithi181.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 1.1 MB/s | 107 kB 00:00 2024-05-10T04:29:16.501 INFO:teuthology.orchestra.run.smithi161.stdout:(20/146): python3-cephfs-18.2.2-1330.g50174015. 2.1 MB/s | 197 kB 00:00 2024-05-10T04:29:16.576 INFO:teuthology.orchestra.run.smithi181.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 916 kB/s | 91 kB 00:00 2024-05-10T04:29:16.602 INFO:teuthology.orchestra.run.smithi161.stdout:(21/146): python3-rados-18.2.2-1330.g50174015.e 3.6 MB/s | 371 kB 00:00 2024-05-10T04:29:16.710 INFO:teuthology.orchestra.run.smithi161.stdout:(22/146): python3-rbd-18.2.2-1330.g50174015.el8 3.3 MB/s | 364 kB 00:00 2024-05-10T04:29:16.756 INFO:teuthology.orchestra.run.smithi150.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1330.g50174 5.1 MB/s | 3.5 MB 00:00 2024-05-10T04:29:16.778 INFO:teuthology.orchestra.run.smithi161.stdout:(23/146): python3-rgw-18.2.2-1330.g50174015.el8 1.6 MB/s | 107 kB 00:00 2024-05-10T04:29:16.834 INFO:teuthology.orchestra.run.smithi150.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 3.1 MB/s | 246 kB 00:00 2024-05-10T04:29:16.852 INFO:teuthology.orchestra.run.smithi161.stdout:(24/146): rbd-fuse-18.2.2-1330.g50174015.el8.x8 1.2 MB/s | 91 kB 00:00 2024-05-10T04:29:16.885 INFO:teuthology.orchestra.run.smithi150.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 982 kB/s | 49 kB 00:00 2024-05-10T04:29:16.968 INFO:teuthology.orchestra.run.smithi150.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 196 kB/s | 16 kB 00:00 2024-05-10T04:29:17.068 INFO:teuthology.orchestra.run.smithi150.stdout:(34/146): ceph-volume-18.2.2-1330.g50174015.el8 2.6 MB/s | 263 kB 00:00 2024-05-10T04:29:17.177 INFO:teuthology.orchestra.run.smithi150.stdout:(35/146): cephadm-18.2.2-1330.g50174015.el8.noa 1.6 MB/s | 173 kB 00:00 2024-05-10T04:29:17.270 INFO:teuthology.orchestra.run.smithi150.stdout:(36/146): cephfs-top-18.2.2-1330.g50174015.el8. 198 kB/s | 18 kB 00:00 2024-05-10T04:29:17.303 INFO:teuthology.orchestra.run.smithi161.stdout:(25/146): rbd-mirror-18.2.2-1330.g50174015.el8. 7.4 MB/s | 3.3 MB 00:00 2024-05-10T04:29:17.378 INFO:teuthology.orchestra.run.smithi161.stdout:(26/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 2.3 MB/s | 179 kB 00:00 2024-05-10T04:29:17.453 INFO:teuthology.orchestra.run.smithi161.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1330.g 324 kB/s | 24 kB 00:00 2024-05-10T04:29:17.528 INFO:teuthology.orchestra.run.smithi161.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 1.8 MB/s | 138 kB 00:00 2024-05-10T04:29:17.692 INFO:teuthology.orchestra.run.smithi181.stdout:(25/146): rbd-mirror-18.2.2-1330.g50174015.el8. 3.0 MB/s | 3.3 MB 00:01 2024-05-10T04:29:17.911 INFO:teuthology.orchestra.run.smithi150.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 477 kB/s | 306 kB 00:00 2024-05-10T04:29:18.028 INFO:teuthology.orchestra.run.smithi150.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 504 kB/s | 59 kB 00:00 2024-05-10T04:29:18.079 INFO:teuthology.orchestra.run.smithi161.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1330.g50174 6.4 MB/s | 3.5 MB 00:00 2024-05-10T04:29:18.145 INFO:teuthology.orchestra.run.smithi150.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 644 kB/s | 75 kB 00:00 2024-05-10T04:29:18.354 INFO:teuthology.orchestra.run.smithi161.stdout:(30/146): ceph-common-18.2.2-1330.g50174015.el8 1.4 MB/s | 23 MB 00:16 2024-05-10T04:29:18.370 INFO:teuthology.orchestra.run.smithi150.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2024-05-10T04:29:18.394 INFO:teuthology.orchestra.run.smithi181.stdout:(26/146): ceph-common-18.2.2-1330.g50174015.el8 1.4 MB/s | 23 MB 00:16 2024-05-10T04:29:18.419 INFO:teuthology.orchestra.run.smithi181.stdout:(27/146): rbd-nbd-18.2.2-1330.g50174015.el8.x86 247 kB/s | 179 kB 00:00 2024-05-10T04:29:18.479 INFO:teuthology.orchestra.run.smithi150.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 402 kB/s | 44 kB 00:00 2024-05-10T04:29:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:(28/146): ceph-grafana-dashboards-18.2.2-1330.g 265 kB/s | 24 kB 00:00 2024-05-10T04:29:18.563 INFO:teuthology.orchestra.run.smithi161.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 1.2 MB/s | 246 kB 00:00 2024-05-10T04:29:18.569 INFO:teuthology.orchestra.run.smithi181.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1330.g5017401 923 kB/s | 138 kB 00:00 2024-05-10T04:29:18.613 INFO:teuthology.orchestra.run.smithi150.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2024-05-10T04:29:18.663 INFO:teuthology.orchestra.run.smithi161.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 493 kB/s | 49 kB 00:00 2024-05-10T04:29:18.721 INFO:teuthology.orchestra.run.smithi150.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2024-05-10T04:29:18.763 INFO:teuthology.orchestra.run.smithi161.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 164 kB/s | 16 kB 00:00 2024-05-10T04:29:18.830 INFO:teuthology.orchestra.run.smithi150.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2024-05-10T04:29:18.947 INFO:teuthology.orchestra.run.smithi150.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2024-05-10T04:29:19.005 INFO:teuthology.orchestra.run.smithi161.stdout:(34/146): ceph-volume-18.2.2-1330.g50174015.el8 1.1 MB/s | 263 kB 00:00 2024-05-10T04:29:19.089 INFO:teuthology.orchestra.run.smithi150.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.7 MB/s | 384 kB 00:00 2024-05-10T04:29:19.172 INFO:teuthology.orchestra.run.smithi161.stdout:(35/146): ceph-mgr-diskprediction-local-18.2.2- 6.8 MB/s | 7.4 MB 00:01 2024-05-10T04:29:19.197 INFO:teuthology.orchestra.run.smithi150.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 750 kB/s | 81 kB 00:00 2024-05-10T04:29:19.272 INFO:teuthology.orchestra.run.smithi161.stdout:(36/146): cephadm-18.2.2-1330.g50174015.el8.noa 646 kB/s | 173 kB 00:00 2024-05-10T04:29:19.298 INFO:teuthology.orchestra.run.smithi161.stdout:(37/146): cephfs-top-18.2.2-1330.g50174015.el8. 145 kB/s | 18 kB 00:00 2024-05-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi150.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-05-10T04:29:19.423 INFO:teuthology.orchestra.run.smithi150.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-05-10T04:29:19.431 INFO:teuthology.orchestra.run.smithi161.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 442 kB/s | 59 kB 00:00 2024-05-10T04:29:19.490 INFO:teuthology.orchestra.run.smithi161.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2024-05-10T04:29:19.515 INFO:teuthology.orchestra.run.smithi161.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 899 kB/s | 75 kB 00:00 2024-05-10T04:29:19.574 INFO:teuthology.orchestra.run.smithi161.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 747 kB/s | 44 kB 00:00 2024-05-10T04:29:19.599 INFO:teuthology.orchestra.run.smithi161.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2024-05-10T04:29:19.615 INFO:teuthology.orchestra.run.smithi150.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2024-05-10T04:29:19.641 INFO:teuthology.orchestra.run.smithi161.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 795 kB/s | 33 kB 00:00 2024-05-10T04:29:19.658 INFO:teuthology.orchestra.run.smithi161.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2024-05-10T04:29:19.692 INFO:teuthology.orchestra.run.smithi161.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2024-05-10T04:29:19.709 INFO:teuthology.orchestra.run.smithi161.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2024-05-10T04:29:19.723 INFO:teuthology.orchestra.run.smithi150.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 451 kB/s | 49 kB 00:00 2024-05-10T04:29:19.751 INFO:teuthology.orchestra.run.smithi161.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2024-05-10T04:29:19.776 INFO:teuthology.orchestra.run.smithi161.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2024-05-10T04:29:19.793 INFO:teuthology.orchestra.run.smithi161.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2024-05-10T04:29:19.818 INFO:teuthology.orchestra.run.smithi161.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 421 kB/s | 18 kB 00:00 2024-05-10T04:29:19.832 INFO:teuthology.orchestra.run.smithi150.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 543 kB/s | 59 kB 00:00 2024-05-10T04:29:19.860 INFO:teuthology.orchestra.run.smithi161.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 1.2 MB/s | 49 kB 00:00 2024-05-10T04:29:19.910 INFO:teuthology.orchestra.run.smithi161.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 10 MB/s | 1.2 MB 00:00 2024-05-10T04:29:19.927 INFO:teuthology.orchestra.run.smithi161.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 877 kB/s | 59 kB 00:00 2024-05-10T04:29:19.953 INFO:teuthology.orchestra.run.smithi161.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2024-05-10T04:29:19.966 INFO:teuthology.orchestra.run.smithi150.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 452 kB/s | 60 kB 00:00 2024-05-10T04:29:19.986 INFO:teuthology.orchestra.run.smithi161.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2024-05-10T04:29:20.003 INFO:teuthology.orchestra.run.smithi161.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2024-05-10T04:29:20.029 INFO:teuthology.orchestra.run.smithi161.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 803 kB/s | 34 kB 00:00 2024-05-10T04:29:20.036 INFO:teuthology.orchestra.run.smithi181.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1330.g50174 2.3 MB/s | 3.5 MB 00:01 2024-05-10T04:29:20.054 INFO:teuthology.orchestra.run.smithi161.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2024-05-10T04:29:20.071 INFO:teuthology.orchestra.run.smithi161.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2024-05-10T04:29:20.108 INFO:teuthology.orchestra.run.smithi150.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.0 MB/s | 283 kB 00:00 2024-05-10T04:29:20.132 INFO:teuthology.orchestra.run.smithi161.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 240 kB/s | 19 kB 00:00 2024-05-10T04:29:20.163 INFO:teuthology.orchestra.run.smithi161.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2024-05-10T04:29:20.170 INFO:teuthology.orchestra.run.smithi181.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1330.g50 1.8 MB/s | 246 kB 00:00 2024-05-10T04:29:20.188 INFO:teuthology.orchestra.run.smithi161.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 527 kB/s | 29 kB 00:00 2024-05-10T04:29:20.213 INFO:teuthology.orchestra.run.smithi161.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2024-05-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi150.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 134 kB/s | 16 kB 00:00 2024-05-10T04:29:20.239 INFO:teuthology.orchestra.run.smithi161.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2024-05-10T04:29:20.264 INFO:teuthology.orchestra.run.smithi161.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2024-05-10T04:29:20.278 INFO:teuthology.orchestra.run.smithi181.stdout:(32/146): ceph-mgr-rook-18.2.2-1330.g50174015.e 455 kB/s | 49 kB 00:00 2024-05-10T04:29:20.289 INFO:teuthology.orchestra.run.smithi161.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2024-05-10T04:29:20.315 INFO:teuthology.orchestra.run.smithi161.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2024-05-10T04:29:20.340 INFO:teuthology.orchestra.run.smithi161.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2024-05-10T04:29:20.357 INFO:teuthology.orchestra.run.smithi161.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-05-10T04:29:20.370 INFO:teuthology.orchestra.run.smithi181.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1330.g5 178 kB/s | 16 kB 00:00 2024-05-10T04:29:20.383 INFO:teuthology.orchestra.run.smithi150.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 212 kB/s | 34 kB 00:00 2024-05-10T04:29:20.424 INFO:teuthology.orchestra.run.smithi161.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2024-05-10T04:29:20.491 INFO:teuthology.orchestra.run.smithi161.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 134 kB/s | 18 kB 00:00 2024-05-10T04:29:20.516 INFO:teuthology.orchestra.run.smithi161.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 464 kB/s | 43 kB 00:00 2024-05-10T04:29:20.529 INFO:teuthology.orchestra.run.smithi181.stdout:(34/146): ceph-mgr-diskprediction-local-18.2.2- 3.8 MB/s | 7.4 MB 00:01 2024-05-10T04:29:20.534 INFO:teuthology.orchestra.run.smithi150.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2024-05-10T04:29:20.554 INFO:teuthology.orchestra.run.smithi181.stdout:(35/146): ceph-volume-18.2.2-1330.g50174015.el8 1.4 MB/s | 263 kB 00:00 2024-05-10T04:29:20.596 INFO:teuthology.orchestra.run.smithi181.stdout:(36/146): cephadm-18.2.2-1330.g50174015.el8.noa 2.5 MB/s | 173 kB 00:00 2024-05-10T04:29:20.638 INFO:teuthology.orchestra.run.smithi181.stdout:(37/146): cephfs-top-18.2.2-1330.g50174015.el8. 216 kB/s | 18 kB 00:00 2024-05-10T04:29:20.659 INFO:teuthology.orchestra.run.smithi150.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 514 kB/s | 64 kB 00:00 2024-05-10T04:29:20.793 INFO:teuthology.orchestra.run.smithi150.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2024-05-10T04:29:20.893 INFO:teuthology.orchestra.run.smithi150.stdout:(60/146): python3-singledispatch-3.4.0.3-18.el8 240 kB/s | 24 kB 00:00 2024-05-10T04:29:20.922 INFO:teuthology.orchestra.run.smithi181.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 208 kB/s | 59 kB 00:00 2024-05-10T04:29:20.958 INFO:teuthology.orchestra.run.smithi161.stdout:(73/146): ceph-test-18.2.2-1330.g50174015.el8.x 7.4 MB/s | 49 MB 00:06 2024-05-10T04:29:20.984 INFO:teuthology.orchestra.run.smithi161.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 234 kB/s | 109 kB 00:00 2024-05-10T04:29:21.014 INFO:teuthology.orchestra.run.smithi181.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 734 kB/s | 306 kB 00:00 2024-05-10T04:29:21.017 INFO:teuthology.orchestra.run.smithi161.stdout:(75/146): python3-oauthlib-2.1.0-1.el8.noarch.r 295 kB/s | 155 kB 00:00 2024-05-10T04:29:21.027 INFO:teuthology.orchestra.run.smithi150.stdout:(61/146): ceph-mgr-diskprediction-local-18.2.2- 1.5 MB/s | 7.4 MB 00:04 2024-05-10T04:29:21.039 INFO:teuthology.orchestra.run.smithi181.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 642 kB/s | 75 kB 00:00 2024-05-10T04:29:21.052 INFO:teuthology.orchestra.run.smithi150.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 184 kB/s | 29 kB 00:00 2024-05-10T04:29:21.126 INFO:teuthology.orchestra.run.smithi161.stdout:(76/146): python3-requests-oauthlib-1.0.0-1.el8 256 kB/s | 43 kB 00:00 2024-05-10T04:29:21.131 INFO:teuthology.orchestra.run.smithi181.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 475 kB/s | 44 kB 00:00 2024-05-10T04:29:21.151 INFO:teuthology.orchestra.run.smithi161.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 116 kB/s | 19 kB 00:00 2024-05-10T04:29:21.156 INFO:teuthology.orchestra.run.smithi181.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2024-05-10T04:29:21.176 INFO:teuthology.orchestra.run.smithi161.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 276 kB/s | 44 kB 00:00 2024-05-10T04:29:21.177 INFO:teuthology.orchestra.run.smithi150.stdout:(63/146): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2024-05-10T04:29:21.218 INFO:teuthology.orchestra.run.smithi161.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2024-05-10T04:29:21.232 INFO:teuthology.orchestra.run.smithi181.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 439 kB/s | 33 kB 00:00 2024-05-10T04:29:21.277 INFO:teuthology.orchestra.run.smithi161.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2024-05-10T04:29:21.294 INFO:teuthology.orchestra.run.smithi161.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-05-10T04:29:21.299 INFO:teuthology.orchestra.run.smithi181.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2024-05-10T04:29:21.302 INFO:teuthology.orchestra.run.smithi150.stdout:(64/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2024-05-10T04:29:21.316 INFO:teuthology.orchestra.run.smithi181.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2024-05-10T04:29:21.344 INFO:teuthology.orchestra.run.smithi161.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.6 MB/s | 200 kB 00:00 2024-05-10T04:29:21.399 INFO:teuthology.orchestra.run.smithi181.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2024-05-10T04:29:21.403 INFO:teuthology.orchestra.run.smithi150.stdout:(65/146): python3-trustme-0.6.0-4.el8.noarch.rp 73 kB/s | 27 kB 00:00 2024-05-10T04:29:21.420 INFO:teuthology.orchestra.run.smithi150.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 524 kB/s | 61 kB 00:00 2024-05-10T04:29:21.424 INFO:teuthology.orchestra.run.smithi181.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2024-05-10T04:29:21.427 INFO:teuthology.orchestra.run.smithi161.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 320 kB/s | 48 kB 00:00 2024-05-10T04:29:21.453 INFO:teuthology.orchestra.run.smithi161.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 548 kB/s | 87 kB 00:00 2024-05-10T04:29:21.495 INFO:teuthology.orchestra.run.smithi161.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 383 kB/s | 58 kB 00:00 2024-05-10T04:29:21.520 INFO:teuthology.orchestra.run.smithi150.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 230 kB/s | 23 kB 00:00 2024-05-10T04:29:21.562 INFO:teuthology.orchestra.run.smithi161.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 765 kB/s | 83 kB 00:00 2024-05-10T04:29:21.587 INFO:teuthology.orchestra.run.smithi161.stdout:(87/146): libcom_err-devel-1.45.6-4.el8.x86_64. 243 kB/s | 39 kB 00:00 2024-05-10T04:29:21.654 INFO:teuthology.orchestra.run.smithi150.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 341 kB/s | 85 kB 00:00 2024-05-10T04:29:21.679 INFO:teuthology.orchestra.run.smithi150.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2024-05-10T04:29:21.679 INFO:teuthology.orchestra.run.smithi161.stdout:(88/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2024-05-10T04:29:21.721 INFO:teuthology.orchestra.run.smithi150.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 8.6 MB/s | 1.7 MB 00:00 2024-05-10T04:29:21.754 INFO:teuthology.orchestra.run.smithi161.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.4 MB/s | 645 kB 00:00 2024-05-10T04:29:21.771 INFO:teuthology.orchestra.run.smithi161.stdout:(90/146): libquadmath-8.5.0-20.el8.x86_64.rpm 825 kB/s | 172 kB 00:00 2024-05-10T04:29:21.830 INFO:teuthology.orchestra.run.smithi161.stdout:(91/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 483 kB/s | 73 kB 00:00 2024-05-10T04:29:21.846 INFO:teuthology.orchestra.run.smithi150.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 167 kB/s | 20 kB 00:00 2024-05-10T04:29:21.892 INFO:teuthology.orchestra.run.smithi181.stdout:(48/146): ceph-test-18.2.2-1330.g50174015.el8.x 4.3 MB/s | 49 MB 00:11 2024-05-10T04:29:21.898 INFO:teuthology.orchestra.run.smithi150.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 222 kB/s | 48 kB 00:00 2024-05-10T04:29:21.908 INFO:teuthology.orchestra.run.smithi181.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 23 kB/s | 11 kB 00:00 2024-05-10T04:29:21.934 INFO:teuthology.orchestra.run.smithi181.stdout:(50/146): python3-google-auth-1.1.1-10.el8.noar 152 kB/s | 81 kB 00:00 2024-05-10T04:29:21.959 INFO:teuthology.orchestra.run.smithi181.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2024-05-10T04:29:21.988 INFO:teuthology.orchestra.run.smithi161.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 264 kB/s | 62 kB 00:00 2024-05-10T04:29:21.989 INFO:teuthology.orchestra.run.smithi150.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 309 kB/s | 44 kB 00:00 2024-05-10T04:29:22.010 INFO:teuthology.orchestra.run.smithi181.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 648 kB/s | 49 kB 00:00 2024-05-10T04:29:22.013 INFO:teuthology.orchestra.run.smithi150.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 941 kB/s | 110 kB 00:00 2024-05-10T04:29:22.035 INFO:teuthology.orchestra.run.smithi181.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 830 kB/s | 59 kB 00:00 2024-05-10T04:29:22.063 INFO:teuthology.orchestra.run.smithi161.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 1.9 MB/s | 562 kB 00:00 2024-05-10T04:29:22.085 INFO:teuthology.orchestra.run.smithi181.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 801 kB/s | 60 kB 00:00 2024-05-10T04:29:22.089 INFO:teuthology.orchestra.run.smithi161.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 469 kB/s | 47 kB 00:00 2024-05-10T04:29:22.127 INFO:teuthology.orchestra.run.smithi181.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2024-05-10T04:29:22.130 INFO:teuthology.orchestra.run.smithi150.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 187 kB/s | 27 kB 00:00 2024-05-10T04:29:22.164 INFO:teuthology.orchestra.run.smithi150.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 841 kB/s | 126 kB 00:00 2024-05-10T04:29:22.186 INFO:teuthology.orchestra.run.smithi181.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2024-05-10T04:29:22.197 INFO:teuthology.orchestra.run.smithi161.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 6.3 MB/s | 2.3 MB 00:00 2024-05-10T04:29:22.202 INFO:teuthology.orchestra.run.smithi181.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 447 kB/s | 34 kB 00:00 2024-05-10T04:29:22.231 INFO:teuthology.orchestra.run.smithi161.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 3.3 MB/s | 559 kB 00:00 2024-05-10T04:29:22.269 INFO:teuthology.orchestra.run.smithi181.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 965 kB/s | 64 kB 00:00 2024-05-10T04:29:22.286 INFO:teuthology.orchestra.run.smithi181.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-05-10T04:29:22.314 INFO:teuthology.orchestra.run.smithi161.stdout:(97/146): NetworkManager-initscripts-updown-1.4 640 kB/s | 144 kB 00:00 2024-05-10T04:29:22.345 INFO:teuthology.orchestra.run.smithi181.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2024-05-10T04:29:22.356 INFO:teuthology.orchestra.run.smithi161.stdout:(98/146): python3-packaging-16.8-9.el8.noarch.r 385 kB/s | 48 kB 00:00 2024-05-10T04:29:22.370 INFO:teuthology.orchestra.run.smithi181.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2024-05-10T04:29:22.381 INFO:teuthology.orchestra.run.smithi150.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 3.1 MB/s | 688 kB 00:00 2024-05-10T04:29:22.412 INFO:teuthology.orchestra.run.smithi181.stdout:(62/146): python3-kubernetes-11.0.0-6.el8.noarc 2.3 MB/s | 1.2 MB 00:00 2024-05-10T04:29:22.415 INFO:teuthology.orchestra.run.smithi150.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 1.6 MB/s | 457 kB 00:00 2024-05-10T04:29:22.457 INFO:teuthology.orchestra.run.smithi161.stdout:(99/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 385 kB/s | 100 kB 00:00 2024-05-10T04:29:22.469 INFO:teuthology.orchestra.run.smithi181.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 237 kB/s | 29 kB 00:00 2024-05-10T04:29:22.496 INFO:teuthology.orchestra.run.smithi181.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2024-05-10T04:29:22.506 INFO:teuthology.orchestra.run.smithi161.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 105 kB/s | 20 kB 00:00 2024-05-10T04:29:22.506 INFO:teuthology.orchestra.run.smithi150.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2024-05-10T04:29:22.522 INFO:teuthology.orchestra.run.smithi181.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2024-05-10T04:29:22.532 INFO:teuthology.orchestra.run.smithi161.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 251 kB/s | 44 kB 00:00 2024-05-10T04:29:22.555 INFO:teuthology.orchestra.run.smithi181.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2024-05-10T04:29:22.557 INFO:teuthology.orchestra.run.smithi150.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2024-05-10T04:29:22.580 INFO:teuthology.orchestra.run.smithi181.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 730 kB/s | 61 kB 00:00 2024-05-10T04:29:22.606 INFO:teuthology.orchestra.run.smithi181.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2024-05-10T04:29:22.616 INFO:teuthology.orchestra.run.smithi161.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 693 kB/s | 110 kB 00:00 2024-05-10T04:29:22.631 INFO:teuthology.orchestra.run.smithi181.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2024-05-10T04:29:22.641 INFO:teuthology.orchestra.run.smithi161.stdout:(103/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.1 MB/s | 126 kB 00:00 2024-05-10T04:29:22.648 INFO:teuthology.orchestra.run.smithi181.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-05-10T04:29:22.666 INFO:teuthology.orchestra.run.smithi161.stdout:(104/146): python3-jsonpatch-1.21-2.el8.noarch. 166 kB/s | 27 kB 00:00 2024-05-10T04:29:22.674 INFO:teuthology.orchestra.run.smithi150.stdout:(81/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2024-05-10T04:29:22.699 INFO:teuthology.orchestra.run.smithi150.stdout:(82/146): python3-markupsafe-0.23-19.el8.x86_64 202 kB/s | 39 kB 00:00 2024-05-10T04:29:22.767 INFO:teuthology.orchestra.run.smithi161.stdout:(105/146): bison-3.0.4-10.el8.x86_64.rpm 5.4 MB/s | 688 kB 00:00 2024-05-10T04:29:22.800 INFO:teuthology.orchestra.run.smithi161.stdout:(106/146): python3-werkzeug-0.12.2-4.el8.noarch 2.4 MB/s | 457 kB 00:00 2024-05-10T04:29:22.815 INFO:teuthology.orchestra.run.smithi181.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 97 kB/s | 18 kB 00:00 2024-05-10T04:29:22.816 INFO:teuthology.orchestra.run.smithi150.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 451 kB/s | 64 kB 00:00 2024-05-10T04:29:22.848 INFO:teuthology.orchestra.run.smithi181.stdout:(72/146): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2024-05-10T04:29:22.874 INFO:teuthology.orchestra.run.smithi181.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 189 kB/s | 43 kB 00:00 2024-05-10T04:29:22.884 INFO:teuthology.orchestra.run.smithi161.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 1.4 MB/s | 320 kB 00:00 2024-05-10T04:29:22.942 INFO:teuthology.orchestra.run.smithi161.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 273 kB/s | 39 kB 00:00 2024-05-10T04:29:22.991 INFO:teuthology.orchestra.run.smithi181.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 883 kB/s | 155 kB 00:00 2024-05-10T04:29:22.993 INFO:teuthology.orchestra.run.smithi161.stdout:(109/146): libtool-2.4.6-25.el8.x86_64.rpm 3.1 MB/s | 709 kB 00:00 2024-05-10T04:29:23.000 INFO:teuthology.orchestra.run.smithi150.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 344 kB/s | 103 kB 00:00 2024-05-10T04:29:23.018 INFO:teuthology.orchestra.run.smithi161.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2024-05-10T04:29:23.074 INFO:teuthology.orchestra.run.smithi181.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 484 kB/s | 109 kB 00:00 2024-05-10T04:29:23.100 INFO:teuthology.orchestra.run.smithi181.stdout:(76/146): python3-requests-oauthlib-1.0.0-1.el8 194 kB/s | 43 kB 00:00 2024-05-10T04:29:23.118 INFO:teuthology.orchestra.run.smithi161.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 828 kB/s | 103 kB 00:00 2024-05-10T04:29:23.142 INFO:teuthology.orchestra.run.smithi181.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 124 kB/s | 19 kB 00:00 2024-05-10T04:29:23.143 INFO:teuthology.orchestra.run.smithi161.stdout:(112/146): uuid-1.6.2-43.el8.x86_64.rpm 319 kB/s | 64 kB 00:00 2024-05-10T04:29:23.250 INFO:teuthology.orchestra.run.smithi181.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 250 kB/s | 44 kB 00:00 2024-05-10T04:29:23.275 INFO:teuthology.orchestra.run.smithi161.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2024-05-10T04:29:23.276 INFO:teuthology.orchestra.run.smithi181.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2024-05-10T04:29:23.311 INFO:teuthology.orchestra.run.smithi161.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.7 MB/s | 538 kB 00:00 2024-05-10T04:29:23.359 INFO:teuthology.orchestra.run.smithi181.stdout:(80/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.3 MB/s | 605 kB 00:00 2024-05-10T04:29:23.403 INFO:teuthology.orchestra.run.smithi161.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2024-05-10T04:29:23.409 INFO:teuthology.orchestra.run.smithi181.stdout:(81/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-05-10T04:29:23.419 INFO:teuthology.orchestra.run.smithi161.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2024-05-10T04:29:23.435 INFO:teuthology.orchestra.run.smithi181.stdout:(82/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2024-05-10T04:29:23.442 INFO:teuthology.orchestra.run.smithi150.stdout:(85/146): ceph-test-18.2.2-1330.g50174015.el8.x 3.7 MB/s | 49 MB 00:13 2024-05-10T04:29:23.470 INFO:teuthology.orchestra.run.smithi161.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.4 MB/s | 225 kB 00:00 2024-05-10T04:29:23.493 INFO:teuthology.orchestra.run.smithi181.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 360 kB/s | 48 kB 00:00 2024-05-10T04:29:23.509 INFO:teuthology.orchestra.run.smithi150.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 6.9 MB/s | 4.8 MB 00:00 2024-05-10T04:29:23.511 INFO:teuthology.orchestra.run.smithi161.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2024-05-10T04:29:23.518 INFO:teuthology.orchestra.run.smithi181.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 691 kB/s | 58 kB 00:00 2024-05-10T04:29:23.544 INFO:teuthology.orchestra.run.smithi181.stdout:(85/146): libsepol-devel-2.9-3.el8.x86_64.rpm 649 kB/s | 87 kB 00:00 2024-05-10T04:29:23.576 INFO:teuthology.orchestra.run.smithi150.stdout:(87/146): python3-jinja2-2.10.1-3.el8.noarch.rp 935 kB/s | 538 kB 00:00 2024-05-10T04:29:23.645 INFO:teuthology.orchestra.run.smithi181.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 256 kB/s | 39 kB 00:00 2024-05-10T04:29:23.670 INFO:teuthology.orchestra.run.smithi161.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 21 MB/s | 5.2 MB 00:00 2024-05-10T04:29:23.704 INFO:teuthology.orchestra.run.smithi161.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 671 kB/s | 157 kB 00:00 2024-05-10T04:29:23.751 INFO:teuthology.orchestra.run.smithi150.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-05-10T04:29:23.776 INFO:teuthology.orchestra.run.smithi150.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.1 MB/s | 225 kB 00:00 2024-05-10T04:29:23.801 INFO:teuthology.orchestra.run.smithi150.stdout:(90/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.1 MB/s | 323 kB 00:00 2024-05-10T04:29:23.804 INFO:teuthology.orchestra.run.smithi161.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-05-10T04:29:23.819 INFO:teuthology.orchestra.run.smithi181.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 276 kB/s | 83 kB 00:00 2024-05-10T04:29:23.880 INFO:teuthology.orchestra.run.smithi161.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 3.3 MB/s | 714 kB 00:00 2024-05-10T04:29:23.936 INFO:teuthology.orchestra.run.smithi181.stdout:(88/146): libgfortran-8.5.0-20.el8.x86_64.rpm 1.6 MB/s | 645 kB 00:00 2024-05-10T04:29:23.963 INFO:teuthology.orchestra.run.smithi161.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 765 kB/s | 198 kB 00:00 2024-05-10T04:29:23.977 INFO:teuthology.orchestra.run.smithi150.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 896 kB/s | 157 kB 00:00 2024-05-10T04:29:23.986 INFO:teuthology.orchestra.run.smithi181.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 504 kB/s | 172 kB 00:00 2024-05-10T04:29:23.988 INFO:teuthology.orchestra.run.smithi161.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-05-10T04:29:24.028 INFO:teuthology.orchestra.run.smithi181.stdout:(90/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.1 MB/s | 238 kB 00:00 2024-05-10T04:29:24.037 INFO:teuthology.orchestra.run.smithi161.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 344 kB/s | 54 kB 00:00 2024-05-10T04:29:24.054 INFO:teuthology.orchestra.run.smithi181.stdout:(91/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 620 kB/s | 73 kB 00:00 2024-05-10T04:29:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 369 kB/s | 62 kB 00:00 2024-05-10T04:29:24.169 INFO:teuthology.orchestra.run.smithi150.stdout:(92/146): librdkafka-0.11.4-3.el8.x86_64.rpm 848 kB/s | 354 kB 00:00 2024-05-10T04:29:24.180 INFO:teuthology.orchestra.run.smithi161.stdout:(126/146): postgresql-10.23-4.module+el8.9.0+21 11 MB/s | 1.5 MB 00:00 2024-05-10T04:29:24.217 INFO:teuthology.orchestra.run.smithi181.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.9 MB/s | 562 kB 00:00 2024-05-10T04:29:24.244 INFO:teuthology.orchestra.run.smithi150.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 11 MB/s | 5.2 MB 00:00 2024-05-10T04:29:24.281 INFO:teuthology.orchestra.run.smithi161.stdout:(127/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 644 kB/s | 188 kB 00:00 2024-05-10T04:29:24.296 INFO:teuthology.orchestra.run.smithi181.stdout:(94/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 9.6 MB/s | 2.3 MB 00:00 2024-05-10T04:29:24.311 INFO:teuthology.orchestra.run.smithi150.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-05-10T04:29:24.321 INFO:teuthology.orchestra.run.smithi181.stdout:(95/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 282 kB/s | 47 kB 00:00 2024-05-10T04:29:24.370 INFO:teuthology.orchestra.run.smithi150.stdout:(95/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-05-10T04:29:24.403 INFO:teuthology.orchestra.run.smithi150.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-05-10T04:29:24.428 INFO:teuthology.orchestra.run.smithi150.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2024-05-10T04:29:24.464 INFO:teuthology.orchestra.run.smithi181.stdout:(96/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 705 kB/s | 100 kB 00:00 2024-05-10T04:29:24.521 INFO:teuthology.orchestra.run.smithi181.stdout:(97/146): python3-cryptography-3.2.1-7.el8_9.x8 1.8 MB/s | 559 kB 00:00 2024-05-10T04:29:24.529 INFO:teuthology.orchestra.run.smithi150.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 341 kB/s | 54 kB 00:00 2024-05-10T04:29:24.539 INFO:teuthology.orchestra.run.smithi161.stdout:(128/146): postgresql-contrib-10.23-4.module+el 2.2 MB/s | 812 kB 00:00 2024-05-10T04:29:24.547 INFO:teuthology.orchestra.run.smithi181.stdout:(98/146): NetworkManager-initscripts-updown-1.4 575 kB/s | 144 kB 00:00 2024-05-10T04:29:24.604 INFO:teuthology.orchestra.run.smithi150.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-10T04:29:24.606 INFO:teuthology.orchestra.run.smithi181.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 339 kB/s | 48 kB 00:00 2024-05-10T04:29:24.673 INFO:teuthology.orchestra.run.smithi181.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 131 kB/s | 20 kB 00:00 2024-05-10T04:29:24.698 INFO:teuthology.orchestra.run.smithi181.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 292 kB/s | 44 kB 00:00 2024-05-10T04:29:24.704 INFO:teuthology.orchestra.run.smithi150.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 8.7 MB/s | 1.5 MB 00:00 2024-05-10T04:29:24.714 INFO:teuthology.orchestra.run.smithi161.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 18 MB/s | 14 MB 00:00 2024-05-10T04:29:24.765 INFO:teuthology.orchestra.run.smithi181.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 693 kB/s | 110 kB 00:00 2024-05-10T04:29:24.823 INFO:teuthology.orchestra.run.smithi161.stdout:(130/146): postgresql-server-10.23-4.module+el8 9.3 MB/s | 5.1 MB 00:00 2024-05-10T04:29:24.849 INFO:teuthology.orchestra.run.smithi181.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 151 kB/s | 27 kB 00:00 2024-05-10T04:29:24.874 INFO:teuthology.orchestra.run.smithi181.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 720 kB/s | 126 kB 00:00 2024-05-10T04:29:24.908 INFO:teuthology.orchestra.run.smithi181.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 3.1 MB/s | 457 kB 00:00 2024-05-10T04:29:24.957 INFO:teuthology.orchestra.run.smithi161.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2024-05-10T04:29:25.016 INFO:teuthology.orchestra.run.smithi181.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 4.0 MB/s | 688 kB 00:00 2024-05-10T04:29:25.023 INFO:teuthology.orchestra.run.smithi161.stdout:(132/146): librados2-18.2.2-1330.g50174015.el8. 7.3 MB/s | 3.5 MB 00:00 2024-05-10T04:29:25.050 INFO:teuthology.orchestra.run.smithi181.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2024-05-10T04:29:25.120 INFO:teuthology.orchestra.run.smithi181.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 3.3 MB/s | 709 kB 00:00 2024-05-10T04:29:25.151 INFO:teuthology.orchestra.run.smithi181.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 289 kB/s | 39 kB 00:00 2024-05-10T04:29:25.154 INFO:teuthology.orchestra.run.smithi150.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 18 MB/s | 14 MB 00:00 2024-05-10T04:29:25.157 INFO:teuthology.orchestra.run.smithi161.stdout:(133/146): librbd1-18.2.2-1330.g50174015.el8.x8 8.0 MB/s | 3.5 MB 00:00 2024-05-10T04:29:25.183 INFO:teuthology.orchestra.run.smithi161.stdout:(134/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.3 MB/s | 303 kB 00:00 2024-05-10T04:29:25.201 INFO:teuthology.orchestra.run.smithi181.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2024-05-10T04:29:25.205 INFO:teuthology.orchestra.run.smithi150.stdout:(102/146): postgresql-contrib-10.23-4.module+el 1.3 MB/s | 812 kB 00:00 2024-05-10T04:29:25.243 INFO:teuthology.orchestra.run.smithi181.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 520 kB/s | 64 kB 00:00 2024-05-10T04:29:25.250 INFO:teuthology.orchestra.run.smithi161.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.7 MB/s | 854 kB 00:00 2024-05-10T04:29:25.285 INFO:teuthology.orchestra.run.smithi181.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 774 kB/s | 103 kB 00:00 2024-05-10T04:29:25.308 INFO:teuthology.orchestra.run.smithi161.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.6 MB/s | 711 kB 00:00 2024-05-10T04:29:25.347 INFO:teuthology.orchestra.run.smithi150.stdout:(103/146): postgresql-server-10.23-4.module+el8 7.9 MB/s | 5.1 MB 00:00 2024-05-10T04:29:25.358 INFO:teuthology.orchestra.run.smithi161.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-05-10T04:29:25.364 INFO:teuthology.orchestra.run.smithi150.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 269 kB/s | 43 kB 00:00 2024-05-10T04:29:25.414 INFO:teuthology.orchestra.run.smithi150.stdout:(105/146): libverto-devel-0.3.0-5.el8.x86_64.rp 69 kB/s | 18 kB 00:00 2024-05-10T04:29:25.425 INFO:teuthology.orchestra.run.smithi161.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-05-10T04:29:25.452 INFO:teuthology.orchestra.run.smithi181.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2024-05-10T04:29:25.467 INFO:teuthology.orchestra.run.smithi161.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.7 MB/s | 1.1 MB 00:00 2024-05-10T04:29:25.485 INFO:teuthology.orchestra.run.smithi181.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.2 MB/s | 538 kB 00:00 2024-05-10T04:29:25.498 INFO:teuthology.orchestra.run.smithi150.stdout:(106/146): python3-requests-oauthlib-1.0.0-1.el 512 kB/s | 43 kB 00:00 2024-05-10T04:29:25.543 INFO:teuthology.orchestra.run.smithi161.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.1 MB/s | 254 kB 00:00 2024-05-10T04:29:25.581 INFO:teuthology.orchestra.run.smithi150.stdout:(107/146): python3-oauthlib-2.1.0-1.el8.noarch. 663 kB/s | 155 kB 00:00 2024-05-10T04:29:25.593 INFO:teuthology.orchestra.run.smithi161.stdout:(141/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 943 kB/s | 221 kB 00:00 2024-05-10T04:29:25.602 INFO:teuthology.orchestra.run.smithi181.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 12 MB/s | 3.7 MB 00:00 2024-05-10T04:29:25.627 INFO:teuthology.orchestra.run.smithi181.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-05-10T04:29:25.648 INFO:teuthology.orchestra.run.smithi150.stdout:(108/146): python3-pycparser-2.14-14.el8.noarch 384 kB/s | 109 kB 00:00 2024-05-10T04:29:25.651 INFO:teuthology.orchestra.run.smithi161.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.6 MB/s | 180 kB 00:00 2024-05-10T04:29:25.677 INFO:teuthology.orchestra.run.smithi161.stdout:(143/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.1 MB/s | 237 kB 00:00 2024-05-10T04:29:25.702 INFO:teuthology.orchestra.run.smithi161.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 913 kB/s | 99 kB 00:00 2024-05-10T04:29:25.707 INFO:teuthology.orchestra.run.smithi150.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 89 kB/s | 19 kB 00:00 2024-05-10T04:29:25.732 INFO:teuthology.orchestra.run.smithi150.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 291 kB/s | 44 kB 00:00 2024-05-10T04:29:25.736 INFO:teuthology.orchestra.run.smithi181.stdout:(117/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2024-05-10T04:29:25.802 INFO:teuthology.orchestra.run.smithi161.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 822 kB/s | 103 kB 00:00 2024-05-10T04:29:25.803 INFO:teuthology.orchestra.run.smithi181.stdout:(118/146): python3-numpy-f2py-1.14.3-10.el8.x86 709 kB/s | 225 kB 00:00 2024-05-10T04:29:25.861 INFO:teuthology.orchestra.run.smithi161.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 12 MB/s | 2.5 MB 00:00 2024-05-10T04:29:25.864 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-05-10T04:29:25.864 INFO:teuthology.orchestra.run.smithi161.stdout:Total 9.0 MB/s | 221 MB 00:24 2024-05-10T04:29:25.874 INFO:teuthology.orchestra.run.smithi150.stdout:(111/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-05-10T04:29:25.924 INFO:teuthology.orchestra.run.smithi150.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.0 MB/s | 229 kB 00:00 2024-05-10T04:29:25.983 INFO:teuthology.orchestra.run.smithi150.stdout:(113/146): pcre2-devel-10.32-2.el8.x86_64.rpm 1.8 MB/s | 605 kB 00:00 2024-05-10T04:29:25.996 INFO:teuthology.orchestra.run.smithi181.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 14 MB/s | 5.2 MB 00:00 2024-05-10T04:29:26.017 INFO:teuthology.orchestra.run.smithi150.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 522 kB/s | 48 kB 00:00 2024-05-10T04:29:26.059 INFO:teuthology.orchestra.run.smithi150.stdout:(115/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.1 MB/s | 200 kB 00:00 2024-05-10T04:29:26.084 INFO:teuthology.orchestra.run.smithi150.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 865 kB/s | 87 kB 00:00 2024-05-10T04:29:26.101 INFO:teuthology.orchestra.run.smithi150.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 923 kB/s | 39 kB 00:00 2024-05-10T04:29:26.117 INFO:teuthology.orchestra.run.smithi150.stdout:(118/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 572 kB/s | 58 kB 00:00 2024-05-10T04:29:26.235 INFO:teuthology.orchestra.run.smithi150.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 552 kB/s | 83 kB 00:00 2024-05-10T04:29:26.293 INFO:teuthology.orchestra.run.smithi150.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.3 MB/s | 645 kB 00:00 2024-05-10T04:29:26.321 INFO:teuthology.orchestra.run.smithi181.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 269 kB/s | 157 kB 00:00 2024-05-10T04:29:26.385 INFO:teuthology.orchestra.run.smithi150.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 645 kB/s | 172 kB 00:00 2024-05-10T04:29:26.411 INFO:teuthology.orchestra.run.smithi150.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-05-10T04:29:26.452 INFO:teuthology.orchestra.run.smithi150.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 457 kB/s | 73 kB 00:00 2024-05-10T04:29:26.496 INFO:teuthology.orchestra.run.smithi181.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 1.4 MB/s | 714 kB 00:00 2024-05-10T04:29:26.511 INFO:teuthology.orchestra.run.smithi150.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 492 kB/s | 62 kB 00:00 2024-05-10T04:29:26.512 INFO:teuthology.orchestra.run.smithi181.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2024-05-10T04:29:26.594 INFO:teuthology.orchestra.run.smithi150.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.0 MB/s | 562 kB 00:00 2024-05-10T04:29:26.629 INFO:teuthology.orchestra.run.smithi181.stdout:(123/146): openblas-0.3.15-6.el8.x86_64.rpm 6.1 MB/s | 5.0 MB 00:00 2024-05-10T04:29:26.645 INFO:teuthology.orchestra.run.smithi150.stdout:(126/146): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2024-05-10T04:29:26.655 INFO:teuthology.orchestra.run.smithi181.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-10T04:29:26.680 INFO:teuthology.orchestra.run.smithi181.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 324 kB/s | 54 kB 00:00 2024-05-10T04:29:26.703 INFO:teuthology.orchestra.run.smithi150.stdout:(127/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 245 kB/s | 47 kB 00:00 2024-05-10T04:29:26.755 INFO:teuthology.orchestra.run.smithi181.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-10T04:29:26.770 INFO:teuthology.orchestra.run.smithi150.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.1 MB/s | 559 kB 00:00 2024-05-10T04:29:26.812 INFO:teuthology.orchestra.run.smithi150.stdout:(129/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 919 kB/s | 100 kB 00:00 2024-05-10T04:29:26.822 INFO:teuthology.orchestra.run.smithi181.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 11 MB/s | 1.5 MB 00:00 2024-05-10T04:29:26.837 INFO:teuthology.orchestra.run.smithi150.stdout:(130/146): NetworkManager-initscripts-updown-1. 750 kB/s | 144 kB 00:00 2024-05-10T04:29:26.874 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-05-10T04:29:26.889 INFO:teuthology.orchestra.run.smithi181.stdout:(128/146): postgresql-contrib-10.23-4.module+el 5.9 MB/s | 812 kB 00:00 2024-05-10T04:29:26.971 INFO:teuthology.orchestra.run.smithi150.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2024-05-10T04:29:27.046 INFO:teuthology.orchestra.run.smithi150.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 4.0 MB/s | 303 kB 00:00 2024-05-10T04:29:27.098 INFO:teuthology.orchestra.run.smithi181.stdout:(129/146): postgresql-server-10.23-4.module+el8 18 MB/s | 5.1 MB 00:00 2024-05-10T04:29:27.222 INFO:teuthology.orchestra.run.smithi150.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.8 MB/s | 854 kB 00:00 2024-05-10T04:29:27.240 INFO:teuthology.orchestra.run.smithi181.stdout:(130/146): python3-scipy-1.0.0-21.module+el8.9. 22 MB/s | 14 MB 00:00 2024-05-10T04:29:27.269 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-05-10T04:29:27.269 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-05-10T04:29:27.380 INFO:teuthology.orchestra.run.smithi150.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.4 MB/s | 711 kB 00:00 2024-05-10T04:29:27.440 INFO:teuthology.orchestra.run.smithi181.stdout:(131/146): librados2-18.2.2-1330.g50174015.el8. 6.4 MB/s | 3.5 MB 00:00 2024-05-10T04:29:27.465 INFO:teuthology.orchestra.run.smithi181.stdout:(132/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.0 MB/s | 232 kB 00:00 2024-05-10T04:29:27.566 INFO:teuthology.orchestra.run.smithi150.stdout:(135/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.9 MB/s | 1.5 MB 00:00 2024-05-10T04:29:27.607 INFO:teuthology.orchestra.run.smithi181.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-05-10T04:29:27.641 INFO:teuthology.orchestra.run.smithi181.stdout:(134/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.8 MB/s | 854 kB 00:00 2024-05-10T04:29:27.656 INFO:teuthology.orchestra.run.smithi150.stdout:(136/146): librbd1-18.2.2-1330.g50174015.el8.x8 4.2 MB/s | 3.5 MB 00:00 2024-05-10T04:29:27.698 INFO:teuthology.orchestra.run.smithi150.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.5 MB/s | 333 kB 00:00 2024-05-10T04:29:27.765 INFO:teuthology.orchestra.run.smithi150.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-05-10T04:29:27.774 INFO:teuthology.orchestra.run.smithi181.stdout:(135/146): librbd1-18.2.2-1330.g50174015.el8.x8 5.2 MB/s | 3.5 MB 00:00 2024-05-10T04:29:27.817 INFO:teuthology.orchestra.run.smithi181.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.3 MB/s | 711 kB 00:00 2024-05-10T04:29:27.823 INFO:teuthology.orchestra.run.smithi150.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-10T04:29:27.888 INFO:teuthology.orchestra.run.smithi181.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.0 MB/s | 1.5 MB 00:00 2024-05-10T04:29:27.934 INFO:teuthology.orchestra.run.smithi181.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2024-05-10T04:29:27.998 INFO:teuthology.orchestra.run.smithi150.stdout:(140/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.3 MB/s | 237 kB 00:00 2024-05-10T04:29:28.024 INFO:teuthology.orchestra.run.smithi150.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 980 kB/s | 254 kB 00:00 2024-05-10T04:29:28.101 INFO:teuthology.orchestra.run.smithi181.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.0 MB/s | 221 kB 00:00 2024-05-10T04:29:28.132 INFO:teuthology.orchestra.run.smithi150.stdout:(142/146): librados2-18.2.2-1330.g50174015.el8. 2.6 MB/s | 3.5 MB 00:01 2024-05-10T04:29:28.158 INFO:teuthology.orchestra.run.smithi150.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 743 kB/s | 99 kB 00:00 2024-05-10T04:29:28.174 INFO:teuthology.orchestra.run.smithi181.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.0 MB/s | 254 kB 00:00 2024-05-10T04:29:28.192 INFO:teuthology.orchestra.run.smithi150.stdout:(144/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 933 kB/s | 180 kB 00:00 2024-05-10T04:29:28.226 INFO:teuthology.orchestra.run.smithi181.stdout:(141/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 2.6 MB/s | 1.1 MB 00:00 2024-05-10T04:29:28.251 INFO:teuthology.orchestra.run.smithi181.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 237 kB 00:00 2024-05-10T04:29:28.292 INFO:teuthology.orchestra.run.smithi150.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 769 kB/s | 103 kB 00:00 2024-05-10T04:29:28.327 INFO:teuthology.orchestra.run.smithi181.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 988 kB/s | 99 kB 00:00 2024-05-10T04:29:28.392 INFO:teuthology.orchestra.run.smithi150.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 9.6 MB/s | 2.5 MB 00:00 2024-05-10T04:29:28.394 INFO:teuthology.orchestra.run.smithi181.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 17 MB/s | 2.5 MB 00:00 2024-05-10T04:29:28.395 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-10T04:29:28.395 INFO:teuthology.orchestra.run.smithi150.stdout:Total 8.1 MB/s | 221 MB 00:27 2024-05-10T04:29:28.419 INFO:teuthology.orchestra.run.smithi181.stdout:(145/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 736 kB/s | 180 kB 00:00 2024-05-10T04:29:28.486 INFO:teuthology.orchestra.run.smithi181.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 646 kB/s | 103 kB 00:00 2024-05-10T04:29:28.489 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-05-10T04:29:28.490 INFO:teuthology.orchestra.run.smithi181.stdout:Total 8.2 MB/s | 221 MB 00:26 2024-05-10T04:29:28.495 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-05-10T04:29:28.506 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-05-10T04:29:29.405 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-05-10T04:29:29.526 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-05-10T04:29:29.799 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-05-10T04:29:29.799 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-05-10T04:29:29.921 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-05-10T04:29:29.921 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-05-10T04:29:31.030 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-05-10T04:29:31.040 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-05-10T04:29:31.148 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-05-10T04:29:31.159 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-05-10T04:29:34.042 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-05-10T04:29:34.175 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T04:29:34.435 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T04:29:34.650 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T04:29:34.857 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T04:29:35.064 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T04:29:35.120 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T04:29:35.281 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T04:29:35.348 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T04:29:35.552 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T04:29:35.762 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-10T04:29:35.956 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T04:29:36.054 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T04:29:36.222 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T04:29:36.623 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-05-10T04:29:36.637 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T04:29:36.758 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T04:29:36.842 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:29:36.922 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-05-10T04:29:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T04:29:37.056 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-10T04:29:37.248 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:29:37.248 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T04:29:37.311 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T04:29:37.405 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:29:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T04:29:37.522 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:29:37.534 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-10T04:29:37.610 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T04:29:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:29:37.710 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T04:29:37.739 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-10T04:29:37.785 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:29:37.869 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T04:29:37.922 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T04:29:37.960 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T04:29:38.021 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-10T04:29:38.107 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T04:29:38.184 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T04:29:38.201 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T04:29:38.257 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-10T04:29:38.330 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T04:29:38.386 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-10T04:29:38.401 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T04:29:38.488 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-10T04:29:38.488 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-10T04:29:38.611 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T04:29:38.679 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-10T04:29:38.699 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-10T04:29:38.885 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T04:29:38.902 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T04:29:38.917 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T04:29:38.997 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-10T04:29:39.075 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T04:29:39.201 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T04:29:39.303 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T04:29:39.426 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T04:29:39.510 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:29:39.613 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-10T04:29:39.685 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T04:29:39.843 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:29:39.849 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:29:39.921 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:29:39.954 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:29:40.097 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T04:29:40.119 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:29:40.175 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T04:29:40.239 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:29:40.246 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:29:40.358 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-10T04:29:40.427 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:29:40.443 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:29:40.550 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:29:40.558 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:29:40.647 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-10T04:29:40.743 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:29:40.829 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:29:40.860 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-10T04:29:40.862 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T04:29:41.071 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T04:29:41.135 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T04:29:41.152 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T04:29:41.203 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T04:29:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T04:29:41.355 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T04:29:41.451 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T04:29:41.475 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T04:29:41.536 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-10T04:29:41.774 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-10T04:29:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T04:29:41.963 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T04:29:41.977 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T04:29:42.031 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-10T04:29:42.156 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T04:29:42.196 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-10T04:29:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T04:29:42.573 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T04:29:42.589 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-10T04:29:42.758 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T04:29:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:29:42.813 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-10T04:29:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:29:42.921 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T04:29:43.009 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:29:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T04:29:43.114 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T04:29:43.115 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:29:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T04:29:43.302 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-10T04:29:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 20/162 2024-05-10T04:29:43.455 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-10T04:29:43.563 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-10T04:29:43.599 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T04:29:43.635 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-10T04:29:43.819 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-10T04:29:43.826 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-10T04:29:44.009 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-10T04:29:44.085 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-10T04:29:44.234 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T04:29:44.260 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-10T04:29:44.263 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T04:29:44.363 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-10T04:29:44.419 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T04:29:44.452 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-10T04:29:44.525 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T04:29:44.548 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T04:29:44.783 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T04:29:45.006 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T04:29:45.057 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-10T04:29:45.137 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-10T04:29:45.210 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-10T04:29:45.341 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T04:29:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-10T04:29:45.942 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-10T04:29:46.142 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T04:29:46.149 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-10T04:29:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-10T04:29:46.513 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-10T04:29:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T04:29:46.786 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-10T04:29:46.820 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-10T04:29:47.089 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-10T04:29:47.089 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-10T04:29:47.090 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-10T04:29:47.290 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-10T04:29:47.361 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T04:29:47.552 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-10T04:29:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T04:29:47.687 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T04:29:47.740 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 31/162 2024-05-10T04:29:47.894 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 32/162 2024-05-10T04:29:47.963 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T04:29:48.049 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-10T04:29:48.129 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 33/162 2024-05-10T04:29:48.350 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T04:29:48.368 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-10T04:29:48.571 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-10T04:29:48.765 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T04:29:48.921 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-10T04:29:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T04:29:49.226 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T04:29:49.290 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-10T04:29:49.386 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T04:29:49.597 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-10T04:29:49.612 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T04:29:49.691 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T04:29:49.834 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-10T04:29:49.906 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T04:29:49.976 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T04:29:50.075 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-10T04:29:50.199 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T04:29:50.427 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-10T04:29:50.447 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T04:29:50.603 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T04:29:51.028 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T04:29:51.321 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T04:29:51.348 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T04:29:51.393 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-10T04:29:51.529 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T04:29:51.575 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-10T04:29:51.605 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T04:29:51.729 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-10T04:29:51.838 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-10T04:29:51.902 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-10T04:29:51.933 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-10T04:29:52.070 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-10T04:29:52.120 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T04:29:52.169 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-10T04:29:52.245 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T04:29:52.315 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T04:29:52.383 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T04:29:52.435 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-10T04:29:52.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-10T04:29:52.556 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T04:29:52.652 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-10T04:29:52.654 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 41/162 2024-05-10T04:29:52.775 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T04:29:52.816 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 42/162 2024-05-10T04:29:52.863 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-10T04:29:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T04:29:53.029 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-10T04:29:53.032 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T04:29:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T04:29:53.112 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-10T04:29:53.208 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-10T04:29:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T04:29:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T04:29:53.357 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T04:29:53.401 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-10T04:29:53.472 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-10T04:29:53.555 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T04:29:53.609 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-10T04:29:53.697 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-10T04:29:53.757 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-10T04:29:53.761 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T04:29:53.910 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-10T04:29:53.940 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-10T04:29:53.992 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T04:29:54.116 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T04:29:54.241 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-10T04:29:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T04:29:54.474 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-10T04:29:54.553 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-10T04:29:54.700 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-10T04:29:54.792 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T04:29:54.809 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T04:29:54.849 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-10T04:29:54.858 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T04:29:54.885 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-10T04:29:54.996 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-10T04:29:55.080 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-10T04:29:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T04:29:55.187 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-10T04:29:55.229 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-10T04:29:55.426 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-10T04:29:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-10T04:29:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-10T04:29:55.636 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-10T04:29:55.681 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-10T04:29:55.716 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-10T04:29:55.836 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-10T04:29:55.924 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T04:29:55.986 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-10T04:29:55.997 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-10T04:29:56.156 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-10T04:29:56.185 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-10T04:29:56.253 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-10T04:29:56.392 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-10T04:29:56.437 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-10T04:29:56.448 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-10T04:29:56.635 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-10T04:29:56.665 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-10T04:29:56.671 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-10T04:29:56.826 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-10T04:29:56.861 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-10T04:29:56.915 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-10T04:29:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-10T04:29:57.085 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-10T04:29:57.112 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-10T04:29:57.330 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-10T04:29:57.474 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-10T04:29:57.533 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-10T04:29:57.622 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-10T04:29:57.786 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-10T04:29:57.795 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-10T04:29:57.845 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-10T04:29:57.952 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T04:29:58.052 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-10T04:29:58.108 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-10T04:29:58.124 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T04:29:58.244 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-10T04:29:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-10T04:29:58.433 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T04:29:58.536 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-10T04:29:58.568 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-10T04:29:58.659 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T04:29:58.762 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-10T04:29:58.829 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-10T04:29:58.857 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T04:29:59.021 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-10T04:29:59.055 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T04:29:59.062 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-10T04:29:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-10T04:29:59.250 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T04:29:59.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-10T04:29:59.443 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-10T04:29:59.477 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-10T04:29:59.482 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T04:29:59.707 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-10T04:29:59.722 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T04:29:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-10T04:29:59.922 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T04:29:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-10T04:30:00.066 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T04:30:00.118 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-10T04:30:00.183 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-10T04:30:00.270 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-10T04:30:00.298 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T04:30:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-10T04:30:00.539 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T04:30:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-10T04:30:00.681 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-10T04:30:00.758 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T04:30:00.817 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-10T04:30:00.889 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-10T04:30:00.975 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-10T04:30:01.006 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T04:30:01.047 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-10T04:30:01.161 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-10T04:30:01.287 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T04:30:01.299 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-10T04:30:01.419 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-10T04:30:01.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T04:30:01.474 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-10T04:30:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-10T04:30:01.694 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-10T04:30:01.700 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T04:30:01.875 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-10T04:30:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-10T04:30:02.080 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-10T04:30:02.108 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T04:30:02.272 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T04:30:02.400 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-10T04:30:02.565 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-10T04:30:02.588 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T04:30:02.784 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T04:30:02.819 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-10T04:30:02.873 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T04:30:03.054 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T04:30:03.081 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T04:30:03.144 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-10T04:30:03.211 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T04:30:03.309 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T04:30:03.405 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-10T04:30:03.450 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T04:30:03.569 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-10T04:30:03.586 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T04:30:03.662 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T04:30:03.742 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-10T04:30:03.835 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T04:30:03.891 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T04:30:03.977 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-10T04:30:04.018 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T04:30:04.102 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T04:30:04.177 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-10T04:30:04.288 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T04:30:04.427 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-10T04:30:04.536 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T04:30:04.668 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-10T04:30:04.753 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T04:30:04.887 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-10T04:30:04.897 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T04:30:04.938 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T04:30:05.060 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T04:30:05.173 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T04:30:05.175 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-10T04:30:05.196 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T04:30:05.361 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T04:30:05.409 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-10T04:30:05.473 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T04:30:05.577 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-10T04:30:05.648 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T04:30:05.883 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-10T04:30:05.899 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T04:30:06.138 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-10T04:30:06.313 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-10T04:30:06.554 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-10T04:30:07.016 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T04:30:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T04:30:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T04:30:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-10T04:30:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T04:30:08.084 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-10T04:30:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T04:30:08.304 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-10T04:30:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T04:30:08.597 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-10T04:30:08.872 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-10T04:30:09.105 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-10T04:30:09.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T04:30:09.473 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T04:30:09.534 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T04:30:09.605 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:09.749 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T04:30:09.868 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:10.004 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-10T04:30:10.166 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T04:30:10.272 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 103/162 2024-05-10T04:30:10.447 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 104/162 2024-05-10T04:30:10.491 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:10.788 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T04:30:10.875 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T04:30:11.001 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T04:30:14.108 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:14.365 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:14.443 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:14.688 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:14.991 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:15.245 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T04:30:15.312 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 105/162 2024-05-10T04:30:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T04:30:15.498 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T04:30:15.576 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T04:30:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 106/162 2024-05-10T04:30:15.788 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T04:30:30.761 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T04:30:30.761 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-05-10T04:30:30.761 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-05-10T04:30:30.761 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2024-05-10T04:30:30.762 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2024-05-10T04:30:30.762 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-05-10T04:30:30.762 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-05-10T04:30:30.762 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-05-10T04:30:30.762 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-05-10T04:30:30.762 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2024-05-10T04:30:30.762 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:30:31.106 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T04:30:32.828 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T04:30:33.009 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T04:30:33.228 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T04:30:33.380 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T04:30:33.898 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T04:30:34.039 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T04:30:34.056 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /run 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-05-10T04:30:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-05-10T04:30:35.131 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2024-05-10T04:30:35.131 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:30:35.478 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 107/162 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2024-05-10T04:30:35.535 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:30:35.807 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T04:30:35.935 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T04:30:35.961 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T04:30:36.103 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T04:30:36.297 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T04:30:36.505 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T04:30:37.212 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T04:30:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T04:30:37.536 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T04:30:37.627 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T04:30:37.663 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 108/162 2024-05-10T04:30:37.775 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T04:30:37.795 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T04:30:37.880 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T04:30:38.099 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 109/162 2024-05-10T04:30:38.276 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T04:30:38.316 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T04:30:38.511 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T04:30:38.529 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T04:30:38.742 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T04:30:38.795 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 110/162 2024-05-10T04:30:38.921 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T04:30:38.992 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T04:30:39.010 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 111/162 2024-05-10T04:30:39.014 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T04:30:39.356 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T04:30:39.525 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T04:30:39.563 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T04:30:39.738 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T04:30:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T04:30:40.394 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T04:30:40.552 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T04:30:40.684 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T04:30:40.797 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T04:30:40.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T04:30:40.982 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.2.2-1330.g50174015.el8.noarch 112/162 2024-05-10T04:30:41.099 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 113/162 2024-05-10T04:30:41.153 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T04:30:41.238 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T04:30:41.326 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 114/162 2024-05-10T04:30:41.433 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T04:30:41.467 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T04:30:41.592 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T04:30:41.651 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T04:30:41.693 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T04:30:41.838 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T04:30:42.189 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T04:30:42.189 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T04:30:42.346 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T04:30:42.457 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T04:30:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 115/162 2024-05-10T04:30:42.879 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T04:30:43.429 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T04:30:43.627 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T04:30:43.721 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T04:30:43.848 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 116/162 2024-05-10T04:30:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T04:30:44.076 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T04:30:44.127 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 117/162 2024-05-10T04:30:44.258 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T04:30:44.297 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T04:30:44.494 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T04:30:44.505 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T04:30:44.511 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 118/162 2024-05-10T04:30:44.713 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T04:30:44.753 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 119/162 2024-05-10T04:30:44.950 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T04:30:44.961 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 120/162 2024-05-10T04:30:46.006 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T04:30:46.207 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T04:30:46.245 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T04:30:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T04:30:46.459 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.2.2-1330.g50174015.el8.x86_64 121/162 2024-05-10T04:30:46.488 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T04:30:46.652 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T04:30:46.687 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T04:30:46.722 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 122/162 2024-05-10T04:30:46.922 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T04:30:46.966 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1330.g50174 123/162 2024-05-10T04:30:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T04:30:47.125 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T04:30:47.177 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T04:30:47.232 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T04:30:47.477 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 124/162 2024-05-10T04:30:47.708 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T04:30:48.214 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T04:30:48.508 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-10T04:30:48.778 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-10T04:30:49.170 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 129/162 2024-05-10T04:30:49.455 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-10T04:30:49.703 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T04:30:49.716 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T04:30:49.938 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T04:30:50.119 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T04:30:50.287 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T04:30:52.078 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T04:30:52.562 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:52.586 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T04:30:52.606 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 125/162 2024-05-10T04:30:52.878 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-10T04:30:52.886 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:52.927 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:53.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-10T04:30:53.326 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T04:30:53.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 126/162 2024-05-10T04:30:53.665 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T04:30:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-10T04:30:53.913 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T04:30:53.986 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-10T04:30:54.086 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T04:30:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-10T04:30:54.249 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-10T04:30:54.277 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T04:30:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-10T04:30:54.581 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T04:30:54.641 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 129/162 2024-05-10T04:30:54.673 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T04:30:54.898 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T04:30:54.903 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T04:30:54.975 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-10T04:30:55.101 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T04:30:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T04:30:55.180 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T04:30:55.231 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T04:30:55.245 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-10T04:30:55.319 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T04:30:55.394 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T04:30:55.407 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T04:30:55.508 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T04:30:55.577 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T04:30:55.712 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-10T04:30:55.749 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-10T04:30:55.902 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T04:30:55.933 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 134/162 2024-05-10T04:30:56.112 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T04:30:56.289 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T04:30:56.290 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T04:30:56.547 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T04:30:56.547 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T04:30:56.772 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T04:30:56.773 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T04:30:57.025 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T04:30:57.220 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T04:30:57.405 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T04:30:57.653 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T04:30:57.654 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:30:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:57.863 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:30:57.863 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T04:30:57.968 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:58.006 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:58.104 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T04:30:58.104 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:30:58.239 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:58.338 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:30:58.338 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T04:30:58.461 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T04:30:58.540 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:58.578 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-10T04:30:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T04:30:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:30:58.846 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:30:58.876 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T04:30:58.988 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T04:30:58.988 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T04:30:59.023 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-10T04:30:59.117 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T04:30:59.301 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T04:30:59.413 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-10T04:30:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T04:30:59.653 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-10T04:30:59.838 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T04:30:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 139/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T04:30:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T04:30:59.893 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T04:30:59.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T04:30:59.895 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T04:30:59.896 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T04:30:59.897 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-10T04:30:59.898 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-10T04:30:59.899 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-10T04:30:59.900 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-10T04:30:59.901 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-10T04:30:59.902 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T04:30:59.903 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:30:59.904 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T04:30:59.905 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:30:59.905 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T04:31:00.081 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 140/162 2024-05-10T04:31:00.135 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T04:31:00.317 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-10T04:31:00.328 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-10T04:31:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 141/162 2024-05-10T04:31:00.513 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-10T04:31:00.766 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-10T04:31:00.768 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-10T04:31:00.847 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-10T04:31:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T04:31:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-10T04:31:01.158 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-10T04:31:01.172 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-10T04:31:01.255 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T04:31:01.269 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-10T04:31:01.352 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T04:31:01.455 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T04:31:01.465 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-10T04:31:01.569 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T04:31:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-10T04:31:01.737 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T04:31:01.738 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T04:31:01.831 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-10T04:31:02.053 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T04:31:02.053 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T04:31:02.057 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-10T04:31:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-10T04:31:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T04:31:02.295 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T04:31:02.296 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T04:31:02.500 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-10T04:31:02.500 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T04:31:02.590 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T04:31:02.767 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-10T04:31:02.768 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T04:31:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T04:31:03.011 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T04:31:03.045 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-10T04:31:03.265 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-10T04:31:03.268 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T04:31:03.268 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:31:03.432 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-10T04:31:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:31:03.511 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T04:31:03.674 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-10T04:31:03.674 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:31:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T04:31:03.752 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:31:03.925 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:31:03.925 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T04:31:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:31:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T04:31:04.174 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-10T04:31:04.175 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:31:04.260 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T04:31:04.260 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:31:04.391 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:31:04.391 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T04:31:04.519 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:31:04.674 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-10T04:31:04.674 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:31:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T04:31:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T04:31:04.949 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:31:05.099 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-10T04:31:05.099 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T04:31:05.831 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:31:05.832 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T04:31:05.833 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T04:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T04:31:05.834 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T04:31:05.846 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T04:31:05.847 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T04:31:05.848 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-10T04:31:05.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-10T04:31:05.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-10T04:31:05.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-10T04:31:05.852 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-10T04:31:05.853 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T04:31:05.854 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T04:31:05.855 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:31:05.856 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T04:31:06.023 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-10T04:31:06.023 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.2.2-1330.g50174015.el8.x86_64 1/162 2024-05-10T04:31:06.023 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2/162 2024-05-10T04:31:06.023 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 3/162 2024-05-10T04:31:06.023 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 4/162 2024-05-10T04:31:06.023 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1330.g50174 5/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 6/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 7/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 8/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 9/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 10/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 11/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 12/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 13/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_ 14/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 15/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 16/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.2.2-1330.g50174015.el8.x86_6 17/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.2.2-1330.g50174015.el8.x86 18/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.2.2-1330.g50174015.el8.x86_64 19/162 2024-05-10T04:31:06.024 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1330.g50174015.el 20/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.2.2-1330.g50174015.el8. 21/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_6 22/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 23/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 24/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 25/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 26/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 27/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 28/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1330.g50174015. 29/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noa 30/162 2024-05-10T04:31:06.025 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.n 31/162 2024-05-10T04:31:06.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1330.g501 32/162 2024-05-10T04:31:06.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el 33/162 2024-05-10T04:31:06.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 34/162 2024-05-10T04:31:06.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1330.g50174015.e 35/162 2024-05-10T04:31:06.038 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 36/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.2.2-1330.g50174015.el8.noarch 37/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 38/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-10T04:31:06.039 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-10T04:31:06.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-10T04:31:06.041 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-10T04:31:06.042 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-10T04:31:06.043 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-10T04:31:06.044 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.2.2-1330.g50174015.el8.x86_64 131/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.2.2-1330.g50174015.el8.x86_64 133/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-10T04:31:06.045 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-10T04:31:06.046 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-10T04:31:06.047 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-10T04:31:13.315 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T04:31:13.316 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.827 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T04:31:14.828 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.829 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T04:31:14.830 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-7.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T04:31:14.831 INFO:teuthology.orchestra.run.smithi161.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:14.832 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T04:31:14.833 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:14.834 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:14.834 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:14.835 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T04:31:14.835 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T04:31:14.835 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T04:31:14.835 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T04:31:14.835 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T04:31:14.835 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.836 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.836 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.836 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T04:31:14.836 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T04:31:14.836 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T04:31:14.836 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T04:31:14.837 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T04:31:14.838 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.839 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T04:31:14.840 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T04:31:14.841 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T04:31:14.842 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T04:31:14.842 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T04:31:14.842 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-05-10T04:31:14.842 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-05-10T04:31:15.349 DEBUG:teuthology.parallel:result is None 2024-05-10T04:31:19.767 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T04:31:19.767 INFO:teuthology.orchestra.run.smithi150.stdout:Installed products updated. 2024-05-10T04:31:20.363 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:31:20.363 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-05-10T04:31:20.363 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.364 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.365 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.366 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: jq-1.6-7.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.367 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T04:31:20.368 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T04:31:20.369 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T04:31:20.370 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T04:31:20.371 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T04:31:20.372 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T04:31:20.373 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T04:31:20.374 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T04:31:20.375 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T04:31:20.375 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:31:20.375 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-10T04:31:20.581 DEBUG:teuthology.parallel:result is None 2024-05-10T04:31:21.519 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-10T04:31:21.519 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.116 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.117 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.117 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.117 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:22.117 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:22.117 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.117 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-8.el8.noarch 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.118 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.119 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:18.2.2-1330.g50174015.el8.noarch 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-7.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.120 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libpq-13.11-1.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-10T04:31:22.121 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-10T04:31:22.122 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-10T04:31:22.123 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-10T04:31:22.124 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-10T04:31:22.125 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-10T04:31:22.126 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-10T04:31:22.127 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-10T04:31:22.127 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-10T04:31:22.127 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-10T04:31:22.127 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-10T04:31:22.127 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-10T04:31:22.127 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-10T04:31:22.128 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-10T04:31:22.128 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.128 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.128 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.2.2-1330.g50174015.el8.x86_64 2024-05-10T04:31:22.128 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-10T04:31:22.128 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-10T04:31:22.128 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-10T04:31:22.129 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-10T04:31:22.129 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-10T04:31:22.129 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-10T04:31:22.129 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:31:22.129 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-05-10T04:31:22.361 DEBUG:teuthology.parallel:result is None 2024-05-10T04:31:22.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:31:22.504 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T04:31:22.533 INFO:teuthology.orchestra.run.smithi150.stdout:18.2.2-1330.g50174015.el8 2024-05-10T04:31:22.533 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T04:31:22.534 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T04:31:22.536 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:31:22.684 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T04:31:22.714 INFO:teuthology.orchestra.run.smithi161.stdout:18.2.2-1330.g50174015.el8 2024-05-10T04:31:22.714 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T04:31:22.714 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T04:31:22.716 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:31:22.864 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-10T04:31:22.888 INFO:teuthology.orchestra.run.smithi181.stdout:18.2.2-1330.g50174015.el8 2024-05-10T04:31:22.888 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1330.g50174015.el8 2024-05-10T04:31:22.889 INFO:teuthology.task.install:The correct ceph version 18.2.2-1330.g50174015 is installed. 2024-05-10T04:31:22.891 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-10T04:31:22.891 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:31:22.891 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T04:31:22.941 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:31:22.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T04:31:22.991 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:31:22.991 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-10T04:31:23.033 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-10T04:31:23.034 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:31:23.034 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T04:31:23.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T04:31:23.137 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:31:23.137 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T04:31:23.169 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T04:31:23.240 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:31:23.240 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-05-10T04:31:23.269 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-10T04:31:23.336 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-10T04:31:23.337 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:31:23.337 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T04:31:23.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T04:31:23.440 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:31:23.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T04:31:23.473 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T04:31:23.545 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:31:23.545 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-10T04:31:23.572 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-10T04:31:23.639 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-10T04:31:23.639 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:31:23.639 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T04:31:23.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T04:31:23.743 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:31:23.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T04:31:23.776 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T04:31:23.847 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:31:23.848 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-05-10T04:31:23.876 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-10T04:31:23.988 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-10T04:31:24.086 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T04:31:24.086 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:31:24.087 INFO:tasks.cephadm:Cluster fsid is 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:31:24.087 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-10T04:31:24.087 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.150', 'mon.b': '172.21.15.161', 'mon.c': '172.21.15.181'} 2024-05-10T04:31:24.087 INFO:tasks.cephadm:First mon is mon.a on smithi150 2024-05-10T04:31:24.087 INFO:tasks.cephadm:First mgr is x 2024-05-10T04:31:24.088 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-10T04:31:24.088 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-05-10T04:31:24.121 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-05-10T04:31:24.154 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-05-10T04:31:24.187 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-10T04:31:24.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:31:24.330 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '50174015a7e79f951b988949abe236791f52d4da', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79270/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1330-g50174015', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1330.g50174015'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-10 02:06:05.327930', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240509.171335-reef-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-10T04:31:24.545 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240509.171335-reef-debug, sha1 50174015a7e79f951b988949abe236791f52d4da from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:31:24.546 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm 2024-05-10T04:31:24.547 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:25.092 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 04:31 /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:25.092 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:25.607 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 04:31 /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:25.608 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240509.171335-reef-debug/50174015a7e79f951b988949abe236791f52d4da/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:25.966 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 10 04:31 /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:25.966 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:25.989 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:26.012 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-10T04:31:26.063 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da on all hosts... 2024-05-10T04:31:26.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T04:31:26.068 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T04:31:26.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da pull 2024-05-10T04:31:27.467 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T04:31:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T04:31:27.828 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T04:31:59.003 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:31:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T04:31:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T04:31:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-05-10T04:31:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T04:31:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-05-10T04:31:59.004 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:32:17.836 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:32:17.837 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T04:32:17.837 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T04:32:17.837 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-05-10T04:32:17.837 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T04:32:17.837 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-05-10T04:32:17.837 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:32:18.244 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:32:18.244 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T04:32:18.244 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b", 2024-05-10T04:32:18.244 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-05-10T04:32:18.244 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:042b1f8e1a7496c1ea8ed86a601031b3e73743a9905239589edc685eef0b229c" 2024-05-10T04:32:18.245 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:32:18.245 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:32:18.264 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-05-10T04:32:18.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-05-10T04:32:18.327 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-05-10T04:32:18.366 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-05-10T04:32:18.393 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-05-10T04:32:18.420 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-05-10T04:32:18.452 INFO:tasks.cephadm:Writing seed config... 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-10T04:32:18.454 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-10T04:32:18.455 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-10T04:32:18.456 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-10T04:32:18.456 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-10T04:32:18.456 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-10T04:32:18.456 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-10T04:32:18.456 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-10T04:32:18.456 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-10T04:32:18.456 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-10T04:32:18.457 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-10T04:32:18.457 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-10T04:32:18.457 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-10T04:32:18.457 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-10T04:32:18.457 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-10T04:32:18.457 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-10T04:32:18.457 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-10T04:32:18.458 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-10T04:32:18.458 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-10T04:32:18.458 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-10T04:32:18.458 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-10T04:32:18.459 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-10T04:32:18.459 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-10T04:32:18.459 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-10T04:32:18.460 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-10T04:32:18.460 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-10T04:32:18.460 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-10T04:32:18.461 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:32:18.461 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-10T04:32:18.478 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 28928ad0-0e86-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-10T04:32:18.478 DEBUG:teuthology.orchestra.run.smithi150:mon.a> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a.service 2024-05-10T04:32:18.521 DEBUG:teuthology.orchestra.run.smithi150:mgr.x> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.x.service 2024-05-10T04:32:18.547 INFO:journalctl@ceph.mon.a.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:32:18.547 INFO:tasks.cephadm:Bootstrapping... 2024-05-10T04:32:18.548 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da -v bootstrap --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.150 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:32:18.573 INFO:journalctl@ceph.mgr.x.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:32:18.708 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-10T04:32:18.708 INFO:teuthology.orchestra.run.smithi150.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da', '-v', 'bootstrap', '--fsid', '28928ad0-0e86-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.150', '--skip-admin-label'] 2024-05-10T04:32:18.740 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 4.6.1 2024-05-10T04:32:18.740 INFO:teuthology.orchestra.run.smithi150.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-10T04:32:18.740 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying podman|docker is present... 2024-05-10T04:32:18.773 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 4.6.1 2024-05-10T04:32:18.773 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying lvm2 is present... 2024-05-10T04:32:18.773 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying time synchronization is in place... 2024-05-10T04:32:18.780 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T04:32:18.780 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T04:32:18.786 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T04:32:18.786 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T04:32:18.794 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout enabled 2024-05-10T04:32:18.798 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout active 2024-05-10T04:32:18.798 INFO:teuthology.orchestra.run.smithi150.stdout:Unit chronyd.service is enabled and running 2024-05-10T04:32:18.798 INFO:teuthology.orchestra.run.smithi150.stdout:Repeating the final host check... 2024-05-10T04:32:18.831 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 4.6.1 2024-05-10T04:32:18.831 INFO:teuthology.orchestra.run.smithi150.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-10T04:32:18.831 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl is present 2024-05-10T04:32:18.831 INFO:teuthology.orchestra.run.smithi150.stdout:lvcreate is present 2024-05-10T04:32:18.837 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-10T04:32:18.837 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-10T04:32:18.843 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-10T04:32:18.843 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T04:32:18.848 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout enabled 2024-05-10T04:32:18.856 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout active 2024-05-10T04:32:18.856 INFO:teuthology.orchestra.run.smithi150.stdout:Unit chronyd.service is enabled and running 2024-05-10T04:32:18.857 INFO:teuthology.orchestra.run.smithi150.stdout:Host looks OK 2024-05-10T04:32:18.857 INFO:teuthology.orchestra.run.smithi150.stdout:Cluster fsid: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:18.857 INFO:teuthology.orchestra.run.smithi150.stdout:Acquiring lock 139933278493160 on /run/cephadm/28928ad0-0e86-11ef-bc98-c7b262605968.lock 2024-05-10T04:32:18.857 INFO:teuthology.orchestra.run.smithi150.stdout:Lock 139933278493160 acquired on /run/cephadm/28928ad0-0e86-11ef-bc98-c7b262605968.lock 2024-05-10T04:32:18.857 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying IP 172.21.15.150 port 3300 ... 2024-05-10T04:32:18.858 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying IP 172.21.15.150 port 6789 ... 2024-05-10T04:32:18.858 INFO:teuthology.orchestra.run.smithi150.stdout:Base mon IP(s) is [172.21.15.150:3300, 172.21.15.150:6789], mon addrv is [v2:172.21.15.150:3300,v1:172.21.15.150:6789] 2024-05-10T04:32:18.860 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-05-10T04:32:18.860 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.150 metric 100 2024-05-10T04:32:18.862 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-10T04:32:18.863 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-05-10T04:32:18.863 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-05-10T04:32:18.865 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout inet6 fe80::c92:6035:7edc:b85a/64 scope link noprefixroute 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:Mon IP `172.21.15.150` is in CIDR network `172.21.0.0/20` 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:Mon IP `172.21.15.150` is in CIDR network `172.21.0.0/20` 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-10T04:32:18.866 INFO:teuthology.orchestra.run.smithi150.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-10T04:32:18.867 INFO:teuthology.orchestra.run.smithi150.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T04:32:19.236 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stdout 0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b 2024-05-10T04:32:19.236 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da... 2024-05-10T04:32:19.236 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Getting image source signatures 2024-05-10T04:32:19.236 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Copying blob sha256:5889b1d3c79fb904da685578f1acb24d7c4091d50b19c3ba92279423a786328a 2024-05-10T04:32:19.236 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-10T04:32:19.236 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Copying config sha256:0e8066d08b37ba491bbdc3407be474324b50b47fdf56caf5acbacae7a4a8fb9b 2024-05-10T04:32:19.236 INFO:teuthology.orchestra.run.smithi150.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-10T04:32:19.902 INFO:teuthology.orchestra.run.smithi150.stdout:ceph: stdout ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable) 2024-05-10T04:32:19.902 INFO:teuthology.orchestra.run.smithi150.stdout:Ceph version: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable) 2024-05-10T04:32:19.902 INFO:teuthology.orchestra.run.smithi150.stdout:Extracting ceph user uid/gid from container image... 2024-05-10T04:32:20.482 INFO:teuthology.orchestra.run.smithi150.stdout:stat: stdout 167 167 2024-05-10T04:32:20.482 INFO:teuthology.orchestra.run.smithi150.stdout:Creating initial keys... 2024-05-10T04:32:21.072 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph-authtool: stdout AQBUoz1mToSLKxAA0x5jRISQ4g5rKAB00NhOGg== 2024-05-10T04:32:21.626 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph-authtool: stdout AQBVoz1moQY1EhAA25WWKCEIHUAmpYLv6TUwsQ== 2024-05-10T04:32:22.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph-authtool: stdout AQBVoz1mLqt/MxAAais2ziYGOQXLHqziM1dq1g== 2024-05-10T04:32:22.193 INFO:teuthology.orchestra.run.smithi150.stdout:Creating initial monmap... 2024-05-10T04:32:22.782 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T04:32:22.782 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-10T04:32:22.782 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:22.783 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T04:32:22.783 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool for a [v2:172.21.15.150:3300,v1:172.21.15.150:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-10T04:32:22.783 INFO:teuthology.orchestra.run.smithi150.stdout:setting min_mon_release = pacific 2024-05-10T04:32:22.783 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: set fsid to 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:22.783 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-10T04:32:22.783 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:32:22.783 INFO:teuthology.orchestra.run.smithi150.stdout:Creating mon... 2024-05-10T04:32:23.401 INFO:teuthology.orchestra.run.smithi150.stdout:create mon.a on 2024-05-10T04:32:23.558 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-10T04:32:23.652 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-10T04:32:23.777 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-28928ad0-0e86-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-28928ad0-0e86-11ef-bc98-c7b262605968.target. 2024-05-10T04:32:23.777 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-28928ad0-0e86-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-28928ad0-0e86-11ef-bc98-c7b262605968.target. 2024-05-10T04:32:23.979 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a 2024-05-10T04:32:23.980 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to reset failed state of unit ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-10T04:32:24.070 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-28928ad0-0e86-11ef-bc98-c7b262605968.target.wants/ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-28928ad0-0e86-11ef-bc98-c7b262605968@.service. 2024-05-10T04:32:24.432 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T04:32:24.432 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T04:32:24.440 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T04:32:24.441 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T04:32:24.441 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T04:32:24.441 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T04:32:24.441 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mon to start... 2024-05-10T04:32:24.441 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mon... 2024-05-10T04:32:24.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:24 smithi150 ceph-mon[41918]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T04:32:25.235 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout cluster: 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout id: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout services: 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.441343s) 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout data: 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-10T04:32:25.236 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout pgs: 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.819+0000 7f42b03dc700 1 Processor -- start 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.819+0000 7f42b03dc700 1 -- start start 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.819+0000 7f42b03dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8107c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.819+0000 7f42b03dc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8108210 con 0x7f42a8107830 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.820+0000 7f42ae178700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.820+0000 7f42ae178700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8107c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41900/0 (socket says 172.21.15.150:41900) 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.820+0000 7f42ae178700 1 -- 172.21.15.150:0/3679066499 learned_addr learned my addr 172.21.15.150:0/3679066499 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.820+0000 7f42ae178700 1 -- 172.21.15.150:0/3679066499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8108350 con 0x7f42a8107830 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.820+0000 7f42ae178700 1 --2- 172.21.15.150:0/3679066499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8107c40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f42a40098a0 tx=0x7f42a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934bc862db74f840 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:25.237 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.820+0000 7f42ad176700 1 -- 172.21.15.150:0/3679066499 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a4004160 con 0x7f42a8107830 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42ad176700 1 -- 172.21.15.150:0/3679066499 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a40042c0 con 0x7f42a8107830 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42ad176700 1 -- 172.21.15.150:0/3679066499 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a40045a0 con 0x7f42a8107830 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42b03dc700 1 -- 172.21.15.150:0/3679066499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 msgr2=0x7f42a8107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42b03dc700 1 --2- 172.21.15.150:0/3679066499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8107c40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f42a40098a0 tx=0x7f42a4009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42b03dc700 1 -- 172.21.15.150:0/3679066499 shutdown_connections 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42b03dc700 1 --2- 172.21.15.150:0/3679066499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8107c40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42b03dc700 1 -- 172.21.15.150:0/3679066499 >> 172.21.15.150:0/3679066499 conn(0x7f42a8101130 msgr2=0x7f42a8103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42b03dc700 1 -- 172.21.15.150:0/3679066499 shutdown_connections 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.821+0000 7f42b03dc700 1 -- 172.21.15.150:0/3679066499 wait complete. 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.822+0000 7f42b03dc700 1 Processor -- start 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.822+0000 7f42b03dc700 1 -- start start 2024-05-10T04:32:25.238 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.822+0000 7f42b03dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.822+0000 7f42b03dc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8106300 con 0x7f42a8107830 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.822+0000 7f42ae178700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.822+0000 7f42ae178700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8105dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41908/0 (socket says 172.21.15.150:41908) 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.822+0000 7f42ae178700 1 -- 172.21.15.150:0/1429694403 learned_addr learned my addr 172.21.15.150:0/1429694403 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f42ae178700 1 -- 172.21.15.150:0/1429694403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a4009550 con 0x7f42a8107830 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f42ae178700 1 --2- 172.21.15.150:0/1429694403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8105dc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f42a4004000 tx=0x7f42a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f429f7fe700 1 -- 172.21.15.150:0/1429694403 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a4011630 con 0x7f42a8107830 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f429f7fe700 1 -- 172.21.15.150:0/1429694403 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4011790 con 0x7f42a8107830 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f429f7fe700 1 -- 172.21.15.150:0/1429694403 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42a4011a30 con 0x7f42a8107830 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a819d9d0 con 0x7f42a8107830 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a819ddb0 con 0x7f42a8107830 2024-05-10T04:32:25.239 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f429f7fe700 1 -- 172.21.15.150:0/1429694403 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f42a4021070 con 0x7f42a8107830 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.823+0000 7f429f7fe700 1 -- 172.21.15.150:0/1429694403 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f42a401b780 con 0x7f42a8107830 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.824+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a81931f0 con 0x7f42a8107830 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.826+0000 7f429f7fe700 1 -- 172.21.15.150:0/1429694403 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f42a401acc0 con 0x7f42a8107830 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.896+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f42a8106950 con 0x7f42a8107830 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.896+0000 7f429f7fe700 1 -- 172.21.15.150:0/1429694403 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f42a401ae60 con 0x7f42a8107830 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.897+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 msgr2=0x7f42a8105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.897+0000 7f42b03dc700 1 --2- 172.21.15.150:0/1429694403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8105dc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f42a4004000 tx=0x7f42a4004750 comp rx=0 tx=0).stop 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.897+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 shutdown_connections 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.897+0000 7f42b03dc700 1 --2- 172.21.15.150:0/1429694403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42a8107830 0x7f42a8105dc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.897+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 >> 172.21.15.150:0/1429694403 conn(0x7f42a8101130 msgr2=0x7f42a818dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.897+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 shutdown_connections 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:24.897+0000 7f42b03dc700 1 -- 172.21.15.150:0/1429694403 wait complete. 2024-05-10T04:32:25.240 INFO:teuthology.orchestra.run.smithi150.stdout:mon is available 2024-05-10T04:32:25.241 INFO:teuthology.orchestra.run.smithi150.stdout:Assimilating anything we can from ceph.conf... 2024-05-10T04:32:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:25 smithi150 ceph-mon[41918]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T04:32:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:25 smithi150 ceph-mon[41918]: monmap e1: 1 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:32:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:25 smithi150 ceph-mon[41918]: fsmap 2024-05-10T04:32:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:25 smithi150 ceph-mon[41918]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T04:32:25.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:25 smithi150 ceph-mon[41918]: mgrmap e1: no daemons active 2024-05-10T04:32:25.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:25 smithi150 ceph-mon[41918]: from='client.? 172.21.15.150:0/1429694403' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-10T04:32:25.987 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout fsid = 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.150:3300,v1:172.21.15.150:6789] 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T04:32:25.988 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef9b71700 1 Processor -- start 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef9b71700 1 -- start start 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef9b71700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef4109f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef9b71700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef410a590 con 0x7f0ef407acf0 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef4109f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:25.989 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef4109f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41910/0 (socket says 172.21.15.150:41910) 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef37fe700 1 -- 172.21.15.150:0/2718434743 learned_addr learned my addr 172.21.15.150:0/2718434743 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.560+0000 7f0ef37fe700 1 -- 172.21.15.150:0/2718434743 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef410a6d0 con 0x7f0ef407acf0 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.561+0000 7f0ef37fe700 1 --2- 172.21.15.150:0/2718434743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef4109f80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40098a0 tx=0x7f0ee4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7302dd09b7b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.561+0000 7f0ef27fc700 1 -- 172.21.15.150:0/2718434743 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ee4004160 con 0x7f0ef407acf0 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.561+0000 7f0ef27fc700 1 -- 172.21.15.150:0/2718434743 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee40042c0 con 0x7f0ef407acf0 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.561+0000 7f0ef27fc700 1 -- 172.21.15.150:0/2718434743 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ee40045a0 con 0x7f0ef407acf0 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.561+0000 7f0ef9b71700 1 -- 172.21.15.150:0/2718434743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 msgr2=0x7f0ef4109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.561+0000 7f0ef9b71700 1 --2- 172.21.15.150:0/2718434743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef4109f80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40098a0 tx=0x7f0ee4009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.562+0000 7f0ef9b71700 1 -- 172.21.15.150:0/2718434743 shutdown_connections 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.562+0000 7f0ef9b71700 1 --2- 172.21.15.150:0/2718434743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef4109f80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:25.990 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.562+0000 7f0ef9b71700 1 -- 172.21.15.150:0/2718434743 >> 172.21.15.150:0/2718434743 conn(0x7f0ef41019d0 msgr2=0x7f0ef4103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.562+0000 7f0ef9b71700 1 -- 172.21.15.150:0/2718434743 shutdown_connections 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.562+0000 7f0ef9b71700 1 -- 172.21.15.150:0/2718434743 wait complete. 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.562+0000 7f0ef9b71700 1 Processor -- start 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.562+0000 7f0ef9b71700 1 -- start start 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef9b71700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef9b71700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef419b140 con 0x7f0ef407acf0 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef37fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41926/0 (socket says 172.21.15.150:41926) 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef37fe700 1 -- 172.21.15.150:0/3412554295 learned_addr learned my addr 172.21.15.150:0/3412554295 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef37fe700 1 -- 172.21.15.150:0/3412554295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4009550 con 0x7f0ef407acf0 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef37fe700 1 --2- 172.21.15.150:0/3412554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef419ac00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4004000 tx=0x7f0ee4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ee4011630 con 0x7f0ef407acf0 2024-05-10T04:32:25.991 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee4011790 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ee4011a30 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef419b340 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.563+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef419b7e0 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.564+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0ee4021070 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.564+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0ee401b780 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.564+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4194720 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.567+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f0ee401acc0 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.634+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0ef410aa40 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.639+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ee401ae60 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.640+0000 7f0ef0ff9700 1 -- 172.21.15.150:0/3412554295 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f0ee40334e0 con 0x7f0ef407acf0 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.640+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 msgr2=0x7f0ef419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.640+0000 7f0ef9b71700 1 --2- 172.21.15.150:0/3412554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef419ac00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4004000 tx=0x7f0ee4004750 comp rx=0 tx=0).stop 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.641+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 shutdown_connections 2024-05-10T04:32:25.992 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.641+0000 7f0ef9b71700 1 --2- 172.21.15.150:0/3412554295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ef407acf0 0x7f0ef419ac00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:25.993 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.641+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 >> 172.21.15.150:0/3412554295 conn(0x7f0ef41019d0 msgr2=0x7f0ef41026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:25.993 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.641+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 shutdown_connections 2024-05-10T04:32:25.993 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:25.641+0000 7f0ef9b71700 1 -- 172.21.15.150:0/3412554295 wait complete. 2024-05-10T04:32:25.993 INFO:teuthology.orchestra.run.smithi150.stdout:Generating new minimal ceph.conf... 2024-05-10T04:32:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:26 smithi150 ceph-mon[41918]: from='client.? 172.21.15.150:0/3412554295' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T04:32:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:26 smithi150 ceph-mon[41918]: from='client.? 172.21.15.150:0/3412554295' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-10T04:32:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:26 smithi150 ceph-mon[41918]: from='client.? 172.21.15.150:0/629238519' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:32:26.725 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.301+0000 7fdfc4412700 1 Processor -- start 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.301+0000 7fdfc4412700 1 -- start start 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc4412700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc108db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc4412700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc109380 con 0x7fdfbc1089a0 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc21ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc108db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc21ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc108db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41940/0 (socket says 172.21.15.150:41940) 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc21ae700 1 -- 172.21.15.150:0/3207492804 learned_addr learned my addr 172.21.15.150:0/3207492804 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc21ae700 1 -- 172.21.15.150:0/3207492804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc1094c0 con 0x7fdfbc1089a0 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc21ae700 1 --2- 172.21.15.150:0/3207492804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc108db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00098a0 tx=0x7fdfb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9beee0a25039e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:26.726 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc11ac700 1 -- 172.21.15.150:0/3207492804 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfb0004160 con 0x7fdfbc1089a0 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc11ac700 1 -- 172.21.15.150:0/3207492804 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdfb00042c0 con 0x7fdfbc1089a0 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.302+0000 7fdfc11ac700 1 -- 172.21.15.150:0/3207492804 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfb0011a00 con 0x7fdfbc1089a0 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.303+0000 7fdfc4412700 1 -- 172.21.15.150:0/3207492804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 msgr2=0x7fdfbc108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.303+0000 7fdfc4412700 1 --2- 172.21.15.150:0/3207492804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc108db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00098a0 tx=0x7fdfb0009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.303+0000 7fdfc4412700 1 -- 172.21.15.150:0/3207492804 shutdown_connections 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.303+0000 7fdfc4412700 1 --2- 172.21.15.150:0/3207492804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc108db0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.303+0000 7fdfc4412700 1 -- 172.21.15.150:0/3207492804 >> 172.21.15.150:0/3207492804 conn(0x7fdfbc101180 msgr2=0x7fdfbc1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.303+0000 7fdfc4412700 1 -- 172.21.15.150:0/3207492804 shutdown_connections 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.303+0000 7fdfc4412700 1 -- 172.21.15.150:0/3207492804 wait complete. 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc4412700 1 Processor -- start 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc4412700 1 -- start start 2024-05-10T04:32:26.727 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc4412700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc4412700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc19b940 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc21ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc21ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41952/0 (socket says 172.21.15.150:41952) 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc21ae700 1 -- 172.21.15.150:0/629238519 learned_addr learned my addr 172.21.15.150:0/629238519 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.304+0000 7fdfc21ae700 1 -- 172.21.15.150:0/629238519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0009550 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.305+0000 7fdfc21ae700 1 --2- 172.21.15.150:0/629238519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc19b400 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0011710 tx=0x7fdfb0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.305+0000 7fdfaf7fe700 1 -- 172.21.15.150:0/629238519 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfb0003aa0 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.305+0000 7fdfaf7fe700 1 -- 172.21.15.150:0/629238519 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdfb0003c00 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.305+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc19bba0 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.305+0000 7fdfaf7fe700 1 -- 172.21.15.150:0/629238519 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfb0024800 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.305+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc19e7e0 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.306+0000 7fdfaf7fe700 1 -- 172.21.15.150:0/629238519 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdfb0024960 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.306+0000 7fdfaf7fe700 1 -- 172.21.15.150:0/629238519 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdfb002da50 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.306+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc062840 con 0x7fdfbc1089a0 2024-05-10T04:32:26.728 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.308+0000 7fdfaf7fe700 1 -- 172.21.15.150:0/629238519 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdfb001a8d0 con 0x7fdfbc1089a0 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.376+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fdfbc19ea80 con 0x7fdfbc1089a0 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.376+0000 7fdfaf7fe700 1 -- 172.21.15.150:0/629238519 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fdfb002d3f0 con 0x7fdfbc1089a0 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.377+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 msgr2=0x7fdfbc19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.377+0000 7fdfc4412700 1 --2- 172.21.15.150:0/629238519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc19b400 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0011710 tx=0x7fdfb0011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.377+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 shutdown_connections 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.377+0000 7fdfc4412700 1 --2- 172.21.15.150:0/629238519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfbc1089a0 0x7fdfbc19b400 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.377+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 >> 172.21.15.150:0/629238519 conn(0x7fdfbc101180 msgr2=0x7fdfbc102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.377+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 shutdown_connections 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:26.377+0000 7fdfc4412700 1 -- 172.21.15.150:0/629238519 wait complete. 2024-05-10T04:32:26.729 INFO:teuthology.orchestra.run.smithi150.stdout:Restarting the monitor... 2024-05-10T04:32:27.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:26 smithi150 systemd[1]: Stopping Ceph mon.a for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:32:27.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:26 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-a[41895]: 2024-05-10T04:32:26.913+0000 7fa8aaa5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:32:27.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:26 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-a[41895]: 2024-05-10T04:32:26.913+0000 7fa8aaa5d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-10T04:32:27.502 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 bash[42246]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-a 2024-05-10T04:32:27.758 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 systemd[1]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a.service: Succeeded. 2024-05-10T04:32:27.758 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 systemd[1]: Stopped Ceph mon.a for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:32:27.758 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 systemd[1]: Starting Ceph mon.a for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:32:27.805 INFO:teuthology.orchestra.run.smithi150.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-10T04:32:27.815 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 bash[42367]: aedb3acd44a1afe472fb205c01998992377a65706fc6aa227cdf784708068283 2024-05-10T04:32:27.815 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 systemd[1]: Started Ceph mon.a for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:32:27.815 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T04:32:28.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable), process ceph-mon, pid 7 2024-05-10T04:32:28.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: pidfile_write: ignore empty --pid-file 2024-05-10T04:32:28.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: load: jerasure load: lrc 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: RocksDB version: 7.9.2 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Git sha 0 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: DB SUMMARY 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: DB Session ID: 977JGLFNLKM1QVYZOI1D 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: CURRENT file: CURRENT 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99243 ; 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.error_if_exists: 0 2024-05-10T04:32:28.164 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.create_if_missing: 0 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.paranoid_checks: 1 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.env: 0x55d26e5c9460 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.info_log: 0x55d26f1cf760 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.statistics: (nil) 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.use_fsync: 0 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_log_file_size: 0 2024-05-10T04:32:28.165 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.allow_fallocate: 1 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.use_direct_reads: 0 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.db_log_dir: 2024-05-10T04:32:28.166 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.wal_dir: 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.write_buffer_manager: 0x55d26f1130e0 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T04:32:28.167 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.unordered_write: 0 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.row_cache: None 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.wal_filter: None 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T04:32:28.168 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.two_write_queues: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.wal_compression: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.atomic_flush: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.log_readahead_size: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T04:32:28.169 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_background_jobs: 2 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_background_compactions: -1 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_subcompactions: 1 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T04:32:28.170 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_open_files: -1 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_background_flushes: -1 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Compression algorithms supported: 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kZSTD supported: 0 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kXpressCompression supported: 0 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kZlibCompression supported: 1 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kSnappyCompression supported: 1 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kLZ4Compression supported: 1 2024-05-10T04:32:28.171 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: kBZip2Compression supported: 0 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.merge_operator: 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_filter: None 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d26f1cf880) 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout: cache_index_and_filter_blocks: 1 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T04:32:28.172 INFO:journalctl@ceph.mon.a.smithi150.stdout: pin_top_level_index_and_filter: 1 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: index_type: 0 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: data_block_index_type: 0 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: index_shortening: 1 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: checksum: 4 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: no_block_cache: 0 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache: 0x55d26f1951f0 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache_name: BinnedLRUCache 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache_options: 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: capacity : 536870912 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: num_shard_bits : 4 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: strict_capacity_limit : 0 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: high_pri_pool_ratio: 0.000 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_cache_compressed: (nil) 2024-05-10T04:32:28.173 INFO:journalctl@ceph.mon.a.smithi150.stdout: persistent_cache: (nil) 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_size: 4096 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_size_deviation: 10 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_restart_interval: 16 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: index_block_restart_interval: 1 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: metadata_block_size: 4096 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: partition_filters: 0 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: use_delta_encoding: 1 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: filter_policy: bloomfilter 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: whole_key_filtering: 1 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: verify_compression: 0 2024-05-10T04:32:28.174 INFO:journalctl@ceph.mon.a.smithi150.stdout: read_amp_bytes_per_bit: 0 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout: format_version: 5 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout: enable_index_compression: 1 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout: block_align: 0 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout: max_auto_readahead_size: 262144 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout: prepopulate_block_cache: 0 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout: initial_auto_readahead_size: 8192 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression: NoCompression 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.num_levels: 7 2024-05-10T04:32:28.175 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T04:32:28.176 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T04:32:28.177 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T04:32:28.178 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.table_properties_collectors: 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.inplace_update_support: 0 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T04:32:28.179 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.bloom_locality: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.max_successive_merges: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.ttl: 2592000 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.enable_blob_files: false 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.min_blob_size: 0 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T04:32:28.180 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9dfec202-b3f3-463b-bae8-8c1a8745ff70 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315547822869, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315547824594, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 92585, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13774, "raw_average_key_size": 50, "raw_value_size": 85074, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715315547, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9dfec202-b3f3-463b-bae8-8c1a8745ff70", "db_session_id": "977JGLFNLKM1QVYZOI1D", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315547824696, "job": 1, "event": "recovery_finished"} 2024-05-10T04:32:28.181 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d26f127c00 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: DB pointer 0x55d273f16000 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** DB Stats ** 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T04:32:28.182 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** Compaction Stats [default] ** 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: L0 2/0 94.21 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: Sum 2/0 94.21 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** Compaction Stats [default] ** 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T04:32:28.183 INFO:journalctl@ceph.mon.a.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: Cumulative compaction: 0.00 GB write, 18.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: Interval compaction: 0.00 GB write, 18.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T04:32:28.184 INFO:journalctl@ceph.mon.a.smithi150.stdout: Block cache BinnedLRUCache@0x55d26f1951f0#7 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: starting mon.a rank 0 at public addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] at bind addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???) e1 preinit fsid 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???).mds e1 new map 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???).mds e1 print_map 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: e1 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: legacy client fscid: -1 2024-05-10T04:32:28.185 INFO:journalctl@ceph.mon.a.smithi150.stdout: 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout: No filesystems configured 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: monmap e1: 1 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: fsmap 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: osdmap e1: 0 total, 0 up, 0 in 2024-05-10T04:32:28.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:27 smithi150 ceph-mon[42404]: mgrmap e1: no daemons active 2024-05-10T04:32:28.649 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.175+0000 7fa809f50700 1 Processor -- start 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.175+0000 7fa809f50700 1 -- start start 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.176+0000 7fa809f50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804109f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.176+0000 7fa809f50700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80410a5a0 con 0x7fa80407acf0 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.176+0000 7fa8037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804109f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.176+0000 7fa8037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804109f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49322/0 (socket says 172.21.15.150:49322) 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.176+0000 7fa8037fe700 1 -- 172.21.15.150:0/2811553998 learned_addr learned my addr 172.21.15.150:0/2811553998 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.176+0000 7fa8037fe700 1 -- 172.21.15.150:0/2811553998 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80410a6e0 con 0x7fa80407acf0 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.176+0000 7fa8037fe700 1 --2- 172.21.15.150:0/2811553998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804109f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80098a0 tx=0x7fa7f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38abe400d75d28b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa8027fc700 1 -- 172.21.15.150:0/2811553998 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f8004160 con 0x7fa80407acf0 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa8027fc700 1 -- 172.21.15.150:0/2811553998 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7f80042c0 con 0x7fa80407acf0 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa8027fc700 1 -- 172.21.15.150:0/2811553998 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f8011a00 con 0x7fa80407acf0 2024-05-10T04:32:28.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa809f50700 1 -- 172.21.15.150:0/2811553998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 msgr2=0x7fa804109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa809f50700 1 --2- 172.21.15.150:0/2811553998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804109f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80098a0 tx=0x7fa7f8009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa809f50700 1 -- 172.21.15.150:0/2811553998 shutdown_connections 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa809f50700 1 --2- 172.21.15.150:0/2811553998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804109f90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa809f50700 1 -- 172.21.15.150:0/2811553998 >> 172.21.15.150:0/2811553998 conn(0x7fa8041019c0 msgr2=0x7fa804103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.177+0000 7fa809f50700 1 -- 172.21.15.150:0/2811553998 shutdown_connections 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.178+0000 7fa809f50700 1 -- 172.21.15.150:0/2811553998 wait complete. 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.178+0000 7fa809f50700 1 Processor -- start 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.178+0000 7fa809f50700 1 -- start start 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.178+0000 7fa809f50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804107700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.178+0000 7fa809f50700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804107c40 con 0x7fa80407acf0 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa8037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804107700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa8037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804107700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49326/0 (socket says 172.21.15.150:49326) 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa8037fe700 1 -- 172.21.15.150:0/4281850206 learned_addr learned my addr 172.21.15.150:0/4281850206 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa8037fe700 1 -- 172.21.15.150:0/4281850206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8009550 con 0x7fa80407acf0 2024-05-10T04:32:28.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa8037fe700 1 --2- 172.21.15.150:0/4281850206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804107700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009130 tx=0x7fa7f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa800ff9700 1 -- 172.21.15.150:0/4281850206 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f80038b0 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa800ff9700 1 -- 172.21.15.150:0/4281850206 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7f8003a10 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa804105e90 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa800ff9700 1 -- 172.21.15.150:0/4281850206 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7f8019550 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.179+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa804106330 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.180+0000 7fa800ff9700 1 -- 172.21.15.150:0/4281850206 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa7f8019af0 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.180+0000 7fa800ff9700 1 -- 172.21.15.150:0/4281850206 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa7f802de30 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.180+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa804062840 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.184+0000 7fa800ff9700 1 -- 172.21.15.150:0/4281850206 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fa7f8003b80 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.251+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fa80410b830 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.257+0000 7fa800ff9700 1 -- 172.21.15.150:0/4281850206 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa7f8005a20 con 0x7fa80407acf0 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.258+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 msgr2=0x7fa804107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.258+0000 7fa809f50700 1 --2- 172.21.15.150:0/4281850206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804107700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009130 tx=0x7fa7f8011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.258+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 shutdown_connections 2024-05-10T04:32:28.652 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.258+0000 7fa809f50700 1 --2- 172.21.15.150:0/4281850206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa80407acf0 0x7fa804107700 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:28.653 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.258+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 >> 172.21.15.150:0/4281850206 conn(0x7fa8041019c0 msgr2=0x7fa8041022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:28.653 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.258+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 shutdown_connections 2024-05-10T04:32:28.653 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:28.258+0000 7fa809f50700 1 -- 172.21.15.150:0/4281850206 wait complete. 2024-05-10T04:32:28.653 INFO:teuthology.orchestra.run.smithi150.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-10T04:32:28.684 INFO:teuthology.orchestra.run.smithi150.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:32:28.684 INFO:teuthology.orchestra.run.smithi150.stdout:Creating mgr... 2024-05-10T04:32:28.685 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-10T04:32:28.685 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-10T04:32:28.914 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.x 2024-05-10T04:32:28.914 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Failed to reset failed state of unit ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-10T04:32:29.009 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-28928ad0-0e86-11ef-bc98-c7b262605968.target.wants/ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-28928ad0-0e86-11ef-bc98-c7b262605968@.service. 2024-05-10T04:32:29.357 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T04:32:29.358 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T04:32:29.366 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T04:32:29.366 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T04:32:29.366 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T04:32:29.366 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to enable service . firewalld.service is not available 2024-05-10T04:32:29.375 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T04:32:29.375 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T04:32:29.383 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T04:32:29.383 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T04:32:29.383 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T04:32:29.383 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-10T04:32:29.383 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr to start... 2024-05-10T04:32:29.383 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr... 2024-05-10T04:32:29.511 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:29 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4281850206' entity='client.admin' 2024-05-10T04:32:29.512 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:29 smithi150 bash[42639]: 506365544db8048ac6812153543bbff98d216e976ea52b28832cc56f9683450f 2024-05-10T04:32:29.512 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:29 smithi150 systemd[1]: Started Ceph mgr.x for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:32:30.050 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:29.773+0000 7f1daec9b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T04:32:30.051 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:29.870+0000 7f1daec9b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T04:32:30.051 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:29.961+0000 7f1daec9b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T04:32:30.051 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:30 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:30.048+0000 7f1daec9b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T04:32:30.189 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:30.189 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:32:30.189 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:32:30.189 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T04:32:30.189 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T04:32:30.189 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T04:32:30.189 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T04:32:30.190 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 1, 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T04:32:30.191 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T04:32:30.192 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:30.193 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T04:32:24.458680+0000", 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.747+0000 7fe88a51a700 1 Processor -- start 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.747+0000 7fe88a51a700 1 -- start start 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.747+0000 7fe88a51a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe884107c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.747+0000 7fe88a51a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe884108260 con 0x7fe884107860 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.748+0000 7fe883fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe884107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.748+0000 7fe883fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe884107c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49356/0 (socket says 172.21.15.150:49356) 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.748+0000 7fe883fff700 1 -- 172.21.15.150:0/733734462 learned_addr learned my addr 172.21.15.150:0/733734462 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.748+0000 7fe883fff700 1 -- 172.21.15.150:0/733734462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8841083a0 con 0x7fe884107860 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.748+0000 7fe883fff700 1 --2- 172.21.15.150:0/733734462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe884107c70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098a0 tx=0x7fe874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7001005863791e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.748+0000 7fe882ffd700 1 -- 172.21.15.150:0/733734462 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe874004160 con 0x7fe884107860 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe882ffd700 1 -- 172.21.15.150:0/733734462 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe8740042c0 con 0x7fe884107860 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe882ffd700 1 -- 172.21.15.150:0/733734462 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe874011a00 con 0x7fe884107860 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe88a51a700 1 -- 172.21.15.150:0/733734462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 msgr2=0x7fe884107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:30.195 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe88a51a700 1 --2- 172.21.15.150:0/733734462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe884107c70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098a0 tx=0x7fe874009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe88a51a700 1 -- 172.21.15.150:0/733734462 shutdown_connections 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe88a51a700 1 --2- 172.21.15.150:0/733734462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe884107c70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe88a51a700 1 -- 172.21.15.150:0/733734462 >> 172.21.15.150:0/733734462 conn(0x7fe884101130 msgr2=0x7fe884103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.749+0000 7fe88a51a700 1 -- 172.21.15.150:0/733734462 shutdown_connections 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.750+0000 7fe88a51a700 1 -- 172.21.15.150:0/733734462 wait complete. 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.750+0000 7fe88a51a700 1 Processor -- start 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.750+0000 7fe88a51a700 1 -- start start 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.751+0000 7fe88a51a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe88419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.751+0000 7fe88a51a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88419dc60 con 0x7fe884107860 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.751+0000 7fe883fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe88419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.751+0000 7fe883fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe88419d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49372/0 (socket says 172.21.15.150:49372) 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.751+0000 7fe883fff700 1 -- 172.21.15.150:0/1738396699 learned_addr learned my addr 172.21.15.150:0/1738396699 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.751+0000 7fe883fff700 1 -- 172.21.15.150:0/1738396699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe874009550 con 0x7fe884107860 2024-05-10T04:32:30.196 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.751+0000 7fe883fff700 1 --2- 172.21.15.150:0/1738396699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe88419d720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe874009130 tx=0x7fe874011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.752+0000 7fe8817fa700 1 -- 172.21.15.150:0/1738396699 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8740038b0 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.752+0000 7fe8817fa700 1 -- 172.21.15.150:0/1738396699 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe874003a10 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.752+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88419de60 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.752+0000 7fe8817fa700 1 -- 172.21.15.150:0/1738396699 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe874019550 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.752+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88419e300 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.752+0000 7fe8817fa700 1 -- 172.21.15.150:0/1738396699 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe874019b10 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.752+0000 7fe8817fa700 1 -- 172.21.15.150:0/1738396699 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe87402dd70 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.753+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88410d730 con 0x7fe884107860 2024-05-10T04:32:30.197 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.755+0000 7fe8817fa700 1 -- 172.21.15.150:0/1738396699 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fe874003b80 con 0x7fe884107860 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.827+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe8841a0b60 con 0x7fe884107860 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.828+0000 7fe8817fa700 1 -- 172.21.15.150:0/1738396699 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fe874035090 con 0x7fe884107860 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.829+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 msgr2=0x7fe88419d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.829+0000 7fe88a51a700 1 --2- 172.21.15.150:0/1738396699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe88419d720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe874009130 tx=0x7fe874011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.829+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 shutdown_connections 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.829+0000 7fe88a51a700 1 --2- 172.21.15.150:0/1738396699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe884107860 0x7fe88419d720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.829+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 >> 172.21.15.150:0/1738396699 conn(0x7fe884101130 msgr2=0x7fe884102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.829+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 shutdown_connections 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:29.829+0000 7fe88a51a700 1 -- 172.21.15.150:0/1738396699 wait complete. 2024-05-10T04:32:30.198 INFO:teuthology.orchestra.run.smithi150.stdout:mgr not available, waiting (1/15)... 2024-05-10T04:32:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:30 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1738396699' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T04:32:31.023 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:30 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:30.689+0000 7f1daec9b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:32:31.023 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:30 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:30.937+0000 7f1daec9b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T04:32:31.023 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:31.021+0000 7f1daec9b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T04:32:31.287 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:31.128+0000 7f1daec9b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T04:32:31.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:31.286+0000 7f1daec9b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T04:32:31.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:31.412+0000 7f1daec9b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T04:32:31.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:31.496+0000 7f1daec9b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T04:32:32.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:31.865+0000 7f1daec9b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T04:32:32.560 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:32.256+0000 7f1daec9b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T04:32:32.560 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:32.461+0000 7f1daec9b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T04:32:32.843 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/921576820' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T04:32:32.843 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:32.558+0000 7f1daec9b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T04:32:33.011 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:33.012 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T04:32:33.013 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T04:32:33.022 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T04:32:33.022 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T04:32:33.022 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T04:32:33.022 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T04:32:33.022 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:33.022 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T04:32:33.024 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T04:32:33.024 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T04:32:33.024 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T04:32:33.024 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T04:32:33.024 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T04:32:33.024 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T04:32:33.024 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T04:32:24.458680+0000", 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.621+0000 7f1543808700 1 Processor -- start 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.621+0000 7f1543808700 1 -- start start 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.621+0000 7f1543808700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c109f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.621+0000 7f1543808700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c10a590 con 0x7f153c07acf0 2024-05-10T04:32:33.025 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.621+0000 7f15415a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c109f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.621+0000 7f15415a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c109f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49386/0 (socket says 172.21.15.150:49386) 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.621+0000 7f15415a4700 1 -- 172.21.15.150:0/2638248071 learned_addr learned my addr 172.21.15.150:0/2638248071 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.622+0000 7f15415a4700 1 -- 172.21.15.150:0/2638248071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c10a6d0 con 0x7f153c07acf0 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.622+0000 7f15415a4700 1 --2- 172.21.15.150:0/2638248071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c109f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15380098a0 tx=0x7f1538009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b52e2494fec786 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.622+0000 7f1533fff700 1 -- 172.21.15.150:0/2638248071 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1538004160 con 0x7f153c07acf0 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.622+0000 7f1533fff700 1 -- 172.21.15.150:0/2638248071 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15380042c0 con 0x7f153c07acf0 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.622+0000 7f1533fff700 1 -- 172.21.15.150:0/2638248071 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1538011a00 con 0x7f153c07acf0 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.623+0000 7f1543808700 1 -- 172.21.15.150:0/2638248071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 msgr2=0x7f153c109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.623+0000 7f1543808700 1 --2- 172.21.15.150:0/2638248071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c109f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15380098a0 tx=0x7f1538009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.623+0000 7f1543808700 1 -- 172.21.15.150:0/2638248071 shutdown_connections 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.623+0000 7f1543808700 1 --2- 172.21.15.150:0/2638248071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c109f80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.623+0000 7f1543808700 1 -- 172.21.15.150:0/2638248071 >> 172.21.15.150:0/2638248071 conn(0x7f153c1019d0 msgr2=0x7f153c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:33.026 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.623+0000 7f1543808700 1 -- 172.21.15.150:0/2638248071 shutdown_connections 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.623+0000 7f1543808700 1 -- 172.21.15.150:0/2638248071 wait complete. 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f1543808700 1 Processor -- start 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f1543808700 1 -- start start 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f1543808700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f1543808700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19ba50 con 0x7f153c07acf0 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f15415a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f15415a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49388/0 (socket says 172.21.15.150:49388) 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f15415a4700 1 -- 172.21.15.150:0/921576820 learned_addr learned my addr 172.21.15.150:0/921576820 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.624+0000 7f15415a4700 1 -- 172.21.15.150:0/921576820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538009550 con 0x7f153c07acf0 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f15415a4700 1 --2- 172.21.15.150:0/921576820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c19b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1538009130 tx=0x7f1538011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f15327fc700 1 -- 172.21.15.150:0/921576820 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15380038b0 con 0x7f153c07acf0 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f15327fc700 1 -- 172.21.15.150:0/921576820 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1538003a10 con 0x7f153c07acf0 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f15327fc700 1 -- 172.21.15.150:0/921576820 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1538019550 con 0x7f153c07acf0 2024-05-10T04:32:33.027 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c19bc50 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c19c0f0 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f15327fc700 1 -- 172.21.15.150:0/921576820 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1538019a80 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.625+0000 7f15327fc700 1 -- 172.21.15.150:0/921576820 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f153802ddc0 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.626+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c10b590 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.630+0000 7f15327fc700 1 -- 172.21.15.150:0/921576820 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f1538003b80 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.702+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f153c19eb60 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.702+0000 7f15327fc700 1 -- 172.21.15.150:0/921576820 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f153802d440 con 0x7f153c07acf0 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.703+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 msgr2=0x7f153c19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.703+0000 7f1543808700 1 --2- 172.21.15.150:0/921576820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c19b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1538009130 tx=0x7f1538011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.703+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 shutdown_connections 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.703+0000 7f1543808700 1 --2- 172.21.15.150:0/921576820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153c07acf0 0x7f153c19b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:33.028 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.703+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 >> 172.21.15.150:0/921576820 conn(0x7f153c1019d0 msgr2=0x7f153c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:33.029 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.703+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 shutdown_connections 2024-05-10T04:32:33.029 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:32.703+0000 7f1543808700 1 -- 172.21.15.150:0/921576820 wait complete. 2024-05-10T04:32:33.029 INFO:teuthology.orchestra.run.smithi150.stdout:mgr not available, waiting (2/15)... 2024-05-10T04:32:33.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:33 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:33.347+0000 7f1daec9b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T04:32:33.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:33 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:33.444+0000 7f1daec9b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T04:32:33.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:33 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:33.592+0000 7f1daec9b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T04:32:34.596 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:34.217+0000 7f1daec9b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T04:32:34.597 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:34.402+0000 7f1daec9b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T04:32:34.911 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:34.595+0000 7f1daec9b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:35.821 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T04:32:35.822 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:35.823 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T04:32:35.823 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T04:32:35.823 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T04:32:35.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T04:32:35.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T04:32:35.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T04:32:35.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:35.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T04:32:35.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T04:32:35.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T04:32:35.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T04:32:24.458680+0000", 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.386+0000 7fd1108f8700 1 Processor -- start 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.386+0000 7fd1108f8700 1 -- start start 2024-05-10T04:32:35.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.386+0000 7fd1108f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c109f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.386+0000 7fd1108f8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c10a590 con 0x7fd10c07acf0 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c109f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c109f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49400/0 (socket says 172.21.15.150:49400) 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10a59c700 1 -- 172.21.15.150:0/1231827870 learned_addr learned my addr 172.21.15.150:0/1231827870 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10a59c700 1 -- 172.21.15.150:0/1231827870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c10a6d0 con 0x7fd10c07acf0 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10a59c700 1 --2- 172.21.15.150:0/1231827870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c109f80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1000098a0 tx=0x7fd100009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4ca71225a9e654 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10959a700 1 -- 172.21.15.150:0/1231827870 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd100004160 con 0x7fd10c07acf0 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10959a700 1 -- 172.21.15.150:0/1231827870 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd1000042c0 con 0x7fd10c07acf0 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.387+0000 7fd10959a700 1 -- 172.21.15.150:0/1231827870 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd100011a00 con 0x7fd10c07acf0 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.388+0000 7fd1108f8700 1 -- 172.21.15.150:0/1231827870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 msgr2=0x7fd10c109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.388+0000 7fd1108f8700 1 --2- 172.21.15.150:0/1231827870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c109f80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1000098a0 tx=0x7fd100009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.388+0000 7fd1108f8700 1 -- 172.21.15.150:0/1231827870 shutdown_connections 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.388+0000 7fd1108f8700 1 --2- 172.21.15.150:0/1231827870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c109f80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.388+0000 7fd1108f8700 1 -- 172.21.15.150:0/1231827870 >> 172.21.15.150:0/1231827870 conn(0x7fd10c1019d0 msgr2=0x7fd10c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:35.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.388+0000 7fd1108f8700 1 -- 172.21.15.150:0/1231827870 shutdown_connections 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.388+0000 7fd1108f8700 1 -- 172.21.15.150:0/1231827870 wait complete. 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.389+0000 7fd1108f8700 1 Processor -- start 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.389+0000 7fd1108f8700 1 -- start start 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.389+0000 7fd1108f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.389+0000 7fd1108f8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c19ba50 con 0x7fd10c07acf0 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.389+0000 7fd10a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.389+0000 7fd10a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49412/0 (socket says 172.21.15.150:49412) 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.389+0000 7fd10a59c700 1 -- 172.21.15.150:0/2338525654 learned_addr learned my addr 172.21.15.150:0/2338525654 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd10a59c700 1 -- 172.21.15.150:0/2338525654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd100009550 con 0x7fd10c07acf0 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd10a59c700 1 --2- 172.21.15.150:0/2338525654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c19b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd100011710 tx=0x7fd100011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd0fb7fe700 1 -- 172.21.15.150:0/2338525654 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd100003af0 con 0x7fd10c07acf0 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c19bc50 con 0x7fd10c07acf0 2024-05-10T04:32:35.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd0fb7fe700 1 -- 172.21.15.150:0/2338525654 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd100003c50 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd0fb7fe700 1 -- 172.21.15.150:0/2338525654 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd100024780 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c19c0f0 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd0fb7fe700 1 -- 172.21.15.150:0/2338525654 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd100024cc0 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.390+0000 7fd0fb7fe700 1 -- 172.21.15.150:0/2338525654 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd1000193c0 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.391+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c10b590 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.393+0000 7fd0fb7fe700 1 -- 172.21.15.150:0/2338525654 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd10001ab40 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.465+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd10c19eb30 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.465+0000 7fd0fb7fe700 1 -- 172.21.15.150:0/2338525654 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd10001ace0 con 0x7fd10c07acf0 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.466+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 msgr2=0x7fd10c19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.466+0000 7fd1108f8700 1 --2- 172.21.15.150:0/2338525654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c19b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd100011710 tx=0x7fd100011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.466+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 shutdown_connections 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.466+0000 7fd1108f8700 1 --2- 172.21.15.150:0/2338525654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd10c07acf0 0x7fd10c19b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:35.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.466+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 >> 172.21.15.150:0/2338525654 conn(0x7fd10c1019d0 msgr2=0x7fd10c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:35.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.466+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 shutdown_connections 2024-05-10T04:32:35.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:35.466+0000 7fd1108f8700 1 -- 172.21.15.150:0/2338525654 wait complete. 2024-05-10T04:32:35.836 INFO:teuthology.orchestra.run.smithi150.stdout:mgr not available, waiting (3/15)... 2024-05-10T04:32:35.851 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:35 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2338525654' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T04:32:35.852 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:35 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:35.558+0000 7f1daec9b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T04:32:36.130 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:35 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:35.850+0000 7f1daec9b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:32:36.130 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:35 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:35.940+0000 7f1daec9b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T04:32:36.130 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:36 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:36.128+0000 7f1daec9b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T04:32:36.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: Activating manager daemon x 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: mgrmap e2: x(active, starting, since 0.00393376s) 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: Manager daemon x is now available 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' 2024-05-10T04:32:36.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:36 smithi150 ceph-mon[42404]: from='mgr.14100 172.21.15.150:0/1739666705' entity='mgr.x' 2024-05-10T04:32:38.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:38 smithi150 ceph-mon[42404]: mgrmap e3: x(active, since 1.00771s) 2024-05-10T04:32:38.818 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:38.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:32:38.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:32:38.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "health": { 2024-05-10T04:32:38.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-10T04:32:38.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-10T04:32:38.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 0 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "a" 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-10T04:32:38.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-10T04:32:38.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-10T04:32:38.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "restful" 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ], 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "modified": "2024-05-10T04:32:24.458680+0000", 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout }, 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.204+0000 7f97ae6fd700 1 Processor -- start 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.204+0000 7f97ae6fd700 1 -- start start 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.204+0000 7f97ae6fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a8107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:38.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.204+0000 7f97ae6fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a8107e40 con 0x7f97a8107440 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.204+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a8107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.204+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a8107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40212/0 (socket says 172.21.15.150:40212) 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.204+0000 7f97a7fff700 1 -- 172.21.15.150:0/2888351065 learned_addr learned my addr 172.21.15.150:0/2888351065 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.205+0000 7f97a7fff700 1 -- 172.21.15.150:0/2888351065 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8108570 con 0x7f97a8107440 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.205+0000 7f97a7fff700 1 --2- 172.21.15.150:0/2888351065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a8107850 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f979c0098a0 tx=0x7f979c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8751e87e0a12cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.205+0000 7f97a6ffd700 1 -- 172.21.15.150:0/2888351065 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f979c004160 con 0x7f97a8107440 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.205+0000 7f97a6ffd700 1 -- 172.21.15.150:0/2888351065 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f979c0042c0 con 0x7f97a8107440 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.205+0000 7f97a6ffd700 1 -- 172.21.15.150:0/2888351065 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f979c011a00 con 0x7f97a8107440 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 -- 172.21.15.150:0/2888351065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 msgr2=0x7f97a8107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:38.835 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 --2- 172.21.15.150:0/2888351065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a8107850 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f979c0098a0 tx=0x7f979c009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 -- 172.21.15.150:0/2888351065 shutdown_connections 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 --2- 172.21.15.150:0/2888351065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a8107850 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 -- 172.21.15.150:0/2888351065 >> 172.21.15.150:0/2888351065 conn(0x7f97a8102aa0 msgr2=0x7f97a8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 -- 172.21.15.150:0/2888351065 shutdown_connections 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 -- 172.21.15.150:0/2888351065 wait complete. 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 Processor -- start 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.206+0000 7f97ae6fd700 1 -- start start 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97ae6fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a807ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97ae6fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a807cfd0 con 0x7f97a8107440 2024-05-10T04:32:38.836 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a807ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a807ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40222/0 (socket says 172.21.15.150:40222) 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97a7fff700 1 -- 172.21.15.150:0/3324434454 learned_addr learned my addr 172.21.15.150:0/3324434454 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97a7fff700 1 -- 172.21.15.150:0/3324434454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c009550 con 0x7f97a8107440 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97a7fff700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a807ca90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f979c009130 tx=0x7f979c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.207+0000 7f97a57fa700 1 -- 172.21.15.150:0/3324434454 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f979c0038b0 con 0x7f97a8107440 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.208+0000 7f97a57fa700 1 -- 172.21.15.150:0/3324434454 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f979c003a10 con 0x7f97a8107440 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.208+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a807b220 con 0x7f97a8107440 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.208+0000 7f97a57fa700 1 -- 172.21.15.150:0/3324434454 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f979c019550 con 0x7f97a8107440 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.208+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a807b6c0 con 0x7f97a8107440 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.209+0000 7f97a57fa700 1 -- 172.21.15.150:0/3324434454 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7f979c0197c0 con 0x7f97a8107440 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.209+0000 7f97a57fa700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f9790038650 0x7f979003ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.209+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a8062840 con 0x7f97a8107440 2024-05-10T04:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.209+0000 7f97a57fa700 1 -- 172.21.15.150:0/3324434454 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f979c04c1a0 con 0x7f97a8107440 2024-05-10T04:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.209+0000 7f97a77fe700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f9790038650 0x7f979003ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.209+0000 7f97a77fe700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f9790038650 0x7f979003ab60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9798006fd0 tx=0x7f9798006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.214+0000 7f97a57fa700 1 -- 172.21.15.150:0/3324434454 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f979c019a70 con 0x7f97a8107440 2024-05-10T04:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.483+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f97a810b7a0 con 0x7f97a8107440 2024-05-10T04:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.483+0000 7f97a57fa700 1 -- 172.21.15.150:0/3324434454 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f979c018350 con 0x7f97a8107440 2024-05-10T04:32:38.838 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f9790038650 msgr2=0x7f979003ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f9790038650 0x7f979003ab60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9798006fd0 tx=0x7f9798006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 msgr2=0x7f97a807ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a807ca90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f979c009130 tx=0x7f979c011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 shutdown_connections 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f9790038650 0x7f979003ab60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 --2- 172.21.15.150:0/3324434454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a8107440 0x7f97a807ca90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 >> 172.21.15.150:0/3324434454 conn(0x7f97a8102aa0 msgr2=0x7f97a8103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:38.839 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 shutdown_connections 2024-05-10T04:32:38.840 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:38.485+0000 7f97ae6fd700 1 -- 172.21.15.150:0/3324434454 wait complete. 2024-05-10T04:32:38.840 INFO:teuthology.orchestra.run.smithi150.stdout:mgr is available 2024-05-10T04:32:39.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:39 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3324434454' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-10T04:32:39.691 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [global] 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout fsid = 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout [osd] 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-10T04:32:39.692 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.142+0000 7fc870276700 1 Processor -- start 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.142+0000 7fc870276700 1 -- start start 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.142+0000 7fc870276700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc868106790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.142+0000 7fc870276700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868106d80 con 0x7fc868106380 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86e012700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc868106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86e012700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc868106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40230/0 (socket says 172.21.15.150:40230) 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86e012700 1 -- 172.21.15.150:0/2165095647 learned_addr learned my addr 172.21.15.150:0/2165095647 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86e012700 1 -- 172.21.15.150:0/2165095647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8681074b0 con 0x7fc868106380 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86e012700 1 --2- 172.21.15.150:0/2165095647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc868106790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8640098a0 tx=0x7fc864009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c88bea1f915c8d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86d010700 1 -- 172.21.15.150:0/2165095647 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc864004160 con 0x7fc868106380 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86d010700 1 -- 172.21.15.150:0/2165095647 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8640042c0 con 0x7fc868106380 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.143+0000 7fc86d010700 1 -- 172.21.15.150:0/2165095647 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc864011a00 con 0x7fc868106380 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.144+0000 7fc870276700 1 -- 172.21.15.150:0/2165095647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 msgr2=0x7fc868106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:39.693 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.144+0000 7fc870276700 1 --2- 172.21.15.150:0/2165095647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc868106790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8640098a0 tx=0x7fc864009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.144+0000 7fc870276700 1 -- 172.21.15.150:0/2165095647 shutdown_connections 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.144+0000 7fc870276700 1 --2- 172.21.15.150:0/2165095647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc868106790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.144+0000 7fc870276700 1 -- 172.21.15.150:0/2165095647 >> 172.21.15.150:0/2165095647 conn(0x7fc868101a20 msgr2=0x7fc868103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.144+0000 7fc870276700 1 -- 172.21.15.150:0/2165095647 shutdown_connections 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.144+0000 7fc870276700 1 -- 172.21.15.150:0/2165095647 wait complete. 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc870276700 1 Processor -- start 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc870276700 1 -- start start 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc870276700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc86819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc870276700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86819a940 con 0x7fc868106380 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc86e012700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc86819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc86e012700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc86819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40244/0 (socket says 172.21.15.150:40244) 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc86e012700 1 -- 172.21.15.150:0/1556632554 learned_addr learned my addr 172.21.15.150:0/1556632554 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.145+0000 7fc86e012700 1 -- 172.21.15.150:0/1556632554 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009550 con 0x7fc868106380 2024-05-10T04:32:39.694 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.146+0000 7fc86e012700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc86819a400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc864009130 tx=0x7fc864011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.146+0000 7fc85f7fe700 1 -- 172.21.15.150:0/1556632554 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8640038b0 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.146+0000 7fc85f7fe700 1 -- 172.21.15.150:0/1556632554 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc864003a10 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.146+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86819ab40 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.146+0000 7fc85f7fe700 1 -- 172.21.15.150:0/1556632554 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc864019550 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.146+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86819afe0 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.147+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86810a6e0 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.150+0000 7fc85f7fe700 1 -- 172.21.15.150:0/1556632554 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fc864019810 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.150+0000 7fc85f7fe700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fc854038720 0x7fc85403ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.150+0000 7fc86d811700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fc854038720 0x7fc85403ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.151+0000 7fc86d811700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fc854038720 0x7fc85403ac30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc858006fd0 tx=0x7fc858006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.152+0000 7fc85f7fe700 1 -- 172.21.15.150:0/1556632554 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc86401a7b0 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.157+0000 7fc85f7fe700 1 -- 172.21.15.150:0/1556632554 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc86401a430 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.348+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc86819d7e0 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.349+0000 7fc85f7fe700 1 -- 172.21.15.150:0/1556632554 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fc86400bbd0 con 0x7fc868106380 2024-05-10T04:32:39.695 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fc854038720 msgr2=0x7fc85403ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fc854038720 0x7fc85403ac30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc858006fd0 tx=0x7fc858006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 msgr2=0x7fc86819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc86819a400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc864009130 tx=0x7fc864011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 shutdown_connections 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fc854038720 0x7fc85403ac30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 --2- 172.21.15.150:0/1556632554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc868106380 0x7fc86819a400 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.351+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 >> 172.21.15.150:0/1556632554 conn(0x7fc868101a20 msgr2=0x7fc8681023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.352+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 shutdown_connections 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:39.352+0000 7fc870276700 1 -- 172.21.15.150:0/1556632554 wait complete. 2024-05-10T04:32:39.696 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling cephadm module... 2024-05-10T04:32:40.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:40 smithi150 ceph-mon[42404]: mgrmap e4: x(active, since 3s) 2024-05-10T04:32:40.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:40 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1556632554' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-10T04:32:41.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:41 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3070178467' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-10T04:32:41.412 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: ignoring --setuser ceph since I am not root 2024-05-10T04:32:41.412 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: ignoring --setgroup ceph since I am not root 2024-05-10T04:32:41.515 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.054+0000 7fabda3c0700 1 Processor -- start 2024-05-10T04:32:41.515 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.054+0000 7fabda3c0700 1 -- start start 2024-05-10T04:32:41.515 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabda3c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd4106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:41.515 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabda3c0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd4106d60 con 0x7fabd4106360 2024-05-10T04:32:41.515 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabd3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd4106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:41.516 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabd3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd4106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40248/0 (socket says 172.21.15.150:40248) 2024-05-10T04:32:41.516 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabd3fff700 1 -- 172.21.15.150:0/4234869632 learned_addr learned my addr 172.21.15.150:0/4234869632 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:41.516 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabd3fff700 1 -- 172.21.15.150:0/4234869632 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4107490 con 0x7fabd4106360 2024-05-10T04:32:41.516 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabd3fff700 1 --2- 172.21.15.150:0/4234869632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd4106770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098a0 tx=0x7fabc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d2f0a52e2154f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:41.516 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.055+0000 7fabd2ffd700 1 -- 172.21.15.150:0/4234869632 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc4004160 con 0x7fabd4106360 2024-05-10T04:32:41.516 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabd2ffd700 1 -- 172.21.15.150:0/4234869632 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabc40042c0 con 0x7fabd4106360 2024-05-10T04:32:41.517 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabd2ffd700 1 -- 172.21.15.150:0/4234869632 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc4011a00 con 0x7fabd4106360 2024-05-10T04:32:41.517 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabda3c0700 1 -- 172.21.15.150:0/4234869632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 msgr2=0x7fabd4106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:41.517 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabda3c0700 1 --2- 172.21.15.150:0/4234869632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd4106770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098a0 tx=0x7fabc4009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:41.517 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabda3c0700 1 -- 172.21.15.150:0/4234869632 shutdown_connections 2024-05-10T04:32:41.517 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabda3c0700 1 --2- 172.21.15.150:0/4234869632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd4106770 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:41.517 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabda3c0700 1 -- 172.21.15.150:0/4234869632 >> 172.21.15.150:0/4234869632 conn(0x7fabd41019c0 msgr2=0x7fabd4103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:41.518 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabda3c0700 1 -- 172.21.15.150:0/4234869632 shutdown_connections 2024-05-10T04:32:41.518 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.056+0000 7fabda3c0700 1 -- 172.21.15.150:0/4234869632 wait complete. 2024-05-10T04:32:41.518 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.057+0000 7fabda3c0700 1 Processor -- start 2024-05-10T04:32:41.518 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.057+0000 7fabda3c0700 1 -- start start 2024-05-10T04:32:41.518 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.057+0000 7fabda3c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd419b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:41.518 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.057+0000 7fabda3c0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd419bad0 con 0x7fabd4106360 2024-05-10T04:32:41.519 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd419b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:41.519 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd419b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40258/0 (socket says 172.21.15.150:40258) 2024-05-10T04:32:41.519 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd3fff700 1 -- 172.21.15.150:0/3070178467 learned_addr learned my addr 172.21.15.150:0/3070178467 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:41.519 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd3fff700 1 -- 172.21.15.150:0/3070178467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4009550 con 0x7fabd4106360 2024-05-10T04:32:41.519 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd3fff700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd419b590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009130 tx=0x7fabc4004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc40039e0 con 0x7fabd4106360 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabc4003b40 con 0x7fabd4106360 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabc4019690 con 0x7fabd4106360 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd419bcd0 con 0x7fabd4106360 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.058+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd419c170 con 0x7fabd4106360 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.059+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fabc4019910 con 0x7fabd4106360 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.059+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd4194c00 con 0x7fabd4106360 2024-05-10T04:32:41.520 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.059+0000 7fabd17fa700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fabbc038760 0x7fabbc03ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.059+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fabc404c810 con 0x7fabd4106360 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.059+0000 7fabd37fe700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fabbc038760 0x7fabbc03ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.060+0000 7fabd37fe700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fabbc038760 0x7fabbc03ac70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.064+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fabc4018d90 con 0x7fabd4106360 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:40.312+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fabd410a6c0 con 0x7fabd4106360 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.156+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fabc400b900 con 0x7fabd4106360 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.158+0000 7fabd17fa700 1 -- 172.21.15.150:0/3070178467 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fabc404c230 con 0x7fabd4106360 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fabbc038760 msgr2=0x7fabbc03ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fabbc038760 0x7fabbc03ac70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006fd0 tx=0x7fabc8006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 msgr2=0x7fabd419b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd419b590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009130 tx=0x7fabc4004250 comp rx=0 tx=0).stop 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 shutdown_connections 2024-05-10T04:32:41.521 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7fabbc038760 0x7fabbc03ac70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:41.522 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 --2- 172.21.15.150:0/3070178467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd4106360 0x7fabd419b590 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:41.522 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 >> 172.21.15.150:0/3070178467 conn(0x7fabd41019c0 msgr2=0x7fabd4191d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:41.522 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 shutdown_connections 2024-05-10T04:32:41.522 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.161+0000 7fabda3c0700 1 -- 172.21.15.150:0/3070178467 wait complete. 2024-05-10T04:32:41.736 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:41.638+0000 7fb59cc49200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T04:32:41.736 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:41.734+0000 7fb59cc49200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T04:32:42.156 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:41.826+0000 7fb59cc49200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T04:32:42.157 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:41.922+0000 7fb59cc49200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T04:32:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:42 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3070178467' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-10T04:32:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:42 smithi150 ceph-mon[42404]: mgrmap e5: x(active, since 5s) 2024-05-10T04:32:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:42 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2682751280' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T04:32:42.507 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:32:42.507 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.878+0000 7febe684d700 1 Processor -- start 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.878+0000 7febe684d700 1 -- start start 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.878+0000 7febe684d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe0109f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.878+0000 7febe684d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe010a520 con 0x7febe007acf0 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.878+0000 7febdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe0109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.879+0000 7febdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe0109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40290/0 (socket says 172.21.15.150:40290) 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.879+0000 7febdffff700 1 -- 172.21.15.150:0/1073336255 learned_addr learned my addr 172.21.15.150:0/1073336255 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.879+0000 7febdffff700 1 -- 172.21.15.150:0/1073336255 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe010a660 con 0x7febe007acf0 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.879+0000 7febdffff700 1 --2- 172.21.15.150:0/1073336255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe0109f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7febd00098a0 tx=0x7febd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94403963b828cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.879+0000 7febdeffd700 1 -- 172.21.15.150:0/1073336255 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febd0004160 con 0x7febe007acf0 2024-05-10T04:32:42.508 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.879+0000 7febdeffd700 1 -- 172.21.15.150:0/1073336255 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7febd00042c0 con 0x7febe007acf0 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.879+0000 7febdeffd700 1 -- 172.21.15.150:0/1073336255 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febd0011a00 con 0x7febe007acf0 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.880+0000 7febe684d700 1 -- 172.21.15.150:0/1073336255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 msgr2=0x7febe0109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.880+0000 7febe684d700 1 --2- 172.21.15.150:0/1073336255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe0109f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7febd00098a0 tx=0x7febd0009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.880+0000 7febe684d700 1 -- 172.21.15.150:0/1073336255 shutdown_connections 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.880+0000 7febe684d700 1 --2- 172.21.15.150:0/1073336255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe0109f10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.880+0000 7febe684d700 1 -- 172.21.15.150:0/1073336255 >> 172.21.15.150:0/1073336255 conn(0x7febe0101980 msgr2=0x7febe0103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.880+0000 7febe684d700 1 -- 172.21.15.150:0/1073336255 shutdown_connections 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.880+0000 7febe684d700 1 -- 172.21.15.150:0/1073336255 wait complete. 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.881+0000 7febe684d700 1 Processor -- start 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.881+0000 7febe684d700 1 -- start start 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.881+0000 7febe684d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.881+0000 7febe684d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe019ba80 con 0x7febe007acf0 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.881+0000 7febdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.881+0000 7febdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe019b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40294/0 (socket says 172.21.15.150:40294) 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.881+0000 7febdffff700 1 -- 172.21.15.150:0/2682751280 learned_addr learned my addr 172.21.15.150:0/2682751280 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.882+0000 7febdffff700 1 -- 172.21.15.150:0/2682751280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd0009550 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.882+0000 7febdffff700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe019b540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7febd0009130 tx=0x7febd0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.882+0000 7febdd7fa700 1 -- 172.21.15.150:0/2682751280 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febd00038b0 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.882+0000 7febdd7fa700 1 -- 172.21.15.150:0/2682751280 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7febd0003a10 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.882+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe019bc80 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.882+0000 7febdd7fa700 1 -- 172.21.15.150:0/2682751280 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febd0019550 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.882+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe019c120 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.883+0000 7febdd7fa700 1 -- 172.21.15.150:0/2682751280 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7febd0019780 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.883+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe010b520 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.883+0000 7febdd7fa700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 0x7febc803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.883+0000 7febdd7fa700 1 -- 172.21.15.150:0/2682751280 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7febd004c520 con 0x7febe007acf0 2024-05-10T04:32:42.510 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.883+0000 7febdf7fe700 1 -- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 msgr2=0x7febc803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/2970915543 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.883+0000 7febdf7fe700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 0x7febc803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:41.892+0000 7febdd7fa700 1 -- 172.21.15.150:0/2682751280 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7febd002b340 con 0x7febe007acf0 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.084+0000 7febdf7fe700 1 -- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 msgr2=0x7febc803acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/2970915543 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.084+0000 7febdf7fe700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 0x7febc803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.149+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7febe019e920 con 0x7febe007acf0 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.150+0000 7febdd7fa700 1 -- 172.21.15.150:0/2682751280 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7febd0018b40 con 0x7febe007acf0 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 msgr2=0x7febc803acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 0x7febc803acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 msgr2=0x7febe019b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe019b540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7febd0009130 tx=0x7febd0011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 shutdown_connections 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7febc80387c0 0x7febc803acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 --2- 172.21.15.150:0/2682751280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febe007acf0 0x7febe019b540 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:42.511 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 >> 172.21.15.150:0/2682751280 conn(0x7febe0101980 msgr2=0x7febe01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:42.512 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 shutdown_connections 2024-05-10T04:32:42.512 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.152+0000 7febe684d700 1 -- 172.21.15.150:0/2682751280 wait complete. 2024-05-10T04:32:42.512 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for the mgr to restart... 2024-05-10T04:32:42.512 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr epoch 5... 2024-05-10T04:32:42.861 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:42.580+0000 7fb59cc49200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:32:43.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:42.860+0000 7fb59cc49200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T04:32:43.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:42.949+0000 7fb59cc49200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T04:32:43.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:43.059+0000 7fb59cc49200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T04:32:43.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:43.219+0000 7fb59cc49200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T04:32:43.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:43.346+0000 7fb59cc49200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T04:32:43.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:43.433+0000 7fb59cc49200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T04:32:44.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:43.804+0000 7fb59cc49200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T04:32:44.493 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:44.201+0000 7fb59cc49200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T04:32:44.493 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:44.398+0000 7fb59cc49200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T04:32:44.493 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:44.491+0000 7fb59cc49200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T04:32:45.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:45.265+0000 7fb59cc49200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T04:32:45.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:45.362+0000 7fb59cc49200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T04:32:45.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:45.512+0000 7fb59cc49200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T04:32:46.411 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:46.143+0000 7fb59cc49200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T04:32:46.411 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:46.327+0000 7fb59cc49200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T04:32:46.911 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:46.524+0000 7fb59cc49200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T04:32:47.749 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:47.458+0000 7fb59cc49200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T04:32:48.030 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:47.749+0000 7fb59cc49200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:32:48.031 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:47.839+0000 7fb59cc49200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T04:32:48.411 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:48 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:32:48.030+0000 7fb59cc49200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: Active manager daemon x restarted 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: Activating manager daemon x 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: osdmap e2: 0 total, 0 up, 0 in 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: mgrmap e6: x(active, starting, since 0.00601694s) 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: Manager daemon x is now available 2024-05-10T04:32:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:48 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:32:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: Found migration_current of "None". Setting to last migration. 2024-05-10T04:32:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:32:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:32:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:32:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T04:32:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T04:32:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:32:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:32:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:32:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:49 smithi150 ceph-mon[42404]: mgrmap e7: x(active, since 1.00728s) 2024-05-10T04:32:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:50 smithi150 ceph-mon[42404]: [10/May/2024:04:32:48] ENGINE Bus STARTING 2024-05-10T04:32:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:50 smithi150 ceph-mon[42404]: [10/May/2024:04:32:48] ENGINE Serving on https://172.21.15.150:7150 2024-05-10T04:32:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:50 smithi150 ceph-mon[42404]: [10/May/2024:04:32:49] ENGINE Serving on http://172.21.15.150:8765 2024-05-10T04:32:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:50 smithi150 ceph-mon[42404]: [10/May/2024:04:32:49] ENGINE Bus STARTED 2024-05-10T04:32:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:50 smithi150 ceph-mon[42404]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T04:32:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:50 smithi150 ceph-mon[42404]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.845+0000 7f77582db700 1 Processor -- start 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.845+0000 7f77582db700 1 -- start start 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.846+0000 7f77582db700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750107c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.846+0000 7f77582db700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750108260 con 0x7f7750107880 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.846+0000 7f7756077700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750107c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.846+0000 7f7756077700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750107c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40300/0 (socket says 172.21.15.150:40300) 2024-05-10T04:32:50.967 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.846+0000 7f7756077700 1 -- 172.21.15.150:0/519852858 learned_addr learned my addr 172.21.15.150:0/519852858 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.846+0000 7f7756077700 1 -- 172.21.15.150:0/519852858 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77501083a0 con 0x7f7750107880 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.846+0000 7f7756077700 1 --2- 172.21.15.150:0/519852858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750107c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f774c00bb40 tx=0x7f774c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d20d23ff74f855 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.847+0000 7f7755075700 1 -- 172.21.15.150:0/519852858 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f774c00dbe0 con 0x7f7750107880 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.847+0000 7f7755075700 1 -- 172.21.15.150:0/519852858 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f774c0044d0 con 0x7f7750107880 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.847+0000 7f7755075700 1 -- 172.21.15.150:0/519852858 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f774c010470 con 0x7f7750107880 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.847+0000 7f77582db700 1 -- 172.21.15.150:0/519852858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 msgr2=0x7f7750107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.847+0000 7f77582db700 1 --2- 172.21.15.150:0/519852858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750107c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f774c00bb40 tx=0x7f774c00be50 comp rx=0 tx=0).stop 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 -- 172.21.15.150:0/519852858 shutdown_connections 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 --2- 172.21.15.150:0/519852858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750107c90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 -- 172.21.15.150:0/519852858 >> 172.21.15.150:0/519852858 conn(0x7f7750101180 msgr2=0x7f77501035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 -- 172.21.15.150:0/519852858 shutdown_connections 2024-05-10T04:32:50.968 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 -- 172.21.15.150:0/519852858 wait complete. 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 Processor -- start 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 -- start start 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.848+0000 7f77582db700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f77582db700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750199770 con 0x7f7750107880 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f7756077700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f7756077700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40314/0 (socket says 172.21.15.150:40314) 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f7756077700 1 -- 172.21.15.150:0/3714001951 learned_addr learned my addr 172.21.15.150:0/3714001951 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f7756077700 1 -- 172.21.15.150:0/3714001951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c00b7f0 con 0x7f7750107880 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f7756077700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750199230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f774c003c60 tx=0x7f774c003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f774c01e070 con 0x7f7750107880 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750199970 con 0x7f7750107880 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f774c004250 con 0x7f7750107880 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f774c022900 con 0x7f7750107880 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.849+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7750199e10 con 0x7f7750107880 2024-05-10T04:32:50.969 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.850+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f774c02c440 con 0x7f7750107880 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.850+0000 7f77477fe700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.850+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 --> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f773c03b3e0 con 0x7f773c0387c0 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.850+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f774c04f660 con 0x7f7750107880 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.850+0000 7f7755876700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 msgr2=0x7f773c03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/2970915543 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:42.850+0000 7f7755876700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:43.051+0000 7f7755876700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 msgr2=0x7f773c03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/2970915543 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:43.051+0000 7f7755876700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:43.452+0000 7f7755876700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 msgr2=0x7f773c03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/2970915543 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:43.452+0000 7f7755876700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:44.254+0000 7f7755876700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 msgr2=0x7f773c03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/2970915543 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:44.254+0000 7f7755876700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:45.855+0000 7f7755876700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 msgr2=0x7f773c03acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/2970915543 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:45.855+0000 7f7755876700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T04:32:50.970 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:48.048+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f774c016d40 con 0x7f7750107880 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:48.048+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 msgr2=0x7f773c03acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:48.048+0000 7f77477fe700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.047+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f774c02c6f0 con 0x7f7750107880 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.047+0000 7f77477fe700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f773c03bfb0 0x7f773c03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.047+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f773c03b3e0 con 0x7f773c03bfb0 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.048+0000 7f7755876700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f773c03bfb0 0x7f773c03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.049+0000 7f7755876700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f773c03bfb0 0x7f773c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7740003a10 tx=0x7f7740009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.049+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f773c03b3e0 con 0x7f773c03bfb0 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.056+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7750062840 con 0x7f773c03bfb0 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77477fe700 1 -- 172.21.15.150:0/3714001951 <== mgr.14120 v2:172.21.15.150:6800/881426482 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7750062840 con 0x7f773c03bfb0 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f773c03bfb0 msgr2=0x7f773c03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f773c03bfb0 0x7f773c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7740003a10 tx=0x7f7740009290 comp rx=0 tx=0).stop 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 msgr2=0x7f7750199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:50.971 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750199230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f774c003c60 tx=0x7f774c003980 comp rx=0 tx=0).stop 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 shutdown_connections 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f773c03bfb0 0x7f773c03e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7750107880 0x7f7750199230 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 --2- 172.21.15.150:0/3714001951 >> [v2:172.21.15.150:6800/2970915543,v1:172.21.15.150:6801/2970915543] conn(0x7f773c0387c0 0x7f773c03acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 >> 172.21.15.150:0/3714001951 conn(0x7f7750101180 msgr2=0x7f775018fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 shutdown_connections 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:49.057+0000 7f77582db700 1 -- 172.21.15.150:0/3714001951 wait complete. 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:mgr epoch 5 is available 2024-05-10T04:32:50.972 INFO:teuthology.orchestra.run.smithi150.stdout:Setting orchestrator backend to cephadm... 2024-05-10T04:32:51.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:51 smithi150 ceph-mon[42404]: mgrmap e8: x(active, since 2s) 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.243+0000 7efd887fd700 1 Processor -- start 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.243+0000 7efd887fd700 1 -- start start 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.243+0000 7efd887fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8010a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.243+0000 7efd887fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8010a5e0 con 0x7efd8007ac50 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.243+0000 7efd86599700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8010a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.243+0000 7efd86599700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8010a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54576/0 (socket says 172.21.15.150:54576) 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.243+0000 7efd86599700 1 -- 172.21.15.150:0/3922515694 learned_addr learned my addr 172.21.15.150:0/3922515694 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.244+0000 7efd86599700 1 -- 172.21.15.150:0/3922515694 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8010a720 con 0x7efd8007ac50 2024-05-10T04:32:53.456 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.244+0000 7efd86599700 1 --2- 172.21.15.150:0/3922515694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8010a010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0098a0 tx=0x7efd7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65443ca430c32386 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.244+0000 7efd85597700 1 -- 172.21.15.150:0/3922515694 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd7c004160 con 0x7efd8007ac50 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.244+0000 7efd85597700 1 -- 172.21.15.150:0/3922515694 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd7c0042c0 con 0x7efd8007ac50 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.244+0000 7efd85597700 1 -- 172.21.15.150:0/3922515694 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd7c011a00 con 0x7efd8007ac50 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.245+0000 7efd887fd700 1 -- 172.21.15.150:0/3922515694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 msgr2=0x7efd8010a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.245+0000 7efd887fd700 1 --2- 172.21.15.150:0/3922515694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8010a010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0098a0 tx=0x7efd7c009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.245+0000 7efd887fd700 1 -- 172.21.15.150:0/3922515694 shutdown_connections 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.245+0000 7efd887fd700 1 --2- 172.21.15.150:0/3922515694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8010a010 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.245+0000 7efd887fd700 1 -- 172.21.15.150:0/3922515694 >> 172.21.15.150:0/3922515694 conn(0x7efd801019c0 msgr2=0x7efd80103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.245+0000 7efd887fd700 1 -- 172.21.15.150:0/3922515694 shutdown_connections 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.245+0000 7efd887fd700 1 -- 172.21.15.150:0/3922515694 wait complete. 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd887fd700 1 Processor -- start 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd887fd700 1 -- start start 2024-05-10T04:32:53.457 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd887fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd887fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8019ba60 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd86599700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd86599700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54590/0 (socket says 172.21.15.150:54590) 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd86599700 1 -- 172.21.15.150:0/3249418493 learned_addr learned my addr 172.21.15.150:0/3249418493 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd86599700 1 -- 172.21.15.150:0/3249418493 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c009550 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.246+0000 7efd86599700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8019b520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009130 tx=0x7efd7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.247+0000 7efd777fe700 1 -- 172.21.15.150:0/3249418493 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd7c0038b0 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.247+0000 7efd777fe700 1 -- 172.21.15.150:0/3249418493 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd7c003a10 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.247+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8019bc60 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.247+0000 7efd777fe700 1 -- 172.21.15.150:0/3249418493 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd7c019550 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.247+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8019c100 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.247+0000 7efd777fe700 1 -- 172.21.15.150:0/3249418493 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7efd7c0197d0 con 0x7efd8007ac50 2024-05-10T04:32:53.458 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.247+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8010b580 con 0x7efd8007ac50 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.248+0000 7efd777fe700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7efd6c038770 0x7efd6c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.248+0000 7efd777fe700 1 -- 172.21.15.150:0/3249418493 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efd7c04c800 con 0x7efd8007ac50 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.248+0000 7efd85d98700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7efd6c038770 0x7efd6c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.248+0000 7efd85d98700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7efd6c038770 0x7efd6c03ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd70006fd0 tx=0x7efd70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.253+0000 7efd777fe700 1 -- 172.21.15.150:0/3249418493 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efd7c018d20 con 0x7efd8007ac50 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.467+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7efd80102a30 con 0x7efd6c038770 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.477+0000 7efd777fe700 1 -- 172.21.15.150:0/3249418493 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efd80102a30 con 0x7efd6c038770 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.479+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7efd6c038770 msgr2=0x7efd6c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.479+0000 7efd887fd700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7efd6c038770 0x7efd6c03ac80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd70006fd0 tx=0x7efd70006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.479+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 msgr2=0x7efd8019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.479+0000 7efd887fd700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8019b520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009130 tx=0x7efd7c011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.480+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 shutdown_connections 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.480+0000 7efd887fd700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7efd6c038770 0x7efd6c03ac80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:53.459 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.480+0000 7efd887fd700 1 --2- 172.21.15.150:0/3249418493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd8007ac50 0x7efd8019b520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:53.460 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.480+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 >> 172.21.15.150:0/3249418493 conn(0x7efd801019c0 msgr2=0x7efd801022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:53.460 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.480+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 shutdown_connections 2024-05-10T04:32:53.460 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:52.480+0000 7efd887fd700 1 -- 172.21.15.150:0/3249418493 wait complete. 2024-05-10T04:32:53.547 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:53 smithi150 ceph-mon[42404]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:32:53.547 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:53 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:32:53.547 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:53 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:32:54.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-10T04:32:54.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.800+0000 7f5e0d7c4700 1 Processor -- start 2024-05-10T04:32:54.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.800+0000 7f5e0d7c4700 1 -- start start 2024-05-10T04:32:54.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.800+0000 7f5e0d7c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e08107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:54.347 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.800+0000 7f5e0d7c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e08107e80 con 0x7f5e08107480 2024-05-10T04:32:54.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e06ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e08107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:54.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e06ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e08107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54598/0 (socket says 172.21.15.150:54598) 2024-05-10T04:32:54.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e06ffd700 1 -- 172.21.15.150:0/3185084211 learned_addr learned my addr 172.21.15.150:0/3185084211 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:54.348 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e06ffd700 1 -- 172.21.15.150:0/3185084211 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e081085b0 con 0x7f5e08107480 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e06ffd700 1 --2- 172.21.15.150:0/3185084211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e08107890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0098a0 tx=0x7f5dfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2492cacd30362f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e05ffb700 1 -- 172.21.15.150:0/3185084211 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc004160 con 0x7f5e08107480 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e05ffb700 1 -- 172.21.15.150:0/3185084211 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5dfc0042c0 con 0x7f5e08107480 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.801+0000 7f5e05ffb700 1 -- 172.21.15.150:0/3185084211 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc011a00 con 0x7f5e08107480 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.802+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/3185084211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 msgr2=0x7f5e08107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.802+0000 7f5e0d7c4700 1 --2- 172.21.15.150:0/3185084211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e08107890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0098a0 tx=0x7f5dfc009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.802+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/3185084211 shutdown_connections 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.802+0000 7f5e0d7c4700 1 --2- 172.21.15.150:0/3185084211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e08107890 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.802+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/3185084211 >> 172.21.15.150:0/3185084211 conn(0x7f5e08102ae0 msgr2=0x7f5e08104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.802+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/3185084211 shutdown_connections 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.802+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/3185084211 wait complete. 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e0d7c4700 1 Processor -- start 2024-05-10T04:32:54.349 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e0d7c4700 1 -- start start 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e0d7c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e0819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e0d7c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0819b9e0 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e06ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e0819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e06ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e0819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54604/0 (socket says 172.21.15.150:54604) 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e06ffd700 1 -- 172.21.15.150:0/2537747870 learned_addr learned my addr 172.21.15.150:0/2537747870 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.803+0000 7f5e06ffd700 1 -- 172.21.15.150:0/2537747870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc009550 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.804+0000 7f5e06ffd700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e0819b4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009130 tx=0x7f5dfc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.804+0000 7f5deffff700 1 -- 172.21.15.150:0/2537747870 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc0038b0 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.804+0000 7f5deffff700 1 -- 172.21.15.150:0/2537747870 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5dfc003a10 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.804+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0819bbe0 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.804+0000 7f5deffff700 1 -- 172.21.15.150:0/2537747870 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc019550 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.804+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0819c080 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.805+0000 7f5deffff700 1 -- 172.21.15.150:0/2537747870 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5dfc0197c0 con 0x7f5e08107480 2024-05-10T04:32:54.350 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.805+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0810b7e0 con 0x7f5e08107480 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.805+0000 7f5deffff700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f5df00387c0 0x7f5df003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.805+0000 7f5deffff700 1 -- 172.21.15.150:0/2537747870 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5dfc04c760 con 0x7f5e08107480 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.805+0000 7f5e067fc700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f5df00387c0 0x7f5df003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.806+0000 7f5e067fc700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f5df00387c0 0x7f5df003acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5df8006fd0 tx=0x7f5df8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:53.810+0000 7f5deffff700 1 -- 172.21.15.150:0/2537747870 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5dfc0189a0 con 0x7f5e08107480 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.001+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e08103b00 con 0x7f5df00387c0 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.002+0000 7f5deffff700 1 -- 172.21.15.150:0/2537747870 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5e08103b00 con 0x7f5df00387c0 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f5df00387c0 msgr2=0x7f5df003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f5df00387c0 0x7f5df003acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5df8006fd0 tx=0x7f5df8006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 msgr2=0x7f5e0819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e0819b4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009130 tx=0x7f5dfc011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:54.351 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 shutdown_connections 2024-05-10T04:32:54.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f5df00387c0 0x7f5df003acd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:54.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 --2- 172.21.15.150:0/2537747870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e08107480 0x7f5e0819b4a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:54.352 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 >> 172.21.15.150:0/2537747870 conn(0x7f5e08102ae0 msgr2=0x7f5e081033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:54.356 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 shutdown_connections 2024-05-10T04:32:54.356 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.004+0000 7f5e0d7c4700 1 -- 172.21.15.150:0/2537747870 wait complete. 2024-05-10T04:32:54.356 INFO:teuthology.orchestra.run.smithi150.stdout:Generating ssh key... 2024-05-10T04:32:54.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:54 smithi150 ceph-mon[42404]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:32:55.310 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: Generating public/private rsa key pair. 2024-05-10T04:32:55.310 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: Your identification has been saved in /tmp/tmpxl4jdbhe/key. 2024-05-10T04:32:55.310 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: Your public key has been saved in /tmp/tmpxl4jdbhe/key.pub. 2024-05-10T04:32:55.310 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: The key fingerprint is: 2024-05-10T04:32:55.310 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: SHA256:SZ5BiEGO0SnHAN9I5htV4Iz/hNn8jhLKSVHyFdp7moY ceph-28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:55.310 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: The key's randomart image is: 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: +---[RSA 3072]----+ 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: |..===*oo. | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | =o@Boo. | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | B**.. o | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | .+.= + + | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | ..+ = S | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | . .+ = | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | o oE.= . | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | + .. o | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: | .. . | 2024-05-10T04:32:55.311 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:32:55 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: +----[SHA256]-----+ 2024-05-10T04:32:55.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4cd8b700 1 Processor -- start 2024-05-10T04:32:55.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4cd8b700 1 -- start start 2024-05-10T04:32:55.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4cd8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e48109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:55.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4cd8b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4810a590 con 0x7f3e4807acd0 2024-05-10T04:32:55.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e48109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:55.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e48109fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54608/0 (socket says 172.21.15.150:54608) 2024-05-10T04:32:55.380 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4659c700 1 -- 172.21.15.150:0/1615213050 learned_addr learned my addr 172.21.15.150:0/1615213050 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.698+0000 7f3e4659c700 1 -- 172.21.15.150:0/1615213050 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4810a6d0 con 0x7f3e4807acd0 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.699+0000 7f3e4659c700 1 --2- 172.21.15.150:0/1615213050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e48109fa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e380098a0 tx=0x7f3e38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb84765e1cdfb26b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.699+0000 7f3e4559a700 1 -- 172.21.15.150:0/1615213050 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e38004160 con 0x7f3e4807acd0 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.699+0000 7f3e4559a700 1 -- 172.21.15.150:0/1615213050 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e380042c0 con 0x7f3e4807acd0 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.699+0000 7f3e4559a700 1 -- 172.21.15.150:0/1615213050 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e38011a00 con 0x7f3e4807acd0 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.699+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1615213050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 msgr2=0x7f3e48109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.699+0000 7f3e4cd8b700 1 --2- 172.21.15.150:0/1615213050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e48109fa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e380098a0 tx=0x7f3e38009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.700+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1615213050 shutdown_connections 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.700+0000 7f3e4cd8b700 1 --2- 172.21.15.150:0/1615213050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e48109fa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.700+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1615213050 >> 172.21.15.150:0/1615213050 conn(0x7f3e481019d0 msgr2=0x7f3e48103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.700+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1615213050 shutdown_connections 2024-05-10T04:32:55.381 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.700+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1615213050 wait complete. 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.700+0000 7f3e4cd8b700 1 Processor -- start 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.700+0000 7f3e4cd8b700 1 -- start start 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4cd8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e4819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4cd8b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4819ba30 con 0x7f3e4807acd0 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e4819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e4819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54616/0 (socket says 172.21.15.150:54616) 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4659c700 1 -- 172.21.15.150:0/1176094482 learned_addr learned my addr 172.21.15.150:0/1176094482 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4659c700 1 -- 172.21.15.150:0/1176094482 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38009550 con 0x7f3e4807acd0 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4659c700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e4819b4f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e38011710 tx=0x7f3e38011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e3f7fe700 1 -- 172.21.15.150:0/1176094482 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e38003af0 con 0x7f3e4807acd0 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e3f7fe700 1 -- 172.21.15.150:0/1176094482 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e38003c50 con 0x7f3e4807acd0 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4819bc30 con 0x7f3e4807acd0 2024-05-10T04:32:55.382 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e3f7fe700 1 -- 172.21.15.150:0/1176094482 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e38024780 con 0x7f3e4807acd0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.701+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4819c0d0 con 0x7f3e4807acd0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.702+0000 7f3e3f7fe700 1 -- 172.21.15.150:0/1176094482 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3e380249c0 con 0x7f3e4807acd0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.702+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4810b590 con 0x7f3e4807acd0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.702+0000 7f3e3f7fe700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f3e340387c0 0x7f3e3403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.702+0000 7f3e3f7fe700 1 -- 172.21.15.150:0/1176094482 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3e3804c870 con 0x7f3e4807acd0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.703+0000 7f3e45d9b700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f3e340387c0 0x7f3e3403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.703+0000 7f3e45d9b700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f3e340387c0 0x7f3e3403acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006fd0 tx=0x7f3e30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.707+0000 7f3e3f7fe700 1 -- 172.21.15.150:0/1176094482 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3e38018d20 con 0x7f3e4807acd0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:54.901+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e4819e8d0 con 0x7f3e340387c0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.044+0000 7f3e3f7fe700 1 -- 172.21.15.150:0/1176094482 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3e4819e8d0 con 0x7f3e340387c0 2024-05-10T04:32:55.383 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f3e340387c0 msgr2=0x7f3e3403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f3e340387c0 0x7f3e3403acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006fd0 tx=0x7f3e30006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 msgr2=0x7f3e4819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e4819b4f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e38011710 tx=0x7f3e38011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 shutdown_connections 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f3e340387c0 0x7f3e3403acd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 --2- 172.21.15.150:0/1176094482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e4807acd0 0x7f3e4819b4f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.046+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 >> 172.21.15.150:0/1176094482 conn(0x7f3e481019d0 msgr2=0x7f3e48102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.047+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 shutdown_connections 2024-05-10T04:32:55.384 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.047+0000 7f3e4cd8b700 1 -- 172.21.15.150:0/1176094482 wait complete. 2024-05-10T04:32:56.243 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:56 smithi150 ceph-mon[42404]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:32:56.243 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:56 smithi150 ceph-mon[42404]: Generating ssh key... 2024-05-10T04:32:56.244 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:56 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:32:56.244 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:56 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe0KzPJi3dfGbQEJEvpk6InMc5wxSLusLVu3ZTnAwu6n2HHH0FznMmtN8Xka9eeVBapg+AK5TFnLG59ssVsFhAXV0gyVaTHCKfPUWCadAsFaJwP1dNJISXqndcsSl1xv2dzZe5bY6tBjTyL9yuIQ5T6H6hcPhXrV5w8Zi5ovE8/wXdNQ0AEmGxjMDwSqy7J85JYahEgnLDUysyHkGtd3kNEORW7Lvk62uBvmIZgWoOCL1izEXPb8quAbazGlRvyghjK3fdLjDo56wUKUUD5dx0fh6e7qn5zI+kp5WAsqbRrmSJFNhkqiBzpZYziyhNsUh+S/G7Sv6RPDko95lyUTsBH/gd5Cu87QH7F0+MnHOnSANNEernkJnZ03jhC7fSPBC2wfTFRb6x29PdBL4/MA9UyqUcM3v/K4XBq1Bo5eEnwNTcGhJ1V1QEPaQPerFeM3Em4x8+1JPIWD5uhE7vJHfQuGW76Tex/l0vgOvJiFxD7gacWI99ng60S4KM2FV7sls= ceph-28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.740+0000 7ff84cba5700 1 Processor -- start 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.741+0000 7ff84cba5700 1 -- start start 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.741+0000 7ff84cba5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff848107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.741+0000 7ff84cba5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848107e50 con 0x7ff848107450 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.741+0000 7ff84659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff848107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.741+0000 7ff84659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff848107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54632/0 (socket says 172.21.15.150:54632) 2024-05-10T04:32:56.338 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.741+0000 7ff84659c700 1 -- 172.21.15.150:0/2670311455 learned_addr learned my addr 172.21.15.150:0/2670311455 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.741+0000 7ff84659c700 1 -- 172.21.15.150:0/2670311455 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff848108580 con 0x7ff848107450 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.742+0000 7ff84659c700 1 --2- 172.21.15.150:0/2670311455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff848107860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8300098a0 tx=0x7ff830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459f18050948ce75 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.742+0000 7ff84559a700 1 -- 172.21.15.150:0/2670311455 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff830004160 con 0x7ff848107450 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.742+0000 7ff84559a700 1 -- 172.21.15.150:0/2670311455 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8300042c0 con 0x7ff848107450 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.742+0000 7ff84559a700 1 -- 172.21.15.150:0/2670311455 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff830011a00 con 0x7ff848107450 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.742+0000 7ff84cba5700 1 -- 172.21.15.150:0/2670311455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 msgr2=0x7ff848107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.742+0000 7ff84cba5700 1 --2- 172.21.15.150:0/2670311455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff848107860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8300098a0 tx=0x7ff830009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.743+0000 7ff84cba5700 1 -- 172.21.15.150:0/2670311455 shutdown_connections 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.743+0000 7ff84cba5700 1 --2- 172.21.15.150:0/2670311455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff848107860 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.743+0000 7ff84cba5700 1 -- 172.21.15.150:0/2670311455 >> 172.21.15.150:0/2670311455 conn(0x7ff848102ad0 msgr2=0x7ff848104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.743+0000 7ff84cba5700 1 -- 172.21.15.150:0/2670311455 shutdown_connections 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.743+0000 7ff84cba5700 1 -- 172.21.15.150:0/2670311455 wait complete. 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.743+0000 7ff84cba5700 1 Processor -- start 2024-05-10T04:32:56.339 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.743+0000 7ff84cba5700 1 -- start start 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84cba5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff84819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84cba5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84819ba10 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff84819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff84819b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54646/0 (socket says 172.21.15.150:54646) 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84659c700 1 -- 172.21.15.150:0/1584603407 learned_addr learned my addr 172.21.15.150:0/1584603407 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84659c700 1 -- 172.21.15.150:0/1584603407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830009550 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84659c700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff84819b4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff830011a20 tx=0x7ff830011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff83f7fe700 1 -- 172.21.15.150:0/1584603407 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff830003aa0 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff83f7fe700 1 -- 172.21.15.150:0/1584603407 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff830003c00 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84819bc10 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff83f7fe700 1 -- 172.21.15.150:0/1584603407 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8300196a0 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.744+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84819c0b0 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.745+0000 7ff83f7fe700 1 -- 172.21.15.150:0/1584603407 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7ff8300198d0 con 0x7ff848107450 2024-05-10T04:32:56.340 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.745+0000 7ff83f7fe700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7ff834038810 0x7ff83403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.745+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84810b7b0 con 0x7ff848107450 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.746+0000 7ff83f7fe700 1 -- 172.21.15.150:0/1584603407 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff83004c950 con 0x7ff848107450 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.746+0000 7ff845d9b700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7ff834038810 0x7ff83403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.746+0000 7ff845d9b700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7ff834038810 0x7ff83403ad20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff838006fd0 tx=0x7ff838006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.751+0000 7ff83f7fe700 1 -- 172.21.15.150:0/1584603407 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff830018db0 con 0x7ff848107450 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.942+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff84819e8b0 con 0x7ff834038810 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.943+0000 7ff83f7fe700 1 -- 172.21.15.150:0/1584603407 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7ff84819e8b0 con 0x7ff834038810 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7ff834038810 msgr2=0x7ff83403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7ff834038810 0x7ff83403ad20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff838006fd0 tx=0x7ff838006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 msgr2=0x7ff84819b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff84819b4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff830011a20 tx=0x7ff830011b00 comp rx=0 tx=0).stop 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 shutdown_connections 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7ff834038810 0x7ff83403ad20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 --2- 172.21.15.150:0/1584603407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff848107450 0x7ff84819b4d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:56.341 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 >> 172.21.15.150:0/1584603407 conn(0x7ff848102ad0 msgr2=0x7ff8481033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:56.342 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 shutdown_connections 2024-05-10T04:32:56.342 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:55.945+0000 7ff84cba5700 1 -- 172.21.15.150:0/1584603407 wait complete. 2024-05-10T04:32:56.342 INFO:teuthology.orchestra.run.smithi150.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-10T04:32:56.342 INFO:teuthology.orchestra.run.smithi150.stdout:Adding key to root@localhost authorized_keys... 2024-05-10T04:32:56.342 INFO:teuthology.orchestra.run.smithi150.stdout:Adding host smithi150... 2024-05-10T04:32:57.186 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:57 smithi150 ceph-mon[42404]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:32:58.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:58 smithi150 ceph-mon[42404]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "addr": "172.21.15.150", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:32:59.368 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:32:59 smithi150 ceph-mon[42404]: Deploying cephadm binary to smithi150 2024-05-10T04:32:59.415 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Added host 'smithi150' with addr '172.21.15.150' 2024-05-10T04:32:59.415 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f33302f4700 1 Processor -- start 2024-05-10T04:32:59.415 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f33302f4700 1 -- start start 2024-05-10T04:32:59.415 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f33302f4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f3328109f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:59.415 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f33302f4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332810a530 con 0x7f332807acf0 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f332e090700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f3328109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f332e090700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f3328109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54652/0 (socket says 172.21.15.150:54652) 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f332e090700 1 -- 172.21.15.150:0/340925915 learned_addr learned my addr 172.21.15.150:0/340925915 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.710+0000 7f332e090700 1 -- 172.21.15.150:0/340925915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332810a670 con 0x7f332807acf0 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.711+0000 7f332e090700 1 --2- 172.21.15.150:0/340925915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f3328109f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f33240098a0 tx=0x7f3324009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48823b651a0b2b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.711+0000 7f332d08e700 1 -- 172.21.15.150:0/340925915 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3324004160 con 0x7f332807acf0 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.711+0000 7f332d08e700 1 -- 172.21.15.150:0/340925915 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33240042c0 con 0x7f332807acf0 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.711+0000 7f332d08e700 1 -- 172.21.15.150:0/340925915 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3324011a00 con 0x7f332807acf0 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.711+0000 7f33302f4700 1 -- 172.21.15.150:0/340925915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 msgr2=0x7f3328109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.711+0000 7f33302f4700 1 --2- 172.21.15.150:0/340925915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f3328109f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f33240098a0 tx=0x7f3324009bb0 comp rx=0 tx=0).stop 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.712+0000 7f33302f4700 1 -- 172.21.15.150:0/340925915 shutdown_connections 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.712+0000 7f33302f4700 1 --2- 172.21.15.150:0/340925915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f3328109f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.712+0000 7f33302f4700 1 -- 172.21.15.150:0/340925915 >> 172.21.15.150:0/340925915 conn(0x7f33281019b0 msgr2=0x7f3328103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:59.416 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.712+0000 7f33302f4700 1 -- 172.21.15.150:0/340925915 shutdown_connections 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.712+0000 7f33302f4700 1 -- 172.21.15.150:0/340925915 wait complete. 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.712+0000 7f33302f4700 1 Processor -- start 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.712+0000 7f33302f4700 1 -- start start 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f33302f4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f332819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f33302f4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332819dc60 con 0x7f332807acf0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f332e090700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f332819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f332e090700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f332819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54654/0 (socket says 172.21.15.150:54654) 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f332e090700 1 -- 172.21.15.150:0/2201905885 learned_addr learned my addr 172.21.15.150:0/2201905885 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f332e090700 1 -- 172.21.15.150:0/2201905885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3324009550 con 0x7f332807acf0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f332e090700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f332819d720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3324011710 tx=0x7f3324011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f331f7fe700 1 -- 172.21.15.150:0/2201905885 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3324003af0 con 0x7f332807acf0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332819de60 con 0x7f332807acf0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332819e300 con 0x7f332807acf0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f331f7fe700 1 -- 172.21.15.150:0/2201905885 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3324003c50 con 0x7f332807acf0 2024-05-10T04:32:59.417 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.713+0000 7f331f7fe700 1 -- 172.21.15.150:0/2201905885 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3324024730 con 0x7f332807acf0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.714+0000 7f331f7fe700 1 -- 172.21.15.150:0/2201905885 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3324024890 con 0x7f332807acf0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.714+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c0052f0 con 0x7f332807acf0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.714+0000 7f331f7fe700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f33140387c0 0x7f331403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.714+0000 7f331f7fe700 1 -- 172.21.15.150:0/2201905885 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f332401a8d0 con 0x7f332807acf0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.714+0000 7f332d88f700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f33140387c0 0x7f331403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.715+0000 7f332d88f700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f33140387c0 0x7f331403acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3318006fd0 tx=0x7f3318006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.719+0000 7f331f7fe700 1 -- 172.21.15.150:0/2201905885 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3324018db0 con 0x7f332807acf0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:56.915+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "addr": "172.21.15.150", "target": ["mon-mgr", ""]}) v1 -- 0x7f330c000bc0 con 0x7f33140387c0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.094+0000 7f331f7fe700 1 -- 172.21.15.150:0/2201905885 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f330c000bc0 con 0x7f33140387c0 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f33140387c0 msgr2=0x7f331403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f33140387c0 0x7f331403acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3318006fd0 tx=0x7f3318006e40 comp rx=0 tx=0).stop 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 msgr2=0x7f332819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:32:59.418 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f332819d720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3324011710 tx=0x7f3324011ad0 comp rx=0 tx=0).stop 2024-05-10T04:32:59.419 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 shutdown_connections 2024-05-10T04:32:59.419 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f33140387c0 0x7f331403acd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:59.419 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 --2- 172.21.15.150:0/2201905885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f332807acf0 0x7f332819d720 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:32:59.419 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 >> 172.21.15.150:0/2201905885 conn(0x7f33281019b0 msgr2=0x7f3328105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:32:59.419 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 shutdown_connections 2024-05-10T04:32:59.419 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.097+0000 7f33302f4700 1 -- 172.21.15.150:0/2201905885 wait complete. 2024-05-10T04:32:59.419 INFO:teuthology.orchestra.run.smithi150.stdout:Deploying unmanaged mon service... 2024-05-10T04:33:00.282 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:00 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:33:00.282 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:00 smithi150 ceph-mon[42404]: Added host smithi150 2024-05-10T04:33:00.282 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:00 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:00.282 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:00 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.839+0000 7fcc5f230700 1 Processor -- start 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.839+0000 7fcc5f230700 1 -- start start 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.839+0000 7fcc5f230700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc58106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.839+0000 7fcc5f230700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58106d40 con 0x7fcc58106340 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc5cfcc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc58106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc5cfcc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc58106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39514/0 (socket says 172.21.15.150:39514) 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc5cfcc700 1 -- 172.21.15.150:0/2403391859 learned_addr learned my addr 172.21.15.150:0/2403391859 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc5cfcc700 1 -- 172.21.15.150:0/2403391859 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58107470 con 0x7fcc58106340 2024-05-10T04:33:00.466 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc5cfcc700 1 --2- 172.21.15.150:0/2403391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc58106750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0098a0 tx=0x7fcc4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acc08c2a776ae82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc577fe700 1 -- 172.21.15.150:0/2403391859 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc4c004160 con 0x7fcc58106340 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc577fe700 1 -- 172.21.15.150:0/2403391859 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc4c0042c0 con 0x7fcc58106340 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.840+0000 7fcc577fe700 1 -- 172.21.15.150:0/2403391859 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc4c011a00 con 0x7fcc58106340 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.841+0000 7fcc5f230700 1 -- 172.21.15.150:0/2403391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 msgr2=0x7fcc58106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.841+0000 7fcc5f230700 1 --2- 172.21.15.150:0/2403391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc58106750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0098a0 tx=0x7fcc4c009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.841+0000 7fcc5f230700 1 -- 172.21.15.150:0/2403391859 shutdown_connections 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.841+0000 7fcc5f230700 1 --2- 172.21.15.150:0/2403391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc58106750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.841+0000 7fcc5f230700 1 -- 172.21.15.150:0/2403391859 >> 172.21.15.150:0/2403391859 conn(0x7fcc581019c0 msgr2=0x7fcc58103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.841+0000 7fcc5f230700 1 -- 172.21.15.150:0/2403391859 shutdown_connections 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.841+0000 7fcc5f230700 1 -- 172.21.15.150:0/2403391859 wait complete. 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.842+0000 7fcc5f230700 1 Processor -- start 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.842+0000 7fcc5f230700 1 -- start start 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.842+0000 7fcc5f230700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc5819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.842+0000 7fcc5f230700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5819ba80 con 0x7fcc58106340 2024-05-10T04:33:00.467 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc5cfcc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc5819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc5cfcc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc5819b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39518/0 (socket says 172.21.15.150:39518) 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc5cfcc700 1 -- 172.21.15.150:0/3535907066 learned_addr learned my addr 172.21.15.150:0/3535907066 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc5cfcc700 1 -- 172.21.15.150:0/3535907066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c009550 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc5cfcc700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc5819b540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009130 tx=0x7fcc4c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc55ffb700 1 -- 172.21.15.150:0/3535907066 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc4c0039e0 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc55ffb700 1 -- 172.21.15.150:0/3535907066 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc4c003b40 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5819bc80 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc55ffb700 1 -- 172.21.15.150:0/3535907066 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc4c019690 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.843+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5819c120 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.844+0000 7fcc55ffb700 1 -- 172.21.15.150:0/3535907066 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fcc4c0198c0 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.844+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc58194b60 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.845+0000 7fcc55ffb700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fcc480387f0 0x7fcc4803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.845+0000 7fcc55ffb700 1 -- 172.21.15.150:0/3535907066 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc4c04c850 con 0x7fcc58106340 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.845+0000 7fcc57fff700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fcc480387f0 0x7fcc4803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:00.468 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.845+0000 7fcc57fff700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fcc480387f0 0x7fcc4803ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006fd0 tx=0x7fcc44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:32:59.850+0000 7fcc55ffb700 1 -- 172.21.15.150:0/3535907066 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcc4c018db0 con 0x7fcc58106340 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.069+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcc5810a670 con 0x7fcc480387f0 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.074+0000 7fcc55ffb700 1 -- 172.21.15.150:0/3535907066 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcc5810a670 con 0x7fcc480387f0 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.076+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fcc480387f0 msgr2=0x7fcc4803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.076+0000 7fcc5f230700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fcc480387f0 0x7fcc4803ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006fd0 tx=0x7fcc44006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.076+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 msgr2=0x7fcc5819b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.076+0000 7fcc5f230700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc5819b540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009130 tx=0x7fcc4c004250 comp rx=0 tx=0).stop 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.077+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 shutdown_connections 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.077+0000 7fcc5f230700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fcc480387f0 0x7fcc4803ad00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.077+0000 7fcc5f230700 1 --2- 172.21.15.150:0/3535907066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc58106340 0x7fcc5819b540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:00.469 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.077+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 >> 172.21.15.150:0/3535907066 conn(0x7fcc581019c0 msgr2=0x7fcc58191ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:00.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.077+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 shutdown_connections 2024-05-10T04:33:00.470 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.077+0000 7fcc5f230700 1 -- 172.21.15.150:0/3535907066 wait complete. 2024-05-10T04:33:00.470 INFO:teuthology.orchestra.run.smithi150.stdout:Deploying unmanaged mgr service... 2024-05-10T04:33:01.368 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:01 smithi150 ceph-mon[42404]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:01.368 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:01 smithi150 ceph-mon[42404]: Saving service mon spec with placement count:5 2024-05-10T04:33:01.369 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:01 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:33:01.369 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:01 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:33:01.369 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:01 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.803+0000 7faf2277b700 1 Processor -- start 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.803+0000 7faf2277b700 1 -- start start 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.803+0000 7faf2277b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c108d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.803+0000 7faf2277b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c109330 con 0x7faf1c108950 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.803+0000 7faf1bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.803+0000 7faf1bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c108d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39526/0 (socket says 172.21.15.150:39526) 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.803+0000 7faf1bfff700 1 -- 172.21.15.150:0/287605472 learned_addr learned my addr 172.21.15.150:0/287605472 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.804+0000 7faf1bfff700 1 -- 172.21.15.150:0/287605472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c109470 con 0x7faf1c108950 2024-05-10T04:33:01.530 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.804+0000 7faf1bfff700 1 --2- 172.21.15.150:0/287605472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c108d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0098a0 tx=0x7faf0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bbdc72401cb7ee server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.804+0000 7faf1affd700 1 -- 172.21.15.150:0/287605472 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf0c004160 con 0x7faf1c108950 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.804+0000 7faf1affd700 1 -- 172.21.15.150:0/287605472 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf0c0042c0 con 0x7faf1c108950 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.804+0000 7faf1affd700 1 -- 172.21.15.150:0/287605472 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf0c0119b0 con 0x7faf1c108950 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.805+0000 7faf2277b700 1 -- 172.21.15.150:0/287605472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 msgr2=0x7faf1c108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.805+0000 7faf2277b700 1 --2- 172.21.15.150:0/287605472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c108d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0098a0 tx=0x7faf0c009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.805+0000 7faf2277b700 1 -- 172.21.15.150:0/287605472 shutdown_connections 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.805+0000 7faf2277b700 1 --2- 172.21.15.150:0/287605472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c108d60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.805+0000 7faf2277b700 1 -- 172.21.15.150:0/287605472 >> 172.21.15.150:0/287605472 conn(0x7faf1c101150 msgr2=0x7faf1c1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.805+0000 7faf2277b700 1 -- 172.21.15.150:0/287605472 shutdown_connections 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.805+0000 7faf2277b700 1 -- 172.21.15.150:0/287605472 wait complete. 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.806+0000 7faf2277b700 1 Processor -- start 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.806+0000 7faf2277b700 1 -- start start 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.806+0000 7faf2277b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.806+0000 7faf2277b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c19b9d0 con 0x7faf1c108950 2024-05-10T04:33:01.531 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.806+0000 7faf1bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.806+0000 7faf1bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39542/0 (socket says 172.21.15.150:39542) 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.806+0000 7faf1bfff700 1 -- 172.21.15.150:0/1770394036 learned_addr learned my addr 172.21.15.150:0/1770394036 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.807+0000 7faf1bfff700 1 -- 172.21.15.150:0/1770394036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c009550 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.807+0000 7faf1bfff700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c19b490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faf0c011670 tx=0x7faf0c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.807+0000 7faf197fa700 1 -- 172.21.15.150:0/1770394036 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf0c003af0 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.807+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c19bc30 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.807+0000 7faf197fa700 1 -- 172.21.15.150:0/1770394036 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf0c003c50 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.807+0000 7faf197fa700 1 -- 172.21.15.150:0/1770394036 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf0c024730 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.807+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1c19e870 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.808+0000 7faf197fa700 1 -- 172.21.15.150:0/1770394036 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7faf0c024970 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.808+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1c062840 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.808+0000 7faf197fa700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7faf040387c0 0x7faf0403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.808+0000 7faf197fa700 1 -- 172.21.15.150:0/1770394036 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faf0c04d5d0 con 0x7faf1c108950 2024-05-10T04:33:01.532 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.808+0000 7faf1b7fe700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7faf040387c0 0x7faf0403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.809+0000 7faf1b7fe700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7faf040387c0 0x7faf0403acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf10006fd0 tx=0x7faf10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:00.813+0000 7faf197fa700 1 -- 172.21.15.150:0/1770394036 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf0c018b80 con 0x7faf1c108950 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.053+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7faf1c1032d0 con 0x7faf040387c0 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.058+0000 7faf197fa700 1 -- 172.21.15.150:0/1770394036 <== mgr.14120 v2:172.21.15.150:6800/881426482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faf1c1032d0 con 0x7faf040387c0 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.060+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7faf040387c0 msgr2=0x7faf0403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.060+0000 7faf2277b700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7faf040387c0 0x7faf0403acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf10006fd0 tx=0x7faf10006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.060+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 msgr2=0x7faf1c19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.060+0000 7faf2277b700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c19b490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faf0c011670 tx=0x7faf0c011a30 comp rx=0 tx=0).stop 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.061+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 shutdown_connections 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.061+0000 7faf2277b700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7faf040387c0 0x7faf0403acd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.061+0000 7faf2277b700 1 --2- 172.21.15.150:0/1770394036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1c108950 0x7faf1c19b490 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.061+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 >> 172.21.15.150:0/1770394036 conn(0x7faf1c101150 msgr2=0x7faf1c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.061+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 shutdown_connections 2024-05-10T04:33:01.533 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.061+0000 7faf2277b700 1 -- 172.21.15.150:0/1770394036 wait complete. 2024-05-10T04:33:02.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:02 smithi150 ceph-mon[42404]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:02.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:02 smithi150 ceph-mon[42404]: Saving service mgr spec with placement count:2 2024-05-10T04:33:02.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:02 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:33:02.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:02 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2198953674' entity='client.admin' 2024-05-10T04:33:02.437 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.861+0000 7f8d6bb11700 1 Processor -- start 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.862+0000 7f8d6bb11700 1 -- start start 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.862+0000 7f8d6bb11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d64106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.862+0000 7f8d6bb11700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d64106d50 con 0x7f8d64106350 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.862+0000 7f8d698ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d64106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.862+0000 7f8d698ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d64106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39558/0 (socket says 172.21.15.150:39558) 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.862+0000 7f8d698ad700 1 -- 172.21.15.150:0/3771691324 learned_addr learned my addr 172.21.15.150:0/3771691324 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.863+0000 7f8d698ad700 1 -- 172.21.15.150:0/3771691324 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d64107480 con 0x7f8d64106350 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.863+0000 7f8d698ad700 1 --2- 172.21.15.150:0/3771691324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d64106760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098a0 tx=0x7f8d60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c00c4ee7d3637a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.863+0000 7f8d688ab700 1 -- 172.21.15.150:0/3771691324 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d60004160 con 0x7f8d64106350 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.863+0000 7f8d688ab700 1 -- 172.21.15.150:0/3771691324 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d600042c0 con 0x7f8d64106350 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.863+0000 7f8d688ab700 1 -- 172.21.15.150:0/3771691324 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d600119b0 con 0x7f8d64106350 2024-05-10T04:33:02.438 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.864+0000 7f8d6bb11700 1 -- 172.21.15.150:0/3771691324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 msgr2=0x7f8d64106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.864+0000 7f8d6bb11700 1 --2- 172.21.15.150:0/3771691324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d64106760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098a0 tx=0x7f8d60009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.864+0000 7f8d6bb11700 1 -- 172.21.15.150:0/3771691324 shutdown_connections 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.864+0000 7f8d6bb11700 1 --2- 172.21.15.150:0/3771691324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d64106760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.864+0000 7f8d6bb11700 1 -- 172.21.15.150:0/3771691324 >> 172.21.15.150:0/3771691324 conn(0x7f8d641019d0 msgr2=0x7f8d64103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.864+0000 7f8d6bb11700 1 -- 172.21.15.150:0/3771691324 shutdown_connections 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.864+0000 7f8d6bb11700 1 -- 172.21.15.150:0/3771691324 wait complete. 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.865+0000 7f8d6bb11700 1 Processor -- start 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.865+0000 7f8d6bb11700 1 -- start start 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.865+0000 7f8d6bb11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d6407a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.865+0000 7f8d6bb11700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6407aed0 con 0x7f8d64106350 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.865+0000 7f8d698ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d6407a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:02.439 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d698ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d6407a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39568/0 (socket says 172.21.15.150:39568) 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d698ad700 1 -- 172.21.15.150:0/2198953674 learned_addr learned my addr 172.21.15.150:0/2198953674 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d698ad700 1 -- 172.21.15.150:0/2198953674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60009550 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d698ad700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d6407a990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d60003f00 tx=0x7f8d60011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d5affd700 1 -- 172.21.15.150:0/2198953674 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d600038b0 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d5affd700 1 -- 172.21.15.150:0/2198953674 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d60003a10 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6419e2c0 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d5affd700 1 -- 172.21.15.150:0/2198953674 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d60019550 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.866+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6419e6a0 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.867+0000 7f8d5affd700 1 -- 172.21.15.150:0/2198953674 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f8d600197d0 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.867+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6410a6b0 con 0x7f8d64106350 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.867+0000 7f8d5affd700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f8d50038770 0x7f8d5003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:02.440 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.868+0000 7f8d5affd700 1 -- 172.21.15.150:0/2198953674 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8d6004c900 con 0x7f8d64106350 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.868+0000 7f8d690ac700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f8d50038770 0x7f8d5003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.868+0000 7f8d690ac700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f8d50038770 0x7f8d5003ac80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d54006fd0 tx=0x7f8d54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:01.877+0000 7f8d5affd700 1 -- 172.21.15.150:0/2198953674 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d60018b80 con 0x7f8d64106350 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.088+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f8d6419e9a0 con 0x7f8d64106350 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.092+0000 7f8d5affd700 1 -- 172.21.15.150:0/2198953674 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8d60018350 con 0x7f8d64106350 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f8d50038770 msgr2=0x7f8d5003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f8d50038770 0x7f8d5003ac80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d54006fd0 tx=0x7f8d54006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 msgr2=0x7f8d6407a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d6407a990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8d60003f00 tx=0x7f8d60011a30 comp rx=0 tx=0).stop 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 shutdown_connections 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f8d50038770 0x7f8d5003ac80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 --2- 172.21.15.150:0/2198953674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d64106350 0x7f8d6407a990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:02.441 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.095+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 >> 172.21.15.150:0/2198953674 conn(0x7f8d641019d0 msgr2=0x7f8d64102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:02.442 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.098+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 shutdown_connections 2024-05-10T04:33:02.442 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.098+0000 7f8d6bb11700 1 -- 172.21.15.150:0/2198953674 wait complete. 2024-05-10T04:33:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9062d1a700 1 Processor -- start 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9062d1a700 1 -- start start 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9062d1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9062d1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c106cf0 con 0x7f905c1062f0 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9060ab6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9060ab6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39574/0 (socket says 172.21.15.150:39574) 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9060ab6700 1 -- 172.21.15.150:0/3784676417 learned_addr learned my addr 172.21.15.150:0/3784676417 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.798+0000 7f9060ab6700 1 -- 172.21.15.150:0/3784676417 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c107420 con 0x7f905c1062f0 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.799+0000 7f9060ab6700 1 --2- 172.21.15.150:0/3784676417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c106700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90500098a0 tx=0x7f9050009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b69e1a07b3c3f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.799+0000 7f905b7fe700 1 -- 172.21.15.150:0/3784676417 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9050004160 con 0x7f905c1062f0 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.799+0000 7f905b7fe700 1 -- 172.21.15.150:0/3784676417 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90500042c0 con 0x7f905c1062f0 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.799+0000 7f905b7fe700 1 -- 172.21.15.150:0/3784676417 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90500119b0 con 0x7f905c1062f0 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.799+0000 7f9062d1a700 1 -- 172.21.15.150:0/3784676417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 msgr2=0x7f905c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.800+0000 7f9062d1a700 1 --2- 172.21.15.150:0/3784676417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c106700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f90500098a0 tx=0x7f9050009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.800+0000 7f9062d1a700 1 -- 172.21.15.150:0/3784676417 shutdown_connections 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.800+0000 7f9062d1a700 1 --2- 172.21.15.150:0/3784676417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c106700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.800+0000 7f9062d1a700 1 -- 172.21.15.150:0/3784676417 >> 172.21.15.150:0/3784676417 conn(0x7f905c101950 msgr2=0x7f905c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.800+0000 7f9062d1a700 1 -- 172.21.15.150:0/3784676417 shutdown_connections 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.800+0000 7f9062d1a700 1 -- 172.21.15.150:0/3784676417 wait complete. 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.800+0000 7f9062d1a700 1 Processor -- start 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9062d1a700 1 -- start start 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9062d1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c07a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9062d1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c07af10 con 0x7f905c1062f0 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9060ab6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c07a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9060ab6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c07a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39582/0 (socket says 172.21.15.150:39582) 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9060ab6700 1 -- 172.21.15.150:0/390824154 learned_addr learned my addr 172.21.15.150:0/390824154 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:03.572 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9060ab6700 1 -- 172.21.15.150:0/390824154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9050009550 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9060ab6700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c07a9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9050003f00 tx=0x7f9050011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9059ffb700 1 -- 172.21.15.150:0/390824154 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90500038b0 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9059ffb700 1 -- 172.21.15.150:0/390824154 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9050003a10 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.801+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c19c090 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.802+0000 7f9059ffb700 1 -- 172.21.15.150:0/390824154 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9050019550 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.802+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c19c530 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.803+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c10a650 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.803+0000 7f9059ffb700 1 -- 172.21.15.150:0/390824154 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f90500197d0 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.803+0000 7f9059ffb700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f904c038770 0x7f904c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.804+0000 7f9059ffb700 1 -- 172.21.15.150:0/390824154 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f905004cd40 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.804+0000 7f905bfff700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f904c038770 0x7f904c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.804+0000 7f905bfff700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f904c038770 0x7f904c03ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9048006fd0 tx=0x7f9048006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:02.809+0000 7f9059ffb700 1 -- 172.21.15.150:0/390824154 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9050003b80 con 0x7f905c1062f0 2024-05-10T04:33:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.020+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f905c19c720 con 0x7f905c1062f0 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.024+0000 7f9059ffb700 1 -- 172.21.15.150:0/390824154 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f905004b0c0 con 0x7f905c1062f0 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.026+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f904c038770 msgr2=0x7f904c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.026+0000 7f9062d1a700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f904c038770 0x7f904c03ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9048006fd0 tx=0x7f9048006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.026+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 msgr2=0x7f905c07a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.026+0000 7f9062d1a700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c07a9d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9050003f00 tx=0x7f9050011a30 comp rx=0 tx=0).stop 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.027+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 shutdown_connections 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.027+0000 7f9062d1a700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f904c038770 0x7f904c03ac80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.027+0000 7f9062d1a700 1 --2- 172.21.15.150:0/390824154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f905c1062f0 0x7f905c07a9d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.027+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 >> 172.21.15.150:0/390824154 conn(0x7f905c101950 msgr2=0x7f905c1022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.027+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 shutdown_connections 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.027+0000 7f9062d1a700 1 -- 172.21.15.150:0/390824154 wait complete. 2024-05-10T04:33:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling the dashboard module... 2024-05-10T04:33:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:03 smithi150 ceph-mon[42404]: from='mgr.14120 172.21.15.150:0/4085592427' entity='mgr.x' 2024-05-10T04:33:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:03 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/390824154' entity='client.admin' 2024-05-10T04:33:04.559 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:04 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3424093416' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-10T04:33:04.559 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:04 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: ignoring --setuser ceph since I am not root 2024-05-10T04:33:04.560 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:04 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: ignoring --setgroup ceph since I am not root 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.932+0000 7f329797d700 1 Processor -- start 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.932+0000 7f329797d700 1 -- start start 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.932+0000 7f329797d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329010c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.932+0000 7f329797d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3290107e50 con 0x7f329007acb0 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.932+0000 7f3295719700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329010c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.933+0000 7f3295719700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329010c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39592/0 (socket says 172.21.15.150:39592) 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.933+0000 7f3295719700 1 -- 172.21.15.150:0/508378390 learned_addr learned my addr 172.21.15.150:0/508378390 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.933+0000 7f3295719700 1 -- 172.21.15.150:0/508378390 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329010c910 con 0x7f329007acb0 2024-05-10T04:33:04.645 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.933+0000 7f3295719700 1 --2- 172.21.15.150:0/508378390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329010c3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32800098a0 tx=0x7f3280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75a6c7d25ab1294 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.933+0000 7f3287fff700 1 -- 172.21.15.150:0/508378390 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3280004160 con 0x7f329007acb0 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.933+0000 7f3287fff700 1 -- 172.21.15.150:0/508378390 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32800042c0 con 0x7f329007acb0 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.933+0000 7f3287fff700 1 -- 172.21.15.150:0/508378390 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32800119b0 con 0x7f329007acb0 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.934+0000 7f329797d700 1 -- 172.21.15.150:0/508378390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 msgr2=0x7f329010c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.934+0000 7f329797d700 1 --2- 172.21.15.150:0/508378390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329010c3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32800098a0 tx=0x7f3280009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.934+0000 7f329797d700 1 -- 172.21.15.150:0/508378390 shutdown_connections 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.934+0000 7f329797d700 1 --2- 172.21.15.150:0/508378390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329010c3d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.934+0000 7f329797d700 1 -- 172.21.15.150:0/508378390 >> 172.21.15.150:0/508378390 conn(0x7f32901019c0 msgr2=0x7f3290103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.934+0000 7f329797d700 1 -- 172.21.15.150:0/508378390 shutdown_connections 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.934+0000 7f329797d700 1 -- 172.21.15.150:0/508378390 wait complete. 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f329797d700 1 Processor -- start 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f329797d700 1 -- start start 2024-05-10T04:33:04.646 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f329797d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f329797d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329019dc40 con 0x7f329007acb0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f3295719700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f3295719700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39596/0 (socket says 172.21.15.150:39596) 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f3295719700 1 -- 172.21.15.150:0/3424093416 learned_addr learned my addr 172.21.15.150:0/3424093416 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f3295719700 1 -- 172.21.15.150:0/3424093416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3280009550 con 0x7f329007acb0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.935+0000 7f3295719700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329019d700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3280006af0 tx=0x7f3280011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.936+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32800038b0 con 0x7f329007acb0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.936+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3280003a10 con 0x7f329007acb0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.936+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329019de40 con 0x7f329007acb0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.936+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3280019550 con 0x7f329007acb0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.936+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329019e2e0 con 0x7f329007acb0 2024-05-10T04:33:04.647 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.937+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f3280019810 con 0x7f329007acb0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.937+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329010d810 con 0x7f329007acb0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.937+0000 7f32867fc700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f327c038770 0x7f327c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.937+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f328004c820 con 0x7f329007acb0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.937+0000 7f3294f18700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f327c038770 0x7f327c03ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.937+0000 7f3294f18700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f327c038770 0x7f327c03ac80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f328c006fd0 tx=0x7f328c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:03.942+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f32800189a0 con 0x7f329007acb0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.191+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f329010da20 con 0x7f329007acb0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.268+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f328002d960 con 0x7f329007acb0 2024-05-10T04:33:04.648 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.270+0000 7f32867fc700 1 -- 172.21.15.150:0/3424093416 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f3280018b80 con 0x7f329007acb0 2024-05-10T04:33:04.649 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f327c038770 msgr2=0x7f327c03ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:04.649 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f327c038770 0x7f327c03ac80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f328c006fd0 tx=0x7f328c006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:04.649 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 msgr2=0x7f329019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:04.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329019d700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3280006af0 tx=0x7f3280011980 comp rx=0 tx=0).stop 2024-05-10T04:33:04.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 shutdown_connections 2024-05-10T04:33:04.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f327c038770 0x7f327c03ac80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:04.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 --2- 172.21.15.150:0/3424093416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f329007acb0 0x7f329019d700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:04.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 >> 172.21.15.150:0/3424093416 conn(0x7f32901019c0 msgr2=0x7f3290102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:04.650 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 shutdown_connections 2024-05-10T04:33:04.651 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:04.272+0000 7f329797d700 1 -- 172.21.15.150:0/3424093416 wait complete. 2024-05-10T04:33:04.871 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:04 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:04.754+0000 7f78c23f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T04:33:04.871 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:04 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:04.869+0000 7f78c23f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T04:33:05.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:04 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:04.963+0000 7f78c23f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T04:33:05.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:05 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:05.056+0000 7f78c23f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T04:33:05.528 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:05 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3424093416' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-10T04:33:05.528 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:05 smithi150 ceph-mon[42404]: mgrmap e9: x(active, since 16s) 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.009+0000 7f1539e41700 1 Processor -- start 2024-05-10T04:33:05.621 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.009+0000 7f1539e41700 1 -- start start 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.009+0000 7f1539e41700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f1534109f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.009+0000 7f1539e41700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153410a520 con 0x7f153407acd0 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.010+0000 7f15337fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f1534109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.010+0000 7f15337fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f1534109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39626/0 (socket says 172.21.15.150:39626) 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.010+0000 7f15337fe700 1 -- 172.21.15.150:0/2017677701 learned_addr learned my addr 172.21.15.150:0/2017677701 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.010+0000 7f15337fe700 1 -- 172.21.15.150:0/2017677701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153410a660 con 0x7f153407acd0 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.010+0000 7f15337fe700 1 --2- 172.21.15.150:0/2017677701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f1534109f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15240098a0 tx=0x7f1524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f33cf5d084ed0f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.010+0000 7f15327fc700 1 -- 172.21.15.150:0/2017677701 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1524004160 con 0x7f153407acd0 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f15327fc700 1 -- 172.21.15.150:0/2017677701 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15240042c0 con 0x7f153407acd0 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f15327fc700 1 -- 172.21.15.150:0/2017677701 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15240119b0 con 0x7f153407acd0 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f1539e41700 1 -- 172.21.15.150:0/2017677701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 msgr2=0x7f1534109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f1539e41700 1 --2- 172.21.15.150:0/2017677701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f1534109f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15240098a0 tx=0x7f1524009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f1539e41700 1 -- 172.21.15.150:0/2017677701 shutdown_connections 2024-05-10T04:33:05.622 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f1539e41700 1 --2- 172.21.15.150:0/2017677701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f1534109f30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f1539e41700 1 -- 172.21.15.150:0/2017677701 >> 172.21.15.150:0/2017677701 conn(0x7f1534101980 msgr2=0x7f1534103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f1539e41700 1 -- 172.21.15.150:0/2017677701 shutdown_connections 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.011+0000 7f1539e41700 1 -- 172.21.15.150:0/2017677701 wait complete. 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.012+0000 7f1539e41700 1 Processor -- start 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.012+0000 7f1539e41700 1 -- start start 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.012+0000 7f1539e41700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f153419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.012+0000 7f1539e41700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153419dc50 con 0x7f153407acd0 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f15337fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f153419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f15337fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f153419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39634/0 (socket says 172.21.15.150:39634) 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f15337fe700 1 -- 172.21.15.150:0/3900274802 learned_addr learned my addr 172.21.15.150:0/3900274802 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f15337fe700 1 -- 172.21.15.150:0/3900274802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009550 con 0x7f153407acd0 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f15337fe700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f153419d710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1524006af0 tx=0x7f1524011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f1530ff9700 1 -- 172.21.15.150:0/3900274802 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15240038b0 con 0x7f153407acd0 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f1530ff9700 1 -- 172.21.15.150:0/3900274802 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1524003a10 con 0x7f153407acd0 2024-05-10T04:33:05.623 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153419de50 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f1530ff9700 1 -- 172.21.15.150:0/3900274802 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1524019550 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.013+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153419e2f0 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.014+0000 7f1530ff9700 1 -- 172.21.15.150:0/3900274802 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f15240197c0 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.014+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153410b560 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.015+0000 7f1530ff9700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 0x7f151c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.015+0000 7f1530ff9700 1 -- 172.21.15.150:0/3900274802 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f152404d740 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.015+0000 7f1532ffd700 1 -- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 msgr2=0x7f151c03ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/881426482 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.015+0000 7f1532ffd700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 0x7f151c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.020+0000 7f1530ff9700 1 -- 172.21.15.150:0/3900274802 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1524018d90 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.216+0000 7f1532ffd700 1 -- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 msgr2=0x7f151c03ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/881426482 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.216+0000 7f1532ffd700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 0x7f151c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.277+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f15341a0af0 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.277+0000 7f1530ff9700 1 -- 172.21.15.150:0/3900274802 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1524018350 con 0x7f153407acd0 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 msgr2=0x7f151c03ad20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:33:05.624 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 0x7f151c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 msgr2=0x7f153419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f153419d710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1524006af0 tx=0x7f1524011980 comp rx=0 tx=0).stop 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 shutdown_connections 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7f151c038810 0x7f151c03ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 --2- 172.21.15.150:0/3900274802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f153407acd0 0x7f153419d710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 >> 172.21.15.150:0/3900274802 conn(0x7f1534101980 msgr2=0x7f1534102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 shutdown_connections 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.279+0000 7f1539e41700 1 -- 172.21.15.150:0/3900274802 wait complete. 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for the mgr to restart... 2024-05-10T04:33:05.625 INFO:teuthology.orchestra.run.smithi150.stdout:Waiting for mgr epoch 9... 2024-05-10T04:33:05.794 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:05 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:05.718+0000 7f78c23f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:33:06.269 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:05 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:05.993+0000 7f78c23f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T04:33:06.269 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:06 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:06.084+0000 7f78c23f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T04:33:06.269 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:06 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:06.197+0000 7f78c23f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T04:33:06.575 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:06 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3900274802' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-10T04:33:06.576 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:06 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:06.358+0000 7f78c23f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T04:33:06.576 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:06 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:06.486+0000 7f78c23f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T04:33:06.911 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:06 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:06.573+0000 7f78c23f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T04:33:07.349 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:06 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:06.951+0000 7f78c23f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T04:33:07.643 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:07 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:07.348+0000 7f78c23f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T04:33:07.643 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:07 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:07.547+0000 7f78c23f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T04:33:08.671 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:07 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:07.642+0000 7f78c23f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T04:33:08.671 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:08 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:08.419+0000 7f78c23f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T04:33:08.671 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:08 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:08.519+0000 7f78c23f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T04:33:09.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:08 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:08.669+0000 7f78c23f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T04:33:09.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:09 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:09.306+0000 7f78c23f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T04:33:09.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:09 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:09.493+0000 7f78c23f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T04:33:10.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:09 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:09.692+0000 7f78c23f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T04:33:10.911 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:10 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:10.642+0000 7f78c23f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T04:33:11.226 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:10 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:10.937+0000 7f78c23f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:33:11.226 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:11 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:11.028+0000 7f78c23f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T04:33:11.661 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:33:11 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:33:11.224+0000 7f78c23f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: Active manager daemon x restarted 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: Activating manager daemon x 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: osdmap e3: 0 total, 0 up, 0 in 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: mgrmap e10: x(active, starting, since 0.00382137s) 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: Manager daemon x is now available 2024-05-10T04:33:11.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:12.588 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout { 2024-05-10T04:33:12.588 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-10T04:33:12.588 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-10T04:33:12.588 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout } 2024-05-10T04:33:12.588 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.992+0000 7fa13ccc7700 1 Processor -- start 2024-05-10T04:33:12.588 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.992+0000 7fa13ccc7700 1 -- start start 2024-05-10T04:33:12.588 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.993+0000 7fa13ccc7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13807b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.993+0000 7fa13ccc7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13807b7e0 con 0x7fa13807cdd0 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.993+0000 7fa13659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13807b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.993+0000 7fa13659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13807b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39642/0 (socket says 172.21.15.150:39642) 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.993+0000 7fa13659c700 1 -- 172.21.15.150:0/928233094 learned_addr learned my addr 172.21.15.150:0/928233094 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.993+0000 7fa13659c700 1 -- 172.21.15.150:0/928233094 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13807b920 con 0x7fa13807cdd0 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.994+0000 7fa13659c700 1 --2- 172.21.15.150:0/928233094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13807b2a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1280098a0 tx=0x7fa128009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d8de731285d477 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.994+0000 7fa13559a700 1 -- 172.21.15.150:0/928233094 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa128004160 con 0x7fa13807cdd0 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.994+0000 7fa13559a700 1 -- 172.21.15.150:0/928233094 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1280042c0 con 0x7fa13807cdd0 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.994+0000 7fa13559a700 1 -- 172.21.15.150:0/928233094 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1280119b0 con 0x7fa13807cdd0 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.995+0000 7fa13ccc7700 1 -- 172.21.15.150:0/928233094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 msgr2=0x7fa13807b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.995+0000 7fa13ccc7700 1 --2- 172.21.15.150:0/928233094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13807b2a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1280098a0 tx=0x7fa128009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.995+0000 7fa13ccc7700 1 -- 172.21.15.150:0/928233094 shutdown_connections 2024-05-10T04:33:12.589 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.995+0000 7fa13ccc7700 1 --2- 172.21.15.150:0/928233094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13807b2a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.995+0000 7fa13ccc7700 1 -- 172.21.15.150:0/928233094 >> 172.21.15.150:0/928233094 conn(0x7fa138102aa0 msgr2=0x7fa138104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.995+0000 7fa13ccc7700 1 -- 172.21.15.150:0/928233094 shutdown_connections 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.995+0000 7fa13ccc7700 1 -- 172.21.15.150:0/928233094 wait complete. 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.996+0000 7fa13ccc7700 1 Processor -- start 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.996+0000 7fa13ccc7700 1 -- start start 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.996+0000 7fa13ccc7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.996+0000 7fa13ccc7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13819dc20 con 0x7fa13807cdd0 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.996+0000 7fa13659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.996+0000 7fa13659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39652/0 (socket says 172.21.15.150:39652) 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.996+0000 7fa13659c700 1 -- 172.21.15.150:0/1042577657 learned_addr learned my addr 172.21.15.150:0/1042577657 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.997+0000 7fa13659c700 1 -- 172.21.15.150:0/1042577657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128009550 con 0x7fa13807cdd0 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.997+0000 7fa13659c700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13819d6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa128003e60 tx=0x7fa128011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.997+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1280038b0 con 0x7fa13807cdd0 2024-05-10T04:33:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.997+0000 7fa13ccc7700 1 -- 172.21.15.150:0/1042577657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13819de20 con 0x7fa13807cdd0 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.997+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa128003a10 con 0x7fa13807cdd0 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.997+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa128019530 con 0x7fa13807cdd0 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.997+0000 7fa13ccc7700 1 -- 172.21.15.150:0/1042577657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13819e2c0 con 0x7fa13807cdd0 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.998+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fa128019810 con 0x7fa13807cdd0 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.998+0000 7fa1277fe700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.998+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 --> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa12003b3e0 con 0x7fa1200387c0 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.998+0000 7fa135d9b700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 msgr2=0x7fa12003acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/881426482 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.998+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa12804c890 con 0x7fa13807cdd0 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:05.998+0000 7fa135d9b700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:06.199+0000 7fa135d9b700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 msgr2=0x7fa12003acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/881426482 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:06.199+0000 7fa135d9b700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:06.600+0000 7fa135d9b700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 msgr2=0x7fa12003acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/881426482 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:06.600+0000 7fa135d9b700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-10T04:33:12.591 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:07.401+0000 7fa135d9b700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 msgr2=0x7fa12003acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/881426482 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:07.401+0000 7fa135d9b700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:09.002+0000 7fa135d9b700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 msgr2=0x7fa12003acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.150:6800/881426482 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:09.002+0000 7fa135d9b700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:11.239+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7fa12802da00 con 0x7fa13807cdd0 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:11.239+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 msgr2=0x7fa12003acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:11.239+0000 7fa1277fe700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.243+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fa12804e450 con 0x7fa13807cdd0 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.243+0000 7fa1277fe700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa12804c480 0x7fa12003e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.243+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa12003b3e0 con 0x7fa12804c480 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.244+0000 7fa135d9b700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa12804c480 0x7fa12003e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.244+0000 7fa135d9b700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa12804c480 0x7fa12003e160 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa12c003a10 tx=0x7fa12c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.245+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fa12003b3e0 con 0x7fa12804c480 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.251+0000 7fa1257fa700 1 -- 172.21.15.150:0/1042577657 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa11c002600 con 0x7fa12804c480 2024-05-10T04:33:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1277fe700 1 -- 172.21.15.150:0/1042577657 <== mgr.14152 v2:172.21.15.150:6800/3064659046 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa11c002600 con 0x7fa12804c480 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa12804c480 msgr2=0x7fa12003e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa12804c480 0x7fa12003e160 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa12c003a10 tx=0x7fa12c0092b0 comp rx=0 tx=0).stop 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 -- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 msgr2=0x7fa13819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13819d6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa128003e60 tx=0x7fa128011a30 comp rx=0 tx=0).stop 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 -- 172.21.15.150:0/1042577657 shutdown_connections 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa12804c480 0x7fa12003e160 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa13807cdd0 0x7fa13819d6e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 --2- 172.21.15.150:0/1042577657 >> [v2:172.21.15.150:6800/881426482,v1:172.21.15.150:6801/881426482] conn(0x7fa1200387c0 0x7fa12003acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 -- 172.21.15.150:0/1042577657 >> 172.21.15.150:0/1042577657 conn(0x7fa138102aa0 msgr2=0x7fa138103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 -- 172.21.15.150:0/1042577657 shutdown_connections 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.252+0000 7fa1257fa700 1 -- 172.21.15.150:0/1042577657 wait complete. 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:mgr epoch 9 is available 2024-05-10T04:33:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:Generating a dashboard self-signed certificate... 2024-05-10T04:33:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-10T04:33:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-10T04:33:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:12 smithi150 ceph-mon[42404]: mgrmap e11: x(active, since 1.00862s) 2024-05-10T04:33:13.512 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:13 smithi150 ceph-mon[42404]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-10T04:33:13.513 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:13 smithi150 ceph-mon[42404]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-10T04:33:13.513 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:13.513 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:13.513 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.939+0000 7f00c0376700 1 Processor -- start 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.939+0000 7f00c0376700 1 -- start start 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.940+0000 7f00c0376700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b810a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.940+0000 7f00c0376700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b807ae00 con 0x7f00b8107f30 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.940+0000 7f00be112700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b810a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.940+0000 7f00be112700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b810a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58998/0 (socket says 172.21.15.150:58998) 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.940+0000 7f00be112700 1 -- 172.21.15.150:0/1516819563 learned_addr learned my addr 172.21.15.150:0/1516819563 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.940+0000 7f00be112700 1 -- 172.21.15.150:0/1516819563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b807af40 con 0x7f00b8107f30 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.940+0000 7f00be112700 1 --2- 172.21.15.150:0/1516819563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b810a350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b40098a0 tx=0x7f00b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dde77e6f9ac4b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:13.928 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.941+0000 7f00bd110700 1 -- 172.21.15.150:0/1516819563 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00b4004160 con 0x7f00b8107f30 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.941+0000 7f00bd110700 1 -- 172.21.15.150:0/1516819563 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00b40042c0 con 0x7f00b8107f30 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.941+0000 7f00bd110700 1 -- 172.21.15.150:0/1516819563 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00b40119b0 con 0x7f00b8107f30 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.941+0000 7f00c0376700 1 -- 172.21.15.150:0/1516819563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 msgr2=0x7f00b810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.941+0000 7f00c0376700 1 --2- 172.21.15.150:0/1516819563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b810a350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b40098a0 tx=0x7f00b4009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 -- 172.21.15.150:0/1516819563 shutdown_connections 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 --2- 172.21.15.150:0/1516819563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b810a350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 -- 172.21.15.150:0/1516819563 >> 172.21.15.150:0/1516819563 conn(0x7f00b81019d0 msgr2=0x7f00b8103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 -- 172.21.15.150:0/1516819563 shutdown_connections 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 -- 172.21.15.150:0/1516819563 wait complete. 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 Processor -- start 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 -- start start 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b81076b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.942+0000 7f00c0376700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b8107bf0 con 0x7f00b8107f30 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00be112700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b81076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:13.929 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00be112700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b81076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59004/0 (socket says 172.21.15.150:59004) 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00be112700 1 -- 172.21.15.150:0/169033956 learned_addr learned my addr 172.21.15.150:0/169033956 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00be112700 1 -- 172.21.15.150:0/169033956 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4009550 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00be112700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b81076b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f00b4006af0 tx=0x7f00b4011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00af7fe700 1 -- 172.21.15.150:0/169033956 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00b40038b0 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b8105e40 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00af7fe700 1 -- 172.21.15.150:0/169033956 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00b4003a10 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00af7fe700 1 -- 172.21.15.150:0/169033956 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00b4019550 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.943+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b81062e0 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.944+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b8062840 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.945+0000 7f00af7fe700 1 -- 172.21.15.150:0/169033956 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f00b4019810 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.945+0000 7f00af7fe700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f00a40386f0 0x7f00a403ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.945+0000 7f00af7fe700 1 -- 172.21.15.150:0/169033956 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f00b404ca20 con 0x7f00b8107f30 2024-05-10T04:33:13.930 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.946+0000 7f00bd911700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f00a40386f0 0x7f00a403ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.946+0000 7f00bd911700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f00a40386f0 0x7f00a403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:12.950+0000 7f00af7fe700 1 -- 172.21.15.150:0/169033956 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f00b4003b80 con 0x7f00b8107f30 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.170+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f00b8106750 con 0x7f00a40386f0 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.255+0000 7f00af7fe700 1 -- 172.21.15.150:0/169033956 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f00b8106750 con 0x7f00a40386f0 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f00a40386f0 msgr2=0x7f00a403ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f00a40386f0 0x7f00a403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a8006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 msgr2=0x7f00b81076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b81076b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f00b4006af0 tx=0x7f00b4011980 comp rx=0 tx=0).stop 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 shutdown_connections 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f00a40386f0 0x7f00a403ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 --2- 172.21.15.150:0/169033956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b8107f30 0x7f00b81076b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 >> 172.21.15.150:0/169033956 conn(0x7f00b81019d0 msgr2=0x7f00b81022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 shutdown_connections 2024-05-10T04:33:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:13.257+0000 7f00c0376700 1 -- 172.21.15.150:0/169033956 wait complete. 2024-05-10T04:33:13.932 INFO:teuthology.orchestra.run.smithi150.stdout:Creating initial admin user... 2024-05-10T04:33:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:14 smithi150 ceph-mon[42404]: [10/May/2024:04:33:12] ENGINE Bus STARTING 2024-05-10T04:33:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:14 smithi150 ceph-mon[42404]: [10/May/2024:04:33:12] ENGINE Serving on http://172.21.15.150:8765 2024-05-10T04:33:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:14 smithi150 ceph-mon[42404]: [10/May/2024:04:33:12] ENGINE Serving on https://172.21.15.150:7150 2024-05-10T04:33:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:14 smithi150 ceph-mon[42404]: [10/May/2024:04:33:12] ENGINE Bus STARTED 2024-05-10T04:33:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:14 smithi150 ceph-mon[42404]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:14.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:14 smithi150 ceph-mon[42404]: mgrmap e12: x(active, since 3s) 2024-05-10T04:33:15.039 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AmxvIjfugzG6OwsXuJd7D.dh7nBCmtFyWTuIXIC2FWd.0EdZtPd.S", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715315594, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-10T04:33:15.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.251+0000 7facf595b700 1 Processor -- start 2024-05-10T04:33:15.044 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.251+0000 7facf595b700 1 -- start start 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.251+0000 7facf595b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf010a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.251+0000 7facf595b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf007ae00 con 0x7facf0107f30 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.251+0000 7faceeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf010a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.251+0000 7faceeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf010a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59016/0 (socket says 172.21.15.150:59016) 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.251+0000 7faceeffd700 1 -- 172.21.15.150:0/2618564732 learned_addr learned my addr 172.21.15.150:0/2618564732 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.252+0000 7faceeffd700 1 -- 172.21.15.150:0/2618564732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf007af40 con 0x7facf0107f30 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.252+0000 7faceeffd700 1 --2- 172.21.15.150:0/2618564732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf010a350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7face00098a0 tx=0x7face0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a5a7691e8fe2c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.252+0000 7facedffb700 1 -- 172.21.15.150:0/2618564732 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7face0004160 con 0x7facf0107f30 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.252+0000 7facedffb700 1 -- 172.21.15.150:0/2618564732 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face00042c0 con 0x7facf0107f30 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.252+0000 7facedffb700 1 -- 172.21.15.150:0/2618564732 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7face00119b0 con 0x7facf0107f30 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 -- 172.21.15.150:0/2618564732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 msgr2=0x7facf010a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 --2- 172.21.15.150:0/2618564732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf010a350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7face00098a0 tx=0x7face0009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.045 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 -- 172.21.15.150:0/2618564732 shutdown_connections 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 --2- 172.21.15.150:0/2618564732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf010a350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 -- 172.21.15.150:0/2618564732 >> 172.21.15.150:0/2618564732 conn(0x7facf01019d0 msgr2=0x7facf0103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 -- 172.21.15.150:0/2618564732 shutdown_connections 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 -- 172.21.15.150:0/2618564732 wait complete. 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.253+0000 7facf595b700 1 Processor -- start 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7facf595b700 1 -- start start 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7facf595b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7facf595b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf019ba50 con 0x7facf0107f30 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7faceeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7faceeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59028/0 (socket says 172.21.15.150:59028) 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7faceeffd700 1 -- 172.21.15.150:0/4271322527 learned_addr learned my addr 172.21.15.150:0/4271322527 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7faceeffd700 1 -- 172.21.15.150:0/4271322527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0009550 con 0x7facf0107f30 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.254+0000 7faceeffd700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf019b510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7face0006af0 tx=0x7face0011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:15.046 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.255+0000 7facf4959700 1 -- 172.21.15.150:0/4271322527 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7face00038b0 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.255+0000 7facf4959700 1 -- 172.21.15.150:0/4271322527 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face0003a10 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.255+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf019bc50 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.255+0000 7facf4959700 1 -- 172.21.15.150:0/4271322527 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7face0019550 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.255+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf019c0f0 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.256+0000 7facf4959700 1 -- 172.21.15.150:0/4271322527 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7face00197c0 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.256+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf010b600 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.256+0000 7facf4959700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7facd8038810 0x7facd803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.256+0000 7facf4959700 1 -- 172.21.15.150:0/4271322527 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7face004c8c0 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.256+0000 7facee7fc700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7facd8038810 0x7facd803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.256+0000 7facee7fc700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7facd8038810 0x7facd803ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7face4006fd0 tx=0x7face4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.261+0000 7facf4959700 1 -- 172.21.15.150:0/4271322527 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7face0018d90 con 0x7facf0107f30 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.469+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7facf0062660 con 0x7facd8038810 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.687+0000 7facf4959700 1 -- 172.21.15.150:0/4271322527 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7facf0062660 con 0x7facd8038810 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7facd8038810 msgr2=0x7facd803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:15.047 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7facd8038810 0x7facd803ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7face4006fd0 tx=0x7face4006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 msgr2=0x7facf019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf019b510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7face0006af0 tx=0x7face0011980 comp rx=0 tx=0).stop 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 shutdown_connections 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7facd8038810 0x7facd803ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 --2- 172.21.15.150:0/4271322527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facf0107f30 0x7facf019b510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 >> 172.21.15.150:0/4271322527 conn(0x7facf01019d0 msgr2=0x7facf0102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 shutdown_connections 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:14.689+0000 7facf595b700 1 -- 172.21.15.150:0/4271322527 wait complete. 2024-05-10T04:33:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:Fetching dashboard port number... 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stdout 8443 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.356+0000 7f9f3f6b2700 1 Processor -- start 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.356+0000 7f9f3f6b2700 1 -- start start 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.356+0000 7f9f3f6b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38107c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.356+0000 7f9f3f6b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38108280 con 0x7f9f38107880 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f3d44e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38107c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f3d44e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38107c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59042/0 (socket says 172.21.15.150:59042) 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f3d44e700 1 -- 172.21.15.150:0/2353896344 learned_addr learned my addr 172.21.15.150:0/2353896344 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f3d44e700 1 -- 172.21.15.150:0/2353896344 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f381083c0 con 0x7f9f38107880 2024-05-10T04:33:15.902 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f3d44e700 1 --2- 172.21.15.150:0/2353896344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38107c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f280098a0 tx=0x7f9f28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddf18d1ec325ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f2ffff700 1 -- 172.21.15.150:0/2353896344 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f28004160 con 0x7f9f38107880 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f2ffff700 1 -- 172.21.15.150:0/2353896344 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f280042c0 con 0x7f9f38107880 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.357+0000 7f9f2ffff700 1 -- 172.21.15.150:0/2353896344 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f280119b0 con 0x7f9f38107880 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.358+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/2353896344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 msgr2=0x7f9f38107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.358+0000 7f9f3f6b2700 1 --2- 172.21.15.150:0/2353896344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38107c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f280098a0 tx=0x7f9f28009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.358+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/2353896344 shutdown_connections 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.358+0000 7f9f3f6b2700 1 --2- 172.21.15.150:0/2353896344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38107c90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.358+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/2353896344 >> 172.21.15.150:0/2353896344 conn(0x7f9f381011a0 msgr2=0x7f9f381035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.358+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/2353896344 shutdown_connections 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.358+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/2353896344 wait complete. 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3f6b2700 1 Processor -- start 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3f6b2700 1 -- start start 2024-05-10T04:33:15.903 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3f6b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3f6b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f381063b0 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3d44e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3d44e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59058/0 (socket says 172.21.15.150:59058) 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3d44e700 1 -- 172.21.15.150:0/188443713 learned_addr learned my addr 172.21.15.150:0/188443713 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3d44e700 1 -- 172.21.15.150:0/188443713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28009550 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f3d44e700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38105e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f28003f00 tx=0x7f9f28011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.359+0000 7f9f2e7fc700 1 -- 172.21.15.150:0/188443713 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f280038b0 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.360+0000 7f9f2e7fc700 1 -- 172.21.15.150:0/188443713 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f28003a10 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.360+0000 7f9f2e7fc700 1 -- 172.21.15.150:0/188443713 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f28019550 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.360+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3819e260 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.360+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3819e760 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.360+0000 7f9f2e7fc700 1 -- 172.21.15.150:0/188443713 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f9f28019780 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.360+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f38062840 con 0x7f9f38107880 2024-05-10T04:33:15.904 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.361+0000 7f9f2e7fc700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9f24038810 0x7f9f2403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.361+0000 7f9f2e7fc700 1 -- 172.21.15.150:0/188443713 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9f2804c970 con 0x7f9f38107880 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.361+0000 7f9f3cc4d700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9f24038810 0x7f9f2403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.361+0000 7f9f3cc4d700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9f24038810 0x7f9f2403ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f34006fd0 tx=0x7f9f34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.366+0000 7f9f2e7fc700 1 -- 172.21.15.150:0/188443713 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9f280187c0 con 0x7f9f38107880 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.558+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f9f3819ec60 con 0x7f9f38107880 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.558+0000 7f9f2e7fc700 1 -- 172.21.15.150:0/188443713 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f9f2800bea0 con 0x7f9f38107880 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9f24038810 msgr2=0x7f9f2403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9f24038810 0x7f9f2403ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f34006fd0 tx=0x7f9f34006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 msgr2=0x7f9f38105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38105e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f28003f00 tx=0x7f9f28011a30 comp rx=0 tx=0).stop 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 shutdown_connections 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9f24038810 0x7f9f2403ad20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 --2- 172.21.15.150:0/188443713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f38107880 0x7f9f38105e70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:15.905 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 >> 172.21.15.150:0/188443713 conn(0x7f9f381011a0 msgr2=0x7f9f38102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:15.906 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 shutdown_connections 2024-05-10T04:33:15.906 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:15.561+0000 7f9f3f6b2700 1 -- 172.21.15.150:0/188443713 wait complete. 2024-05-10T04:33:15.909 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-10T04:33:15.909 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout disabled 2024-05-10T04:33:15.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:15 smithi150 ceph-mon[42404]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:15.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:15 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:15.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:15 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/188443713' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-10T04:33:15.916 INFO:teuthology.orchestra.run.smithi150.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-10T04:33:15.916 INFO:teuthology.orchestra.run.smithi150.stdout:systemctl: stdout inactive 2024-05-10T04:33:15.916 INFO:teuthology.orchestra.run.smithi150.stdout:firewalld.service is not enabled 2024-05-10T04:33:15.916 INFO:teuthology.orchestra.run.smithi150.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-10T04:33:15.917 INFO:teuthology.orchestra.run.smithi150.stdout:Ceph Dashboard is now available at: 2024-05-10T04:33:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: URL: https://smithi150:8443/ 2024-05-10T04:33:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: User: admin 2024-05-10T04:33:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: Password: 6ja7zotdmy 2024-05-10T04:33:15.918 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:15.918 INFO:teuthology.orchestra.run.smithi150.stdout:Saving cluster configuration to /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config directory 2024-05-10T04:33:15.918 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling autotune for osd_memory_target 2024-05-10T04:33:16.852 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.264+0000 7f9a2f196700 1 Processor -- start 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.264+0000 7f9a2f196700 1 -- start start 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a2f196700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a28109f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a2f196700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2810a5a0 con 0x7f9a2807acf0 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a28109f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a28109f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59068/0 (socket says 172.21.15.150:59068) 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a2cf32700 1 -- 172.21.15.150:0/591334333 learned_addr learned my addr 172.21.15.150:0/591334333 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a2cf32700 1 -- 172.21.15.150:0/591334333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2810a6e0 con 0x7f9a2807acf0 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a2cf32700 1 --2- 172.21.15.150:0/591334333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a28109f90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf30321e2dcb192 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.265+0000 7f9a277fe700 1 -- 172.21.15.150:0/591334333 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c004160 con 0x7f9a2807acf0 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a277fe700 1 -- 172.21.15.150:0/591334333 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1c0042c0 con 0x7f9a2807acf0 2024-05-10T04:33:16.853 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a277fe700 1 -- 172.21.15.150:0/591334333 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c0119b0 con 0x7f9a2807acf0 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a2f196700 1 -- 172.21.15.150:0/591334333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 msgr2=0x7f9a28109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a2f196700 1 --2- 172.21.15.150:0/591334333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a28109f90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a2f196700 1 -- 172.21.15.150:0/591334333 shutdown_connections 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a2f196700 1 --2- 172.21.15.150:0/591334333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a28109f90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a2f196700 1 -- 172.21.15.150:0/591334333 >> 172.21.15.150:0/591334333 conn(0x7f9a281019c0 msgr2=0x7f9a28103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a2f196700 1 -- 172.21.15.150:0/591334333 shutdown_connections 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.266+0000 7f9a2f196700 1 -- 172.21.15.150:0/591334333 wait complete. 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.267+0000 7f9a2f196700 1 Processor -- start 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.267+0000 7f9a2f196700 1 -- start start 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.267+0000 7f9a2f196700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a2819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.267+0000 7f9a2f196700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2819dca0 con 0x7f9a2807acf0 2024-05-10T04:33:16.854 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.267+0000 7f9a2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a2819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.267+0000 7f9a2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a2819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59072/0 (socket says 172.21.15.150:59072) 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.267+0000 7f9a2cf32700 1 -- 172.21.15.150:0/882620452 learned_addr learned my addr 172.21.15.150:0/882620452 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.268+0000 7f9a2cf32700 1 -- 172.21.15.150:0/882620452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c009550 con 0x7f9a2807acf0 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.268+0000 7f9a2cf32700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a2819d760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c011670 tx=0x7f9a1c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.268+0000 7f9a25ffb700 1 -- 172.21.15.150:0/882620452 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c003af0 con 0x7f9a2807acf0 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.268+0000 7f9a25ffb700 1 -- 172.21.15.150:0/882620452 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1c003c50 con 0x7f9a2807acf0 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.268+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2819dea0 con 0x7f9a2807acf0 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.268+0000 7f9a25ffb700 1 -- 172.21.15.150:0/882620452 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c024730 con 0x7f9a2807acf0 2024-05-10T04:33:16.855 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.268+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2819e340 con 0x7f9a2807acf0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.269+0000 7f9a25ffb700 1 -- 172.21.15.150:0/882620452 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f9a1c024920 con 0x7f9a2807acf0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.269+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28197630 con 0x7f9a2807acf0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.269+0000 7f9a25ffb700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9a18038860 0x7f9a1803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.269+0000 7f9a25ffb700 1 -- 172.21.15.150:0/882620452 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a1c04c7b0 con 0x7f9a2807acf0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.270+0000 7f9a27fff700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9a18038860 0x7f9a1803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.270+0000 7f9a27fff700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9a18038860 0x7f9a1803ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a14006fd0 tx=0x7f9a14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.274+0000 7f9a25ffb700 1 -- 172.21.15.150:0/882620452 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9a1c018b80 con 0x7f9a2807acf0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.476+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f9a2810b5e0 con 0x7f9a2807acf0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.476+0000 7f9a25ffb700 1 -- 172.21.15.150:0/882620452 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f9a1c018350 con 0x7f9a2807acf0 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9a18038860 msgr2=0x7f9a1803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9a18038860 0x7f9a1803ad70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a14006fd0 tx=0x7f9a14006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 msgr2=0x7f9a2819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a2819d760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c011670 tx=0x7f9a1c011a30 comp rx=0 tx=0).stop 2024-05-10T04:33:16.856 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 shutdown_connections 2024-05-10T04:33:16.857 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9a18038860 0x7f9a1803ad70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:16.857 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 --2- 172.21.15.150:0/882620452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2807acf0 0x7f9a2819d760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:16.857 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.478+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 >> 172.21.15.150:0/882620452 conn(0x7f9a281019c0 msgr2=0x7f9a28105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:16.857 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.479+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 shutdown_connections 2024-05-10T04:33:16.857 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:16.479+0000 7f9a2f196700 1 -- 172.21.15.150:0/882620452 wait complete. 2024-05-10T04:33:17.829 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.164+0000 7fa43f0d3700 1 Processor -- start 2024-05-10T04:33:17.829 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.164+0000 7fa43f0d3700 1 -- start start 2024-05-10T04:33:17.829 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.164+0000 7fa43f0d3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa438105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:17.829 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.164+0000 7fa43f0d3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa438106490 con 0x7fa438105a90 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.164+0000 7fa43ce6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa438105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.164+0000 7fa43ce6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa438105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59080/0 (socket says 172.21.15.150:59080) 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.164+0000 7fa43ce6f700 1 -- 172.21.15.150:0/3234056773 learned_addr learned my addr 172.21.15.150:0/3234056773 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.165+0000 7fa43ce6f700 1 -- 172.21.15.150:0/3234056773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa438106bc0 con 0x7fa438105a90 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.165+0000 7fa43ce6f700 1 --2- 172.21.15.150:0/3234056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa438105ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0098a0 tx=0x7fa42c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a6c0a03ec8390c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.165+0000 7fa4377fe700 1 -- 172.21.15.150:0/3234056773 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa42c004160 con 0x7fa438105a90 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.165+0000 7fa4377fe700 1 -- 172.21.15.150:0/3234056773 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa42c0042c0 con 0x7fa438105a90 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.165+0000 7fa4377fe700 1 -- 172.21.15.150:0/3234056773 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa42c0119b0 con 0x7fa438105a90 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3234056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 msgr2=0x7fa438105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:17.830 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 --2- 172.21.15.150:0/3234056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa438105ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0098a0 tx=0x7fa42c009bb0 comp rx=0 tx=0).stop 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3234056773 shutdown_connections 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 --2- 172.21.15.150:0/3234056773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa438105ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3234056773 >> 172.21.15.150:0/3234056773 conn(0x7fa438101150 msgr2=0x7fa4381035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3234056773 shutdown_connections 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3234056773 wait complete. 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 Processor -- start 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.166+0000 7fa43f0d3700 1 -- start start 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43f0d3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa43819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43f0d3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43819aa00 con 0x7fa438105a90 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43ce6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa43819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43ce6f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa43819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59096/0 (socket says 172.21.15.150:59096) 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43ce6f700 1 -- 172.21.15.150:0/3014296563 learned_addr learned my addr 172.21.15.150:0/3014296563 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:17.831 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43ce6f700 1 -- 172.21.15.150:0/3014296563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c009550 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43ce6f700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa43819a4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa42c003f00 tx=0x7fa42c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa42c0038b0 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa42c003a10 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43819ac00 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa42c019550 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.167+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43819b0a0 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.168+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fa42c0197d0 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.168+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa438193fb0 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.169+0000 7fa435ffb700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4280387c0 0x7fa42803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.169+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa42c04ca20 con 0x7fa438105a90 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.169+0000 7fa437fff700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4280387c0 0x7fa42803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.169+0000 7fa437fff700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4280387c0 0x7fa42803acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa424006fd0 tx=0x7fa424006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:17.832 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.174+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa42c018d20 con 0x7fa438105a90 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.328+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fa42c02d960 con 0x7fa438105a90 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.478+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa438109df0 con 0x7fa438105a90 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.481+0000 7fa435ffb700 1 -- 172.21.15.150:0/3014296563 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa42c018d20 con 0x7fa438105a90 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4280387c0 msgr2=0x7fa42803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4280387c0 0x7fa42803acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa424006fd0 tx=0x7fa424006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 msgr2=0x7fa43819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa43819a4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa42c003f00 tx=0x7fa42c011a30 comp rx=0 tx=0).stop 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 shutdown_connections 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4280387c0 0x7fa42803acd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 --2- 172.21.15.150:0/3014296563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa438105a90 0x7fa43819a4c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:17.833 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 >> 172.21.15.150:0/3014296563 conn(0x7fa438101150 msgr2=0x7fa4381910f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 shutdown_connections 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr 2024-05-10T04:33:17.483+0000 7fa43f0d3700 1 -- 172.21.15.150:0/3014296563 wait complete. 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout:Or, if you are only running a single cluster on this host: 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.834 INFO:teuthology.orchestra.run.smithi150.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout: ceph telemetry on 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout:For more information see: 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:17.835 INFO:teuthology.orchestra.run.smithi150.stdout:Bootstrap complete. 2024-05-10T04:33:17.859 INFO:tasks.cephadm:Fetching config... 2024-05-10T04:33:17.860 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:33:17.860 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-10T04:33:17.877 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-10T04:33:17.877 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:33:17.877 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-10T04:33:17.936 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-10T04:33:17.936 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:33:17.936 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-10T04:33:18.008 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-10T04:33:18.009 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:33:18.009 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-10T04:33:18.067 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-10T04:33:18.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe0KzPJi3dfGbQEJEvpk6InMc5wxSLusLVu3ZTnAwu6n2HHH0FznMmtN8Xka9eeVBapg+AK5TFnLG59ssVsFhAXV0gyVaTHCKfPUWCadAsFaJwP1dNJISXqndcsSl1xv2dzZe5bY6tBjTyL9yuIQ5T6H6hcPhXrV5w8Zi5ovE8/wXdNQ0AEmGxjMDwSqy7J85JYahEgnLDUysyHkGtd3kNEORW7Lvk62uBvmIZgWoOCL1izEXPb8quAbazGlRvyghjK3fdLjDo56wUKUUD5dx0fh6e7qn5zI+kp5WAsqbRrmSJFNhkqiBzpZYziyhNsUh+S/G7Sv6RPDko95lyUTsBH/gd5Cu87QH7F0+MnHOnSANNEernkJnZ03jhC7fSPBC2wfTFRb6x29PdBL4/MA9UyqUcM3v/K4XBq1Bo5eEnwNTcGhJ1V1QEPaQPerFeM3Em4x8+1JPIWD5uhE7vJHfQuGW76Tex/l0vgOvJiFxD7gacWI99ng60S4KM2FV7sls= ceph-28928ad0-0e86-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T04:33:18.145 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe0KzPJi3dfGbQEJEvpk6InMc5wxSLusLVu3ZTnAwu6n2HHH0FznMmtN8Xka9eeVBapg+AK5TFnLG59ssVsFhAXV0gyVaTHCKfPUWCadAsFaJwP1dNJISXqndcsSl1xv2dzZe5bY6tBjTyL9yuIQ5T6H6hcPhXrV5w8Zi5ovE8/wXdNQ0AEmGxjMDwSqy7J85JYahEgnLDUysyHkGtd3kNEORW7Lvk62uBvmIZgWoOCL1izEXPb8quAbazGlRvyghjK3fdLjDo56wUKUUD5dx0fh6e7qn5zI+kp5WAsqbRrmSJFNhkqiBzpZYziyhNsUh+S/G7Sv6RPDko95lyUTsBH/gd5Cu87QH7F0+MnHOnSANNEernkJnZ03jhC7fSPBC2wfTFRb6x29PdBL4/MA9UyqUcM3v/K4XBq1Bo5eEnwNTcGhJ1V1QEPaQPerFeM3Em4x8+1JPIWD5uhE7vJHfQuGW76Tex/l0vgOvJiFxD7gacWI99ng60S4KM2FV7sls= ceph-28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:18.157 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe0KzPJi3dfGbQEJEvpk6InMc5wxSLusLVu3ZTnAwu6n2HHH0FznMmtN8Xka9eeVBapg+AK5TFnLG59ssVsFhAXV0gyVaTHCKfPUWCadAsFaJwP1dNJISXqndcsSl1xv2dzZe5bY6tBjTyL9yuIQ5T6H6hcPhXrV5w8Zi5ovE8/wXdNQ0AEmGxjMDwSqy7J85JYahEgnLDUysyHkGtd3kNEORW7Lvk62uBvmIZgWoOCL1izEXPb8quAbazGlRvyghjK3fdLjDo56wUKUUD5dx0fh6e7qn5zI+kp5WAsqbRrmSJFNhkqiBzpZYziyhNsUh+S/G7Sv6RPDko95lyUTsBH/gd5Cu87QH7F0+MnHOnSANNEernkJnZ03jhC7fSPBC2wfTFRb6x29PdBL4/MA9UyqUcM3v/K4XBq1Bo5eEnwNTcGhJ1V1QEPaQPerFeM3Em4x8+1JPIWD5uhE7vJHfQuGW76Tex/l0vgOvJiFxD7gacWI99ng60S4KM2FV7sls= ceph-28928ad0-0e86-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T04:33:18.203 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe0KzPJi3dfGbQEJEvpk6InMc5wxSLusLVu3ZTnAwu6n2HHH0FznMmtN8Xka9eeVBapg+AK5TFnLG59ssVsFhAXV0gyVaTHCKfPUWCadAsFaJwP1dNJISXqndcsSl1xv2dzZe5bY6tBjTyL9yuIQ5T6H6hcPhXrV5w8Zi5ovE8/wXdNQ0AEmGxjMDwSqy7J85JYahEgnLDUysyHkGtd3kNEORW7Lvk62uBvmIZgWoOCL1izEXPb8quAbazGlRvyghjK3fdLjDo56wUKUUD5dx0fh6e7qn5zI+kp5WAsqbRrmSJFNhkqiBzpZYziyhNsUh+S/G7Sv6RPDko95lyUTsBH/gd5Cu87QH7F0+MnHOnSANNEernkJnZ03jhC7fSPBC2wfTFRb6x29PdBL4/MA9UyqUcM3v/K4XBq1Bo5eEnwNTcGhJ1V1QEPaQPerFeM3Em4x8+1JPIWD5uhE7vJHfQuGW76Tex/l0vgOvJiFxD7gacWI99ng60S4KM2FV7sls= ceph-28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:18.216 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe0KzPJi3dfGbQEJEvpk6InMc5wxSLusLVu3ZTnAwu6n2HHH0FznMmtN8Xka9eeVBapg+AK5TFnLG59ssVsFhAXV0gyVaTHCKfPUWCadAsFaJwP1dNJISXqndcsSl1xv2dzZe5bY6tBjTyL9yuIQ5T6H6hcPhXrV5w8Zi5ovE8/wXdNQ0AEmGxjMDwSqy7J85JYahEgnLDUysyHkGtd3kNEORW7Lvk62uBvmIZgWoOCL1izEXPb8quAbazGlRvyghjK3fdLjDo56wUKUUD5dx0fh6e7qn5zI+kp5WAsqbRrmSJFNhkqiBzpZYziyhNsUh+S/G7Sv6RPDko95lyUTsBH/gd5Cu87QH7F0+MnHOnSANNEernkJnZ03jhC7fSPBC2wfTFRb6x29PdBL4/MA9UyqUcM3v/K4XBq1Bo5eEnwNTcGhJ1V1QEPaQPerFeM3Em4x8+1JPIWD5uhE7vJHfQuGW76Tex/l0vgOvJiFxD7gacWI99ng60S4KM2FV7sls= ceph-28928ad0-0e86-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-10T04:33:18.261 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe0KzPJi3dfGbQEJEvpk6InMc5wxSLusLVu3ZTnAwu6n2HHH0FznMmtN8Xka9eeVBapg+AK5TFnLG59ssVsFhAXV0gyVaTHCKfPUWCadAsFaJwP1dNJISXqndcsSl1xv2dzZe5bY6tBjTyL9yuIQ5T6H6hcPhXrV5w8Zi5ovE8/wXdNQ0AEmGxjMDwSqy7J85JYahEgnLDUysyHkGtd3kNEORW7Lvk62uBvmIZgWoOCL1izEXPb8quAbazGlRvyghjK3fdLjDo56wUKUUD5dx0fh6e7qn5zI+kp5WAsqbRrmSJFNhkqiBzpZYziyhNsUh+S/G7Sv6RPDko95lyUTsBH/gd5Cu87QH7F0+MnHOnSANNEernkJnZ03jhC7fSPBC2wfTFRb6x29PdBL4/MA9UyqUcM3v/K4XBq1Bo5eEnwNTcGhJ1V1QEPaQPerFeM3Em4x8+1JPIWD5uhE7vJHfQuGW76Tex/l0vgOvJiFxD7gacWI99ng60S4KM2FV7sls= ceph-28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:18.275 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-10T04:33:18.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:18 smithi150 ceph-mon[42404]: mgrmap e13: x(active, since 6s) 2024-05-10T04:33:18.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:18 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3014296563' entity='client.admin' 2024-05-10T04:33:18.503 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.574+0000 7f1ef07e3700 1 -- 172.21.15.150:0/2488656746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 msgr2=0x7f1ee8104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.574+0000 7f1ef07e3700 1 --2- 172.21.15.150:0/2488656746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8104450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009910 tx=0x7f1ee4009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.574+0000 7f1ef07e3700 1 -- 172.21.15.150:0/2488656746 shutdown_connections 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.574+0000 7f1ef07e3700 1 --2- 172.21.15.150:0/2488656746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8104450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.574+0000 7f1ef07e3700 1 -- 172.21.15.150:0/2488656746 >> 172.21.15.150:0/2488656746 conn(0x7f1ee80fba50 msgr2=0x7f1ee80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.575+0000 7f1ef07e3700 1 -- 172.21.15.150:0/2488656746 shutdown_connections 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.575+0000 7f1ef07e3700 1 -- 172.21.15.150:0/2488656746 wait complete. 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.575+0000 7f1ef07e3700 1 Processor -- start 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.575+0000 7f1ef07e3700 1 -- start start 2024-05-10T04:33:19.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.575+0000 7f1ef07e3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1ef07e3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8197370 con 0x7f1ee8102030 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1eee57f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1eee57f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35502/0 (socket says 172.21.15.150:35502) 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1eee57f700 1 -- 172.21.15.150:0/3899182761 learned_addr learned my addr 172.21.15.150:0/3899182761 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1eee57f700 1 -- 172.21.15.150:0/3899182761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40095f0 con 0x7f1ee8102030 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1eee57f700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8196e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40092e0 tx=0x7f1ee4009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1edf7fe700 1 -- 172.21.15.150:0/3899182761 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ee401d070 con 0x7f1ee8102030 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee8197570 con 0x7f1ee8102030 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1edf7fe700 1 -- 172.21.15.150:0/3899182761 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ee400be20 con 0x7f1ee8102030 2024-05-10T04:33:19.579 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1edf7fe700 1 -- 172.21.15.150:0/3899182761 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ee4022d60 con 0x7f1ee8102030 2024-05-10T04:33:19.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.576+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee8197a10 con 0x7f1ee8102030 2024-05-10T04:33:19.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.577+0000 7f1edf7fe700 1 -- 172.21.15.150:0/3899182761 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f1ee400f460 con 0x7f1ee8102030 2024-05-10T04:33:19.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.578+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee8062840 con 0x7f1ee8102030 2024-05-10T04:33:19.580 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.578+0000 7f1edf7fe700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1ed40387a0 0x7f1ed403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:19.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.578+0000 7f1edf7fe700 1 -- 172.21.15.150:0/3899182761 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ee4022500 con 0x7f1ee8102030 2024-05-10T04:33:19.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.578+0000 7f1eedd7e700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1ed40387a0 0x7f1ed403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:19.581 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.579+0000 7f1eedd7e700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1ed40387a0 0x7f1ed403acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006fd0 tx=0x7f1ed8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:19.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.584+0000 7f1edf7fe700 1 -- 172.21.15.150:0/3899182761 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ee4017440 con 0x7f1ee8102030 2024-05-10T04:33:19.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.782+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f1ee81902c0 con 0x7f1ee8102030 2024-05-10T04:33:19.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.786+0000 7f1edf7fe700 1 -- 172.21.15.150:0/3899182761 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1ee4027070 con 0x7f1ee8102030 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1ed40387a0 msgr2=0x7f1ed403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1ed40387a0 0x7f1ed403acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006fd0 tx=0x7f1ed8006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 msgr2=0x7f1ee8196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8196e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40092e0 tx=0x7f1ee4009f40 comp rx=0 tx=0).stop 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 shutdown_connections 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1ed40387a0 0x7f1ed403acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 --2- 172.21.15.150:0/3899182761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee8102030 0x7f1ee8196e30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:19.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.788+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 >> 172.21.15.150:0/3899182761 conn(0x7f1ee80fba50 msgr2=0x7f1ee80fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:19.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.789+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 shutdown_connections 2024-05-10T04:33:19.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:19.789+0000 7f1ef07e3700 1 -- 172.21.15.150:0/3899182761 wait complete. 2024-05-10T04:33:20.150 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-10T04:33:20.151 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-10T04:33:20.403 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:21.040 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:20 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3899182761' entity='client.admin' 2024-05-10T04:33:21.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 -- 172.21.15.150:0/3659370880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 msgr2=0x7ff1ec104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:21.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 --2- 172.21.15.150:0/3659370880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec104450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009910 tx=0x7ff1dc009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:21.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 -- 172.21.15.150:0/3659370880 shutdown_connections 2024-05-10T04:33:21.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 --2- 172.21.15.150:0/3659370880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec104450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:21.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 -- 172.21.15.150:0/3659370880 >> 172.21.15.150:0/3659370880 conn(0x7ff1ec0fba50 msgr2=0x7ff1ec0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:21.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 -- 172.21.15.150:0/3659370880 shutdown_connections 2024-05-10T04:33:21.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 -- 172.21.15.150:0/3659370880 wait complete. 2024-05-10T04:33:21.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.484+0000 7ff1f24ad700 1 Processor -- start 2024-05-10T04:33:21.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1f24ad700 1 -- start start 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1f24ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1f24ad700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1995a0 con 0x7ff1ec102030 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35520/0 (socket says 172.21.15.150:35520) 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1ebfff700 1 -- 172.21.15.150:0/504761234 learned_addr learned my addr 172.21.15.150:0/504761234 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1ebfff700 1 -- 172.21.15.150:0/504761234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0095f0 con 0x7ff1ec102030 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.485+0000 7ff1ebfff700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec199060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0092e0 tx=0x7ff1dc009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.486+0000 7ff1e97fa700 1 -- 172.21.15.150:0/504761234 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1dc01d070 con 0x7ff1ec102030 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.486+0000 7ff1e97fa700 1 -- 172.21.15.150:0/504761234 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1dc00be20 con 0x7ff1ec102030 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.486+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec1997a0 con 0x7ff1ec102030 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.486+0000 7ff1e97fa700 1 -- 172.21.15.150:0/504761234 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1dc022d60 con 0x7ff1ec102030 2024-05-10T04:33:21.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.486+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec199c40 con 0x7ff1ec102030 2024-05-10T04:33:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.487+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec1926c0 con 0x7ff1ec102030 2024-05-10T04:33:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.487+0000 7ff1e97fa700 1 -- 172.21.15.150:0/504761234 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7ff1dc00f460 con 0x7ff1ec102030 2024-05-10T04:33:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.487+0000 7ff1e97fa700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff1d40387f0 0x7ff1d403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.487+0000 7ff1e97fa700 1 -- 172.21.15.150:0/504761234 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff1dc04c890 con 0x7ff1ec102030 2024-05-10T04:33:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.487+0000 7ff1eb7fe700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff1d40387f0 0x7ff1d403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:21.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.487+0000 7ff1eb7fe700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff1d40387f0 0x7ff1d403ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0006fd0 tx=0x7ff1e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:21.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.492+0000 7ff1e97fa700 1 -- 172.21.15.150:0/504761234 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff1dc020920 con 0x7ff1ec102030 2024-05-10T04:33:21.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.686+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1ec061650 con 0x7ff1d40387f0 2024-05-10T04:33:21.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.690+0000 7ff1e97fa700 1 -- 172.21.15.150:0/504761234 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff1ec061650 con 0x7ff1d40387f0 2024-05-10T04:33:21.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff1d40387f0 msgr2=0x7ff1d403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:21.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff1d40387f0 0x7ff1d403ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0006fd0 tx=0x7ff1e0006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:21.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 msgr2=0x7ff1ec199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:21.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec199060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0092e0 tx=0x7ff1dc009f40 comp rx=0 tx=0).stop 2024-05-10T04:33:21.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 shutdown_connections 2024-05-10T04:33:21.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff1d40387f0 0x7ff1d403ad00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:21.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 --2- 172.21.15.150:0/504761234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1ec102030 0x7ff1ec199060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:21.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 >> 172.21.15.150:0/504761234 conn(0x7ff1ec0fba50 msgr2=0x7ff1ec0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:21.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 shutdown_connections 2024-05-10T04:33:21.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:21.692+0000 7ff1f24ad700 1 -- 172.21.15.150:0/504761234 wait complete. 2024-05-10T04:33:21.982 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:21.982 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:21.982 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:33:21.982 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:21.982 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:21.982 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:21.983 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:21.983 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:21.983 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:21.983 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:22.087 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-05-10T04:33:22.088 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:33:22.088 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-05-10T04:33:22.109 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:33:22.109 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:22.169 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-05-10T04:33:22.170 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch host add smithi161 2024-05-10T04:33:22.407 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:23.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:22 smithi150 ceph-mon[42404]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:23.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:22 smithi150 ceph-mon[42404]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-10T04:33:23.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:22 smithi150 ceph-mon[42404]: Updating smithi150:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T04:33:23.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:22 smithi150 ceph-mon[42404]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:23.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:23.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:23.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.332+0000 7f05030f2700 1 -- 172.21.15.150:0/2796712629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 msgr2=0x7f04fc1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.332+0000 7f05030f2700 1 --2- 172.21.15.150:0/2796712629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1068a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009910 tx=0x7f04f0009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.332+0000 7f05030f2700 1 -- 172.21.15.150:0/2796712629 shutdown_connections 2024-05-10T04:33:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.332+0000 7f05030f2700 1 --2- 172.21.15.150:0/2796712629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1068a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.332+0000 7f05030f2700 1 -- 172.21.15.150:0/2796712629 >> 172.21.15.150:0/2796712629 conn(0x7f04fc0fbef0 msgr2=0x7f04fc0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.332+0000 7f05030f2700 1 -- 172.21.15.150:0/2796712629 shutdown_connections 2024-05-10T04:33:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.332+0000 7f05030f2700 1 -- 172.21.15.150:0/2796712629 wait complete. 2024-05-10T04:33:23.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.333+0000 7f05030f2700 1 Processor -- start 2024-05-10T04:33:23.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.333+0000 7f05030f2700 1 -- start start 2024-05-10T04:33:23.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.333+0000 7f05030f2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:23.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.333+0000 7f05030f2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc1975f0 con 0x7f04fc106490 2024-05-10T04:33:23.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.333+0000 7f0500e8e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.333+0000 7f0500e8e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35534/0 (socket says 172.21.15.150:35534) 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.333+0000 7f0500e8e700 1 -- 172.21.15.150:0/1363409234 learned_addr learned my addr 172.21.15.150:0/1363409234 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.334+0000 7f0500e8e700 1 -- 172.21.15.150:0/1363409234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00095f0 con 0x7f04fc106490 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.334+0000 7f0500e8e700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1970b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04f00092e0 tx=0x7f04f0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.334+0000 7f04f9ffb700 1 -- 172.21.15.150:0/1363409234 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04f001d070 con 0x7f04fc106490 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.334+0000 7f04f9ffb700 1 -- 172.21.15.150:0/1363409234 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f000b6f0 con 0x7f04fc106490 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.334+0000 7f04f9ffb700 1 -- 172.21.15.150:0/1363409234 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04f0003810 con 0x7f04fc106490 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.334+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc197850 con 0x7f04fc106490 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.334+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc197cd0 con 0x7f04fc106490 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.335+0000 7f04f9ffb700 1 -- 172.21.15.150:0/1363409234 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f04f0004260 con 0x7f04fc106490 2024-05-10T04:33:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.335+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc101b10 con 0x7f04fc106490 2024-05-10T04:33:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.335+0000 7f04f9ffb700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04ec0387f0 0x7f04ec03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.335+0000 7f04f9ffb700 1 -- 172.21.15.150:0/1363409234 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f04f004c3e0 con 0x7f04fc106490 2024-05-10T04:33:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.335+0000 7f04fbfff700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04ec0387f0 0x7f04ec03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.336+0000 7f04fbfff700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04ec0387f0 0x7f04ec03ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04e8006fd0 tx=0x7f04e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:23.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.341+0000 7f04f9ffb700 1 -- 172.21.15.150:0/1363409234 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f04f0029540 con 0x7f04fc106490 2024-05-10T04:33:23.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:23.536+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) v1 -- 0x7f04fc10a7d0 con 0x7f04ec0387f0 2024-05-10T04:33:24.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:23 smithi150 ceph-mon[42404]: Updating smithi150:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T04:33:25.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:24 smithi150 ceph-mon[42404]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:25.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:24 smithi150 ceph-mon[42404]: Deploying cephadm binary to smithi161 2024-05-10T04:33:25.714 INFO:teuthology.orchestra.run.smithi150.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-05-10T04:33:25.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.713+0000 7f04f9ffb700 1 -- 172.21.15.150:0/1363409234 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f04fc10a7d0 con 0x7f04ec0387f0 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04ec0387f0 msgr2=0x7f04ec03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04ec0387f0 0x7f04ec03ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04e8006fd0 tx=0x7f04e8006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 msgr2=0x7f04fc1970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1970b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04f00092e0 tx=0x7f04f0004ff0 comp rx=0 tx=0).stop 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 shutdown_connections 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04ec0387f0 0x7f04ec03ad00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 --2- 172.21.15.150:0/1363409234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04fc106490 0x7f04fc1970b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:25.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.715+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 >> 172.21.15.150:0/1363409234 conn(0x7f04fc0fbef0 msgr2=0x7f04fc103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:25.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.716+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 shutdown_connections 2024-05-10T04:33:25.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:25.716+0000 7f05030f2700 1 -- 172.21.15.150:0/1363409234 wait complete. 2024-05-10T04:33:26.074 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T04:33:26.289 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:26.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:26.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:26 smithi150 ceph-mon[42404]: Added host smithi161 2024-05-10T04:33:26.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:26.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:26.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:27.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.156+0000 7f5b20f3e700 1 -- 172.21.15.150:0/2866442893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 msgr2=0x7f5b1c102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:27.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.156+0000 7f5b20f3e700 1 --2- 172.21.15.150:0/2866442893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c102a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009910 tx=0x7f5b0c009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:27.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.156+0000 7f5b20f3e700 1 -- 172.21.15.150:0/2866442893 shutdown_connections 2024-05-10T04:33:27.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.156+0000 7f5b20f3e700 1 --2- 172.21.15.150:0/2866442893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c102a80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:27.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.156+0000 7f5b20f3e700 1 -- 172.21.15.150:0/2866442893 >> 172.21.15.150:0/2866442893 conn(0x7f5b1c075f10 msgr2=0x7f5b1c076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:27.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.156+0000 7f5b20f3e700 1 -- 172.21.15.150:0/2866442893 shutdown_connections 2024-05-10T04:33:27.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.156+0000 7f5b20f3e700 1 -- 172.21.15.150:0/2866442893 wait complete. 2024-05-10T04:33:27.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.157+0000 7f5b20f3e700 1 Processor -- start 2024-05-10T04:33:27.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.157+0000 7f5b20f3e700 1 -- start start 2024-05-10T04:33:27.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.157+0000 7f5b20f3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:27.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.157+0000 7f5b20f3e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c197630 con 0x7f5b1c102670 2024-05-10T04:33:27.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.157+0000 7f5b1a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.157+0000 7f5b1a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:35552/0 (socket says 172.21.15.150:35552) 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.157+0000 7f5b1a59c700 1 -- 172.21.15.150:0/3546922051 learned_addr learned my addr 172.21.15.150:0/3546922051 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.158+0000 7f5b1a59c700 1 -- 172.21.15.150:0/3546922051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0095f0 con 0x7f5b1c102670 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.158+0000 7f5b1a59c700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c1970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c006010 tx=0x7f5b0c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.158+0000 7f5b0b7fe700 1 -- 172.21.15.150:0/3546922051 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c01d070 con 0x7f5b1c102670 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.158+0000 7f5b0b7fe700 1 -- 172.21.15.150:0/3546922051 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0c00bc10 con 0x7f5b1c102670 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.158+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c197830 con 0x7f5b1c102670 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.158+0000 7f5b0b7fe700 1 -- 172.21.15.150:0/3546922051 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c0038f0 con 0x7f5b1c102670 2024-05-10T04:33:27.160 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.158+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c197c50 con 0x7f5b1c102670 2024-05-10T04:33:27.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.159+0000 7f5b0b7fe700 1 -- 172.21.15.150:0/3546922051 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f5b0c003b60 con 0x7f5b1c102670 2024-05-10T04:33:27.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.159+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c068e30 con 0x7f5b1c102670 2024-05-10T04:33:27.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.159+0000 7f5b0b7fe700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5b040387a0 0x7f5b0403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:27.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.159+0000 7f5b0b7fe700 1 -- 172.21.15.150:0/3546922051 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5b0c04c490 con 0x7f5b1c102670 2024-05-10T04:33:27.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.159+0000 7f5b19d9b700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5b040387a0 0x7f5b0403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:27.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.160+0000 7f5b19d9b700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5b040387a0 0x7f5b0403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b10006fd0 tx=0x7f5b10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:27.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.164+0000 7f5b0b7fe700 1 -- 172.21.15.150:0/3546922051 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5b0c029950 con 0x7f5b1c102670 2024-05-10T04:33:27.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.363+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b1c10b880 con 0x7f5b040387a0 2024-05-10T04:33:27.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.364+0000 7f5b0b7fe700 1 -- 172.21.15.150:0/3546922051 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f5b1c10b880 con 0x7f5b040387a0 2024-05-10T04:33:27.366 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:27.366 INFO:teuthology.orchestra.run.smithi150.stdout:[{"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-05-10T04:33:27.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.366+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5b040387a0 msgr2=0x7f5b0403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:27.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.366+0000 7f5b20f3e700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5b040387a0 0x7f5b0403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b10006fd0 tx=0x7f5b10006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:27.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.366+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 msgr2=0x7f5b1c1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:27.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.366+0000 7f5b20f3e700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c1970f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c006010 tx=0x7f5b0c004ff0 comp rx=0 tx=0).stop 2024-05-10T04:33:27.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.367+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 shutdown_connections 2024-05-10T04:33:27.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.367+0000 7f5b20f3e700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5b040387a0 0x7f5b0403acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:27.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.367+0000 7f5b20f3e700 1 --2- 172.21.15.150:0/3546922051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b1c102670 0x7f5b1c1970f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:27.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.367+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 >> 172.21.15.150:0/3546922051 conn(0x7f5b1c075f10 msgr2=0x7f5b1c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:27.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.367+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 shutdown_connections 2024-05-10T04:33:27.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:27.367+0000 7f5b20f3e700 1 -- 172.21.15.150:0/3546922051 wait complete. 2024-05-10T04:33:27.715 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2024-05-10T04:33:27.715 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:33:27.715 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2024-05-10T04:33:27.736 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:33:27.736 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:27.797 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2024-05-10T04:33:27.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch host add smithi181 2024-05-10T04:33:28.014 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:28.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:28 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:28.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.929+0000 7f2585897700 1 -- 172.21.15.150:0/2943120808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 msgr2=0x7f25801068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:28.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.929+0000 7f2585897700 1 --2- 172.21.15.150:0/2943120808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801068b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2574009910 tx=0x7f2574009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.929+0000 7f2585897700 1 -- 172.21.15.150:0/2943120808 shutdown_connections 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.929+0000 7f2585897700 1 --2- 172.21.15.150:0/2943120808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801068b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.929+0000 7f2585897700 1 -- 172.21.15.150:0/2943120808 >> 172.21.15.150:0/2943120808 conn(0x7f2580075670 msgr2=0x7f2580075ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.929+0000 7f2585897700 1 -- 172.21.15.150:0/2943120808 shutdown_connections 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.929+0000 7f2585897700 1 -- 172.21.15.150:0/2943120808 wait complete. 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.930+0000 7f2585897700 1 Processor -- start 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.930+0000 7f2585897700 1 -- start start 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.930+0000 7f2585897700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.930+0000 7f2585897700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2580197620 con 0x7f25801064a0 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.930+0000 7f257effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.930+0000 7f257effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58126/0 (socket says 172.21.15.150:58126) 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f257effd700 1 -- 172.21.15.150:0/2540251615 learned_addr learned my addr 172.21.15.150:0/2540251615 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:28.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f257effd700 1 -- 172.21.15.150:0/2540251615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740095f0 con 0x7f25801064a0 2024-05-10T04:33:28.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f257effd700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801970e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2574004f40 tx=0x7f2574005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:28.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f2584895700 1 -- 172.21.15.150:0/2540251615 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f257401d070 con 0x7f25801064a0 2024-05-10T04:33:28.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2580197820 con 0x7f25801064a0 2024-05-10T04:33:28.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f2584895700 1 -- 172.21.15.150:0/2540251615 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257400be20 con 0x7f25801064a0 2024-05-10T04:33:28.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f2584895700 1 -- 172.21.15.150:0/2540251615 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2574022d30 con 0x7f25801064a0 2024-05-10T04:33:28.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.931+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2580197c40 con 0x7f25801064a0 2024-05-10T04:33:28.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.932+0000 7f2584895700 1 -- 172.21.15.150:0/2540251615 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f257400f460 con 0x7f25801064a0 2024-05-10T04:33:28.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.932+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2580101ce0 con 0x7f25801064a0 2024-05-10T04:33:28.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.932+0000 7f2584895700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25680387a0 0x7f256803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:28.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.932+0000 7f2584895700 1 -- 172.21.15.150:0/2540251615 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f257404c7b0 con 0x7f25801064a0 2024-05-10T04:33:28.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.932+0000 7f257e7fc700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25680387a0 0x7f256803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:28.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.933+0000 7f257e7fc700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25680387a0 0x7f256803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2570006fd0 tx=0x7f2570006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:28.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:28.937+0000 7f2584895700 1 -- 172.21.15.150:0/2540251615 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2574020960 con 0x7f25801064a0 2024-05-10T04:33:29.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:29.135+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) v1 -- 0x7f258010a7b0 con 0x7f25680387a0 2024-05-10T04:33:29.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:29 smithi150 ceph-mon[42404]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:33:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:30.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:30.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:33:30.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:30.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:31.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.301+0000 7f2584895700 1 -- 172.21.15.150:0/2540251615 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f258010a7b0 con 0x7f25680387a0 2024-05-10T04:33:31.303 INFO:teuthology.orchestra.run.smithi150.stdout:Added host 'smithi181' with addr '172.21.15.181' 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25680387a0 msgr2=0x7f256803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25680387a0 0x7f256803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2570006fd0 tx=0x7f2570006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 msgr2=0x7f25801970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801970e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2574004f40 tx=0x7f2574005020 comp rx=0 tx=0).stop 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 shutdown_connections 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25680387a0 0x7f256803acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 --2- 172.21.15.150:0/2540251615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25801064a0 0x7f25801970e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 >> 172.21.15.150:0/2540251615 conn(0x7f2580075670 msgr2=0x7f2580103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 shutdown_connections 2024-05-10T04:33:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:31.303+0000 7f2585897700 1 -- 172.21.15.150:0/2540251615 wait complete. 2024-05-10T04:33:31.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:31 smithi150 ceph-mon[42404]: Updating smithi161:/etc/ceph/ceph.conf 2024-05-10T04:33:31.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:31 smithi150 ceph-mon[42404]: Updating smithi161:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T04:33:31.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:31 smithi150 ceph-mon[42404]: Deploying cephadm binary to smithi181 2024-05-10T04:33:31.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:31 smithi150 ceph-mon[42404]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:31.327 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:31 smithi150 ceph-mon[42404]: Updating smithi161:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T04:33:31.666 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-10T04:33:31.882 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: Added host smithi181 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:32.261 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:32.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 -- 172.21.15.150:0/1501188385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 msgr2=0x7f74241057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:32.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 --2- 172.21.15.150:0/1501188385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f74241057a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7414009910 tx=0x7f7414009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:32.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 -- 172.21.15.150:0/1501188385 shutdown_connections 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 --2- 172.21.15.150:0/1501188385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f74241057a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 -- 172.21.15.150:0/1501188385 >> 172.21.15.150:0/1501188385 conn(0x7f74240fb7b0 msgr2=0x7f74240fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 -- 172.21.15.150:0/1501188385 shutdown_connections 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 -- 172.21.15.150:0/1501188385 wait complete. 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.838+0000 7f74297d6700 1 Processor -- start 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f74297d6700 1 -- start start 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f74297d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f7424195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f74297d6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7424196470 con 0x7f7424105390 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f7422ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f7424195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f7422ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f7424195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58136/0 (socket says 172.21.15.150:58136) 2024-05-10T04:33:32.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f7422ffd700 1 -- 172.21.15.150:0/3891575260 learned_addr learned my addr 172.21.15.150:0/3891575260 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f7422ffd700 1 -- 172.21.15.150:0/3891575260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74140095f0 con 0x7f7424105390 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.839+0000 7f7422ffd700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f7424195f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7414004f40 tx=0x7f7414005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.840+0000 7f740bfff700 1 -- 172.21.15.150:0/3891575260 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f741401d070 con 0x7f7424105390 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.840+0000 7f740bfff700 1 -- 172.21.15.150:0/3891575260 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f741400bc10 con 0x7f7424105390 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.840+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7424196670 con 0x7f7424105390 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.840+0000 7f740bfff700 1 -- 172.21.15.150:0/3891575260 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74140037d0 con 0x7f7424105390 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.840+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7424196b10 con 0x7f7424105390 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.841+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7424101b70 con 0x7f7424105390 2024-05-10T04:33:32.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.841+0000 7f740bfff700 1 -- 172.21.15.150:0/3891575260 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f7414003a90 con 0x7f7424105390 2024-05-10T04:33:32.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.841+0000 7f740bfff700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f740c0387a0 0x7f740c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:32.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.841+0000 7f740bfff700 1 -- 172.21.15.150:0/3891575260 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f741404c3f0 con 0x7f7424105390 2024-05-10T04:33:32.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.841+0000 7f74227fc700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f740c0387a0 0x7f740c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:32.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.842+0000 7f74227fc700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f740c0387a0 0x7f740c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7418006fd0 tx=0x7f7418006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:32.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:32.846+0000 7f740bfff700 1 -- 172.21.15.150:0/3891575260 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f74140293a0 con 0x7f7424105390 2024-05-10T04:33:33.045 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.044+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f742410a810 con 0x7f740c0387a0 2024-05-10T04:33:33.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.045+0000 7f740bfff700 1 -- 172.21.15.150:0/3891575260 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f742410a810 con 0x7f740c0387a0 2024-05-10T04:33:33.047 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:33:33.047 INFO:teuthology.orchestra.run.smithi150.stdout:[{"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f740c0387a0 msgr2=0x7f740c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f740c0387a0 0x7f740c03acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7418006fd0 tx=0x7f7418006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 msgr2=0x7f7424195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f7424195f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7414004f40 tx=0x7f7414005020 comp rx=0 tx=0).stop 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 shutdown_connections 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f740c0387a0 0x7f740c03acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 --2- 172.21.15.150:0/3891575260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7424105390 0x7f7424195f30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:33.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.047+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 >> 172.21.15.150:0/3891575260 conn(0x7f74240fb7b0 msgr2=0x7f7424101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:33.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.048+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 shutdown_connections 2024-05-10T04:33:33.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:33.048+0000 7f74297d6700 1 -- 172.21.15.150:0/3891575260 wait complete. 2024-05-10T04:33:33.449 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-10T04:33:33.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-10T04:33:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:34.060 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:33 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:34.060 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:33 smithi150 ceph-mon[42404]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:33:34.060 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:33 smithi150 ceph-mon[42404]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:34.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.556+0000 7f5392bd1700 1 -- 172.21.15.150:0/2268355561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 msgr2=0x7f538c102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:34.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.556+0000 7f5392bd1700 1 --2- 172.21.15.150:0/2268355561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c102a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5380009910 tx=0x7f5380009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:34.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.556+0000 7f5392bd1700 1 -- 172.21.15.150:0/2268355561 shutdown_connections 2024-05-10T04:33:34.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.556+0000 7f5392bd1700 1 --2- 172.21.15.150:0/2268355561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c102a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:34.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.556+0000 7f5392bd1700 1 -- 172.21.15.150:0/2268355561 >> 172.21.15.150:0/2268355561 conn(0x7f538c075f10 msgr2=0x7f538c076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:34.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.556+0000 7f5392bd1700 1 -- 172.21.15.150:0/2268355561 shutdown_connections 2024-05-10T04:33:34.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.556+0000 7f5392bd1700 1 -- 172.21.15.150:0/2268355561 wait complete. 2024-05-10T04:33:34.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f5392bd1700 1 Processor -- start 2024-05-10T04:33:34.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f5392bd1700 1 -- start start 2024-05-10T04:33:34.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f5392bd1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:34.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f5392bd1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c197630 con 0x7f538c102670 2024-05-10T04:33:34.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f539096d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f539096d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58148/0 (socket says 172.21.15.150:58148) 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f539096d700 1 -- 172.21.15.150:0/2551684131 learned_addr learned my addr 172.21.15.150:0/2551684131 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f539096d700 1 -- 172.21.15.150:0/2551684131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53800095f0 con 0x7f538c102670 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.557+0000 7f539096d700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c1970f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5380004f40 tx=0x7f5380005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.558+0000 7f5389ffb700 1 -- 172.21.15.150:0/2551684131 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f538001d070 con 0x7f538c102670 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.558+0000 7f5389ffb700 1 -- 172.21.15.150:0/2551684131 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5380004370 con 0x7f538c102670 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.558+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c197830 con 0x7f538c102670 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.558+0000 7f5389ffb700 1 -- 172.21.15.150:0/2551684131 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5380003970 con 0x7f538c102670 2024-05-10T04:33:34.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.558+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c197c50 con 0x7f538c102670 2024-05-10T04:33:34.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.559+0000 7f5389ffb700 1 -- 172.21.15.150:0/2551684131 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f5380003be0 con 0x7f538c102670 2024-05-10T04:33:34.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.559+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c068e30 con 0x7f538c102670 2024-05-10T04:33:34.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.559+0000 7f5389ffb700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f537c0387f0 0x7f537c03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:34.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.559+0000 7f5389ffb700 1 -- 172.21.15.150:0/2551684131 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f538004c450 con 0x7f538c102670 2024-05-10T04:33:34.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.559+0000 7f538bfff700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f537c0387f0 0x7f537c03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:34.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.559+0000 7f538bfff700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f537c0387f0 0x7f537c03ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5378006fd0 tx=0x7f5378006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:34.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.564+0000 7f5389ffb700 1 -- 172.21.15.150:0/2551684131 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5380029950 con 0x7f538c102670 2024-05-10T04:33:34.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.762+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f538c10b880 con 0x7f538c102670 2024-05-10T04:33:34.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.816+0000 7f5389ffb700 1 -- 172.21.15.150:0/2551684131 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5380026090 con 0x7f538c102670 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.818+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f537c0387f0 msgr2=0x7f537c03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.818+0000 7f5392bd1700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f537c0387f0 0x7f537c03ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5378006fd0 tx=0x7f5378006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.818+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 msgr2=0x7f538c1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.818+0000 7f5392bd1700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c1970f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5380004f40 tx=0x7f5380005020 comp rx=0 tx=0).stop 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.819+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 shutdown_connections 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.819+0000 7f5392bd1700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f537c0387f0 0x7f537c03ad00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.819+0000 7f5392bd1700 1 --2- 172.21.15.150:0/2551684131 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f538c102670 0x7f538c1970f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.819+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 >> 172.21.15.150:0/2551684131 conn(0x7f538c075f10 msgr2=0x7f538c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:34.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.819+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 shutdown_connections 2024-05-10T04:33:34.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:34.819+0000 7f5392bd1700 1 -- 172.21.15.150:0/2551684131 wait complete. 2024-05-10T04:33:34.821 INFO:teuthology.orchestra.run.smithi150.stderr:adjusted tunables profile to default 2024-05-10T04:33:35.073 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2551684131' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-10T04:33:35.173 INFO:tasks.cephadm:Adding mon.a on smithi150 2024-05-10T04:33:35.173 INFO:tasks.cephadm:Adding mon.b on smithi161 2024-05-10T04:33:35.173 INFO:tasks.cephadm:Adding mon.c on smithi181 2024-05-10T04:33:35.173 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c' 2024-05-10T04:33:35.430 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2551684131' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:36.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T04:33:36.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: Updating smithi181:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T04:33:36.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:35 smithi150 ceph-mon[42404]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 -- 172.21.15.181:0/2832609000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 msgr2=0x7f15a4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 --2- 172.21.15.181:0/2832609000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4100a20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1594009910 tx=0x7f1594009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 -- 172.21.15.181:0/2832609000 shutdown_connections 2024-05-10T04:33:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 --2- 172.21.15.181:0/2832609000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4100a20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 -- 172.21.15.181:0/2832609000 >> 172.21.15.181:0/2832609000 conn(0x7f15a4075f10 msgr2=0x7f15a4076350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 -- 172.21.15.181:0/2832609000 shutdown_connections 2024-05-10T04:33:36.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 -- 172.21.15.181:0/2832609000 wait complete. 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.397+0000 7f15ab61d700 1 Processor -- start 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15ab61d700 1 -- start start 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15ab61d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15ab61d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41975b0 con 0x7f15a4100610 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15a93b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15a93b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.181:60520/0 (socket says 172.21.15.181:60520) 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15a93b9700 1 -- 172.21.15.181:0/1735840293 learned_addr learned my addr 172.21.15.181:0/1735840293 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15a93b9700 1 -- 172.21.15.181:0/1735840293 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940095f0 con 0x7f15a4100610 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.398+0000 7f15a93b9700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4197070 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f1594004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.399+0000 7f159a7fc700 1 -- 172.21.15.181:0/1735840293 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f159401d070 con 0x7f15a4100610 2024-05-10T04:33:36.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.399+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a4197810 con 0x7f15a4100610 2024-05-10T04:33:36.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.399+0000 7f159a7fc700 1 -- 172.21.15.181:0/1735840293 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f159400bbb0 con 0x7f15a4100610 2024-05-10T04:33:36.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.399+0000 7f159a7fc700 1 -- 172.21.15.181:0/1735840293 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1594003810 con 0x7f15a4100610 2024-05-10T04:33:36.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.399+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a4197c90 con 0x7f15a4100610 2024-05-10T04:33:36.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.399+0000 7f159a7fc700 1 -- 172.21.15.181:0/1735840293 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f1594003a10 con 0x7f15a4100610 2024-05-10T04:33:36.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.400+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a4101ac0 con 0x7f15a4100610 2024-05-10T04:33:36.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.400+0000 7f159a7fc700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f15900387a0 0x7f159003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:36.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.400+0000 7f159a7fc700 1 -- 172.21.15.181:0/1735840293 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f159404c520 con 0x7f15a4100610 2024-05-10T04:33:36.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.400+0000 7f15a8bb8700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f15900387a0 0x7f159003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:36.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.400+0000 7f15a8bb8700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f15900387a0 0x7f159003acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15a0006fd0 tx=0x7f15a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:36.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.405+0000 7f159a7fc700 1 -- 172.21.15.181:0/1735840293 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1594029770 con 0x7f15a4100610 2024-05-10T04:33:36.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.605+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f15a410a7e0 con 0x7f15900387a0 2024-05-10T04:33:36.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.612+0000 7f159a7fc700 1 -- 172.21.15.181:0/1735840293 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f15a410a7e0 con 0x7f15900387a0 2024-05-10T04:33:36.613 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mon update... 2024-05-10T04:33:36.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f15900387a0 msgr2=0x7f159003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f15900387a0 0x7f159003acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15a0006fd0 tx=0x7f15a0006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 msgr2=0x7f15a4197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4197070 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1594006010 tx=0x7f1594004ff0 comp rx=0 tx=0).stop 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 shutdown_connections 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f15900387a0 0x7f159003acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 --2- 172.21.15.181:0/1735840293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15a4100610 0x7f15a4197070 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 >> 172.21.15.181:0/1735840293 conn(0x7f15a4075f10 msgr2=0x7f15a4068550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 shutdown_connections 2024-05-10T04:33:36.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:36.614+0000 7f15ab61d700 1 -- 172.21.15.181:0/1735840293 wait complete. 2024-05-10T04:33:37.001 DEBUG:teuthology.orchestra.run.smithi161:mon.b> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.b.service 2024-05-10T04:33:37.004 DEBUG:teuthology.orchestra.run.smithi181:mon.c> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.c.service 2024-05-10T04:33:37.006 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T04:33:37.006 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T04:33:37.036 INFO:journalctl@ceph.mon.c.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:33:37.040 INFO:journalctl@ceph.mon.b.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:33:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: Updating smithi181:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T04:33:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:37.418 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T04:33:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:37 smithi150 ceph-mon[42404]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:37 smithi150 ceph-mon[42404]: Saving service mon spec with placement smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T04:33:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:37 smithi150 ceph-mon[42404]: Deploying daemon mon.c on smithi181 2024-05-10T04:33:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:37 smithi150 ceph-mon[42404]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:38.918 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 -- 172.21.15.181:0/2208908202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 msgr2=0x7efd2c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:38.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 --2- 172.21.15.181:0/2208908202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c100a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efd28009910 tx=0x7efd28009c20 comp rx=0 tx=0).stop 2024-05-10T04:33:38.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 -- 172.21.15.181:0/2208908202 shutdown_connections 2024-05-10T04:33:38.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 --2- 172.21.15.181:0/2208908202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c100a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:38.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 -- 172.21.15.181:0/2208908202 >> 172.21.15.181:0/2208908202 conn(0x7efd2c0fbed0 msgr2=0x7efd2c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:38.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 -- 172.21.15.181:0/2208908202 shutdown_connections 2024-05-10T04:33:38.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 -- 172.21.15.181:0/2208908202 wait complete. 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 Processor -- start 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.915+0000 7efd33fe2700 1 -- start start 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.916+0000 7efd33fe2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.916+0000 7efd33fe2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c1964c0 con 0x7efd2c100640 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.916+0000 7efd31d7e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.916+0000 7efd31d7e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.181:60536/0 (socket says 172.21.15.181:60536) 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.916+0000 7efd31d7e700 1 -- 172.21.15.181:0/199688130 learned_addr learned my addr 172.21.15.181:0/199688130 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:33:38.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.916+0000 7efd31d7e700 1 -- 172.21.15.181:0/199688130 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280095f0 con 0x7efd2c100640 2024-05-10T04:33:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.917+0000 7efd31d7e700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c195f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd28004f40 tx=0x7efd28005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.917+0000 7efd22ffd700 1 -- 172.21.15.181:0/199688130 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd2801f070 con 0x7efd2c100640 2024-05-10T04:33:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.917+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c1966c0 con 0x7efd2c100640 2024-05-10T04:33:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.917+0000 7efd22ffd700 1 -- 172.21.15.181:0/199688130 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd2800bbb0 con 0x7efd2c100640 2024-05-10T04:33:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.917+0000 7efd22ffd700 1 -- 172.21.15.181:0/199688130 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd28003810 con 0x7efd2c100640 2024-05-10T04:33:38.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.917+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c196b40 con 0x7efd2c100640 2024-05-10T04:33:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.918+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c101af0 con 0x7efd2c100640 2024-05-10T04:33:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.918+0000 7efd22ffd700 1 -- 172.21.15.181:0/199688130 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7efd28003a90 con 0x7efd2c100640 2024-05-10T04:33:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.918+0000 7efd22ffd700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efd180387a0 0x7efd1803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.918+0000 7efd22ffd700 1 -- 172.21.15.181:0/199688130 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd2804e7a0 con 0x7efd2c100640 2024-05-10T04:33:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.918+0000 7efd3157d700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efd180387a0 0x7efd1803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:38.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.919+0000 7efd3157d700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efd180387a0 0x7efd1803acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006fd0 tx=0x7efd1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:38.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:38.924+0000 7efd22ffd700 1 -- 172.21.15.181:0/199688130 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efd2802b360 con 0x7efd2c100640 2024-05-10T04:33:39.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.243+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efd2c10a750 con 0x7efd2c100640 2024-05-10T04:33:39.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.244+0000 7efd22ffd700 1 -- 172.21.15.181:0/199688130 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7efd28028020 con 0x7efd2c100640 2024-05-10T04:33:39.246 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:33:39.246 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","modified":"2024-05-10T04:32:22.444014Z","created":"2024-05-10T04:32:22.444014Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efd180387a0 msgr2=0x7efd1803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efd180387a0 0x7efd1803acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd1c006fd0 tx=0x7efd1c006e40 comp rx=0 tx=0).stop 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 msgr2=0x7efd2c195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c195f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd28004f40 tx=0x7efd28005020 comp rx=0 tx=0).stop 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 shutdown_connections 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efd180387a0 0x7efd1803acb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 --2- 172.21.15.181:0/199688130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd2c100640 0x7efd2c195f80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 >> 172.21.15.181:0/199688130 conn(0x7efd2c0fbed0 msgr2=0x7efd2c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 shutdown_connections 2024-05-10T04:33:39.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:39.246+0000 7efd33fe2700 1 -- 172.21.15.181:0/199688130 wait complete. 2024-05-10T04:33:39.249 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2024-05-10T04:33:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:39 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/199688130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T04:33:40.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 systemd[1]: Started Ceph mon.c for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:33:40.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: set uid:gid to 167:167 (ceph:ceph) 2024-05-10T04:33:40.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable), process ceph-mon, pid 7 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: pidfile_write: ignore empty --pid-file 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: load: jerasure load: lrc 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: RocksDB version: 7.9.2 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Git sha 0 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: DB SUMMARY 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: DB Session ID: 4ZU96TZBCXE3KEVMMHCN 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: CURRENT file: CURRENT 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.error_if_exists: 0 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.create_if_missing: 0 2024-05-10T04:33:40.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.paranoid_checks: 1 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.env: 0x55d405270460 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.info_log: 0x55d405f9d760 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.statistics: (nil) 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.use_fsync: 0 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_log_file_size: 0 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T04:33:40.408 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.allow_fallocate: 1 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.use_direct_reads: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.db_log_dir: 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.wal_dir: 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T04:33:40.409 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.write_buffer_manager: 0x55d405ee10e0 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.unordered_write: 0 2024-05-10T04:33:40.410 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.row_cache: None 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.wal_filter: None 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.two_write_queues: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.wal_compression: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.atomic_flush: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T04:33:40.411 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.log_readahead_size: 0 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_background_jobs: 2 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_background_compactions: -1 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_subcompactions: 1 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T04:33:40.412 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_open_files: -1 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_background_flushes: -1 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Compression algorithms supported: 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kZSTD supported: 0 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kXpressCompression supported: 0 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T04:33:40.413 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kZlibCompression supported: 1 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kSnappyCompression supported: 1 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kLZ4Compression supported: 1 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: kBZip2Compression supported: 0 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.merge_operator: 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_filter: None 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T04:33:40.414 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d405f9d880) 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks: 1 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: pin_top_level_index_and_filter: 1 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_type: 0 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_index_type: 0 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_shortening: 1 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: checksum: 4 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: no_block_cache: 0 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache: 0x55d405f631f0 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_name: BinnedLRUCache 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_options: 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: capacity : 536870912 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_shard_bits : 4 2024-05-10T04:33:40.415 INFO:journalctl@ceph.mon.c.smithi181.stdout: strict_capacity_limit : 0 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: high_pri_pool_ratio: 0.000 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_cache_compressed: (nil) 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: persistent_cache: (nil) 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size: 4096 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_size_deviation: 10 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_restart_interval: 16 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: index_block_restart_interval: 1 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: metadata_block_size: 4096 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: partition_filters: 0 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: use_delta_encoding: 1 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: filter_policy: bloomfilter 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: whole_key_filtering: 1 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: verify_compression: 0 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: read_amp_bytes_per_bit: 0 2024-05-10T04:33:40.416 INFO:journalctl@ceph.mon.c.smithi181.stdout: format_version: 5 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_index_compression: 1 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout: block_align: 0 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout: max_auto_readahead_size: 262144 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout: prepopulate_block_cache: 0 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout: initial_auto_readahead_size: 8192 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression: NoCompression 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.num_levels: 7 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T04:33:40.417 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T04:33:40.418 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T04:33:40.419 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T04:33:40.420 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.table_properties_collectors: 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.inplace_update_support: 0 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.bloom_locality: 0 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.max_successive_merges: 0 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T04:33:40.421 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.ttl: 2592000 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.enable_blob_files: false 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.min_blob_size: 0 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T04:33:40.422 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 604dcaf0-5f00-4f25-959c-4f6a15ff5901 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315619938144, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315619938590, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715315619, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "604dcaf0-5f00-4f25-959c-4f6a15ff5901", "db_session_id": "4ZU96TZBCXE3KEVMMHCN", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315619938660, "job": 1, "event": "recovery_finished"} 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d405ef5c00 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: DB pointer 0x55d40ace4000 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T04:33:40.423 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** DB Stats ** 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:40.424 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** Compaction Stats [default] ** 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T04:33:40.425 INFO:journalctl@ceph.mon.c.smithi181.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache BinnedLRUCache@0x55d405f631f0#7 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: using public_addr v2:172.21.15.181:0/0 -> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: starting mon.c rank -1 at public addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] at bind addrs [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(???) e0 preinit fsid 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).mds e1 new map 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: e1 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T04:33:40.426 INFO:journalctl@ceph.mon.c.smithi181.stdout: legacy client fscid: -1 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout: 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout: No filesystems configured 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2551684131' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:40.427 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: Updating smithi181:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: Updating smithi181:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.428 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: Saving service mon spec with placement smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: Deploying daemon mon.c on smithi181 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/199688130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T04:33:40.429 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:39 smithi181 ceph-mon[47314]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T04:33:40.653 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T04:33:40.653 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T04:33:40.880 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.104+0000 7f01493c4700 1 -- 172.21.15.181:0/1917811389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144102750 msgr2=0x7f0130005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.104+0000 7f01493c4700 1 --2- 172.21.15.181:0/1917811389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144102750 0x7f0130005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f012c005fa0 tx=0x7f012c00f4a0 comp rx=0 tx=0).stop 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.104+0000 7f01493c4700 1 -- 172.21.15.181:0/1917811389 shutdown_connections 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.104+0000 7f01493c4700 1 --2- 172.21.15.181:0/1917811389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144102750 0x7f0130005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.104+0000 7f01493c4700 1 -- 172.21.15.181:0/1917811389 >> 172.21.15.181:0/1917811389 conn(0x7f0144076030 msgr2=0x7f0144076470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.104+0000 7f01493c4700 1 -- 172.21.15.181:0/1917811389 shutdown_connections 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.104+0000 7f01493c4700 1 -- 172.21.15.181:0/1917811389 wait complete. 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.106+0000 7f01493c4700 1 Processor -- start 2024-05-10T04:33:45.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.106+0000 7f01493c4700 1 -- start start 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.106+0000 7f01493c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 0x7f0144100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.106+0000 7f01493c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144100980 0x7f0144100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.106+0000 7f01493c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0144102310 con 0x7f0144100980 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.106+0000 7f01493c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441a32e0 con 0x7f0144102750 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f0142ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 0x7f0144100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f0142ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 0x7f0144100440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33586/0 (socket says 172.21.15.181:33586) 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f0142ffd700 1 -- 172.21.15.181:0/2063768885 learned_addr learned my addr 172.21.15.181:0/2063768885 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f01427fc700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144100980 0x7f0144100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:45.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f0142ffd700 1 -- 172.21.15.181:0/2063768885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 msgr2=0x7f0144100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f0142ffd700 1 -- 172.21.15.181:0/2063768885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 msgr2=0x7f0144100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f0142ffd700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 0x7f0144100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f0142ffd700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 0x7f0144100440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f01427fc700 1 -- 172.21.15.181:0/2063768885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 msgr2=0x7f0144100440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f01427fc700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 0x7f0144100440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f01427fc700 1 -- 172.21.15.181:0/2063768885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0139a0 con 0x7f0144100980 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.107+0000 7f01427fc700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144100980 0x7f0144100d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f013400b4d0 tx=0x7f013400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.108+0000 7f013bfff700 1 -- 172.21.15.181:0/2063768885 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0134011630 con 0x7f0144100980 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.108+0000 7f013bfff700 1 -- 172.21.15.181:0/2063768885 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0134011c50 con 0x7f0144100980 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.108+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01441a3480 con 0x7f0144100980 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.108+0000 7f013bfff700 1 -- 172.21.15.181:0/2063768885 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0134010d90 con 0x7f0144100980 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.108+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01441a39a0 con 0x7f0144100980 2024-05-10T04:33:45.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.109+0000 7f013bfff700 1 -- 172.21.15.181:0/2063768885 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f013401a400 con 0x7f0144100980 2024-05-10T04:33:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.109+0000 7f013bfff700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f01240385e0 0x7f012403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.109+0000 7f013bfff700 1 -- 172.21.15.181:0/2063768885 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f013404bca0 con 0x7f0144100980 2024-05-10T04:33:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.109+0000 7f0142ffd700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f01240385e0 0x7f012403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.109+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0144190cd0 con 0x7f0144100980 2024-05-10T04:33:45.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.109+0000 7f0142ffd700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f01240385e0 0x7f012403aaf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f012c004f10 tx=0x7f012c004e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:45.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.115+0000 7f013bfff700 1 -- 172.21.15.181:0/2063768885 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f01340178b0 con 0x7f0144100980 2024-05-10T04:33:45.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.391+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0144069240 con 0x7f0144100980 2024-05-10T04:33:45.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.392+0000 7f013bfff700 1 -- 172.21.15.181:0/2063768885 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f01340172c0 con 0x7f0144100980 2024-05-10T04:33:45.393 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:33:45.393 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","modified":"2024-05-10T04:33:39.969604Z","created":"2024-05-10T04:32:22.444014Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f01240385e0 msgr2=0x7f012403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f01240385e0 0x7f012403aaf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f012c004f10 tx=0x7f012c004e80 comp rx=0 tx=0).stop 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144100980 msgr2=0x7f0144100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144100980 0x7f0144100d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f013400b4d0 tx=0x7f013400b890 comp rx=0 tx=0).stop 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 shutdown_connections 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f01240385e0 0x7f012403aaf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0144102750 0x7f0144100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 --2- 172.21.15.181:0/2063768885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0144100980 0x7f0144100d90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 >> 172.21.15.181:0/2063768885 conn(0x7f0144076030 msgr2=0x7f01440ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 shutdown_connections 2024-05-10T04:33:45.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:45.394+0000 7f01493c4700 1 -- 172.21.15.181:0/2063768885 wait complete. 2024-05-10T04:33:45.396 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2024-05-10T04:33:45.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: mon.a calling monitor election 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: mon.c calling monitor election 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: monmap e2: 2 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:45.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: fsmap 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: mgrmap e13: x(active, since 33s) 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: overall HEALTH_OK 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:45.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:45.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:33:45.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: mon.a calling monitor election 2024-05-10T04:33:45.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: mon.c calling monitor election 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: monmap e2: 2 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: fsmap 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:45.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: mgrmap e13: x(active, since 33s) 2024-05-10T04:33:45.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: overall HEALTH_OK 2024-05-10T04:33:45.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:45.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:45.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:45.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:45.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:46.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:46 smithi181 ceph-mon[47314]: Deploying daemon mon.b on smithi161 2024-05-10T04:33:46.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:46 smithi181 ceph-mon[47314]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:46.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:46 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/2063768885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T04:33:46.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:46 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:46.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:46 smithi150 ceph-mon[42404]: Deploying daemon mon.b on smithi161 2024-05-10T04:33:46.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:46 smithi150 ceph-mon[42404]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:46.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:46 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/2063768885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T04:33:46.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:46 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:46.779 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-10T04:33:46.780 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-10T04:33:47.004 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:33:47.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 systemd[1]: Started Ceph mon.b for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:33:47.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: load: jerasure load: lrc 2024-05-10T04:33:47.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: RocksDB version: 7.9.2 2024-05-10T04:33:47.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Git sha 0 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Compile date 2024-05-10 00:51:33 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: DB SUMMARY 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: DB Session ID: IDDGRBS0LV3TW2AM739P 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: CURRENT file: CURRENT 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: IDENTITY file: IDENTITY 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.error_if_exists: 0 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.create_if_missing: 0 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.paranoid_checks: 1 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-10T04:33:47.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.env: 0x560411d48460 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.fs: PosixFileSystem 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.info_log: 0x56041364d760 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_file_opening_threads: 16 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.statistics: (nil) 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.use_fsync: 0 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_log_file_size: 0 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.keep_log_file_num: 1000 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.recycle_log_file_num: 0 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.allow_fallocate: 1 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.allow_mmap_reads: 0 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.allow_mmap_writes: 0 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.use_direct_reads: 0 2024-05-10T04:33:47.838 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-10T04:33:47.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.create_missing_column_families: 0 2024-05-10T04:33:47.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.db_log_dir: 2024-05-10T04:33:47.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.wal_dir: 2024-05-10T04:33:47.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-10T04:33:47.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-10T04:33:47.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-10T04:33:47.840 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-10T04:33:47.840 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.advise_random_on_open: 1 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.db_write_buffer_size: 0 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.write_buffer_manager: 0x5604135910e0 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.rate_limiter: (nil) 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.wal_recovery_mode: 2 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.enable_thread_tracking: 0 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.enable_pipelined_write: 0 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.unordered_write: 0 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-10T04:33:47.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.row_cache: None 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.wal_filter: None 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.allow_ingest_behind: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.two_write_queues: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.manual_wal_flush: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.wal_compression: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.atomic_flush: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-10T04:33:47.842 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.log_readahead_size: 0 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.best_efforts_recovery: 0 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.allow_data_in_errors: 0 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.db_host_id: __hostname__ 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_background_jobs: 2 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_background_compactions: -1 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_subcompactions: 1 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_total_wal_size: 0 2024-05-10T04:33:47.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_open_files: -1 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bytes_per_sync: 0 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_readahead_size: 0 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_background_flushes: -1 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Compression algorithms supported: 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kZSTD supported: 0 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kXpressCompression supported: 0 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kLZ4HCCompression supported: 1 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kZlibCompression supported: 1 2024-05-10T04:33:47.844 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kSnappyCompression supported: 1 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kLZ4Compression supported: 1 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: kBZip2Compression supported: 0 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.merge_operator: 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_filter: None 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_filter_factory: None 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.sst_partitioner_factory: None 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56041364d880) 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks: 1 2024-05-10T04:33:47.845 INFO:journalctl@ceph.mon.b.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: pin_top_level_index_and_filter: 1 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_type: 0 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_index_type: 0 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_shortening: 1 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: checksum: 4 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: no_block_cache: 0 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache: 0x5604136131f0 2024-05-10T04:33:47.846 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_name: BinnedLRUCache 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_options: 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: capacity : 536870912 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_shard_bits : 4 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: strict_capacity_limit : 0 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: high_pri_pool_ratio: 0.000 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_cache_compressed: (nil) 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: persistent_cache: (nil) 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size: 4096 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_size_deviation: 10 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_restart_interval: 16 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: index_block_restart_interval: 1 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: metadata_block_size: 4096 2024-05-10T04:33:47.847 INFO:journalctl@ceph.mon.b.smithi161.stdout: partition_filters: 0 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: use_delta_encoding: 1 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: filter_policy: bloomfilter 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: whole_key_filtering: 1 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: verify_compression: 0 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: read_amp_bytes_per_bit: 0 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: format_version: 5 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_index_compression: 1 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: block_align: 0 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: max_auto_readahead_size: 262144 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: prepopulate_block_cache: 0 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: initial_auto_readahead_size: 8192 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.write_buffer_size: 33554432 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_write_buffer_number: 2 2024-05-10T04:33:47.848 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression: NoCompression 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression: Disabled 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.prefix_extractor: nullptr 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.num_levels: 7 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-10T04:33:47.849 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.level: 32767 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.strategy: 0 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.enabled: false 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.target_file_size_base: 67108864 2024-05-10T04:33:47.850 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.arena_block_size: 1048576 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-10T04:33:47.851 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.disable_auto_compactions: 0 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.table_properties_collectors: 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.inplace_update_support: 0 2024-05-10T04:33:47.852 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.bloom_locality: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.max_successive_merges: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.paranoid_file_checks: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.force_consistency_checks: 1 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.report_bg_io_stats: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.ttl: 2592000 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-10T04:33:47.853 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.enable_blob_files: false 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.min_blob_size: 0 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.blob_file_size: 268435456 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.blob_file_starting_level: 0 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-10T04:33:47.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a7e52641-bdb0-4021-972a-0af7950c8929 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315627581834, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315627582243, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715315627, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a7e52641-bdb0-4021-972a-0af7950c8929", "db_session_id": "IDDGRBS0LV3TW2AM739P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715315627582310, "job": 1, "event": "recovery_finished"} 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5604135a5c00 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: DB pointer 0x560418394000 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** DB Stats ** 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-10T04:33:47.855 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** Compaction Stats [default] ** 2024-05-10T04:33:47.856 INFO:journalctl@ceph.mon.b.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-10T04:33:47.857 INFO:journalctl@ceph.mon.b.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache BinnedLRUCache@0x5604136131f0#7 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: starting mon.b rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(???) e0 preinit fsid 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).mds e1 new map 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: e1 2024-05-10T04:33:47.858 INFO:journalctl@ceph.mon.b.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout: legacy client fscid: -1 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout: 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout: No filesystems configured 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2551684131' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-10T04:33:47.859 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: Updating smithi181:/etc/ceph/ceph.conf 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: Updating smithi181:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: Updating smithi181:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.860 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: Saving service mon spec with placement smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: Deploying daemon mon.c on smithi181 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/199688130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:33:47.861 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.a calling monitor election 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.c calling monitor election 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: monmap e2: 2 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: fsmap 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mgrmap e13: x(active, since 33s) 2024-05-10T04:33:47.862 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: overall HEALTH_OK 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: Deploying daemon mon.b on smithi161 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/2063768885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:47.863 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:47 smithi161 ceph-mon[46912]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: mon.c calling monitor election 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: mon.a calling monitor election 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: fsmap 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: mgrmap e13: x(active, since 41s) 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: mon.b (rank 2) addr [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] is down (out of quorum) 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.157 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: mon.c calling monitor election 2024-05-10T04:33:53.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: mon.a calling monitor election 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: fsmap 2024-05-10T04:33:53.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: mgrmap e13: x(active, since 41s) 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: mon.b (rank 2) addr [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] is down (out of quorum) 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:53.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.975+0000 7f73dffcd700 1 -- 172.21.15.181:0/2516460451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 msgr2=0x7f73c4007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:53.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.975+0000 7f73dffcd700 1 --2- 172.21.15.181:0/2516460451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73c4007820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005fd0 tx=0x7f73c8012450 comp rx=0 tx=0).stop 2024-05-10T04:33:53.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.975+0000 7f73dffcd700 1 -- 172.21.15.181:0/2516460451 shutdown_connections 2024-05-10T04:33:53.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.975+0000 7f73dffcd700 1 --2- 172.21.15.181:0/2516460451 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73c4007820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:53.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.975+0000 7f73dffcd700 1 -- 172.21.15.181:0/2516460451 >> 172.21.15.181:0/2516460451 conn(0x7f73d8076030 msgr2=0x7f73d8076470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:53.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.975+0000 7f73dffcd700 1 -- 172.21.15.181:0/2516460451 shutdown_connections 2024-05-10T04:33:53.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.975+0000 7f73dffcd700 1 -- 172.21.15.181:0/2516460451 wait complete. 2024-05-10T04:33:53.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.976+0000 7f73dffcd700 1 Processor -- start 2024-05-10T04:33:53.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.976+0000 7f73dffcd700 1 -- start start 2024-05-10T04:33:53.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dffcd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73d8194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dffcd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d81954f0 0x7f73d8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dffcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 0x7f73d819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dffcd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8195bf0 con 0x7f73d81065c0 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dffcd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d81a1b00 con 0x7f73d81954f0 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dffcd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d81a6050 con 0x7f73d8199d90 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73d8194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73de56a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 0x7f73d819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73d8194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.181:49288/0 (socket says 172.21.15.181:49288) 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 -- 172.21.15.181:0/768161306 learned_addr learned my addr 172.21.15.181:0/768161306 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73de56a700 1 -- 172.21.15.181:0/768161306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 msgr2=0x7f73d819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-10T04:33:53.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73de56a700 1 -- 172.21.15.181:0/768161306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 msgr2=0x7f73d819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73de56a700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 0x7f73d819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73de56a700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 0x7f73d819a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dd568700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d81954f0 0x7f73d8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 -- 172.21.15.181:0/768161306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 msgr2=0x7f73d819a1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 0x7f73d819a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 -- 172.21.15.181:0/768161306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d81954f0 msgr2=0x7f73d8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d81954f0 0x7f73d8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 -- 172.21.15.181:0/768161306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c800fa30 con 0x7f73d81065c0 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73dd568700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d81954f0 0x7f73d8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.977+0000 7f73ddd69700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73d8194fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005cf0 tx=0x7f73c8005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.978+0000 7f73ceffd700 1 -- 172.21.15.181:0/768161306 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c802c270 con 0x7f73d81065c0 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.978+0000 7f73ceffd700 1 -- 172.21.15.181:0/768161306 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c802c890 con 0x7f73d81065c0 2024-05-10T04:33:53.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.978+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d81a1c40 con 0x7f73d81065c0 2024-05-10T04:33:53.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.978+0000 7f73ceffd700 1 -- 172.21.15.181:0/768161306 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8014110 con 0x7f73d81065c0 2024-05-10T04:33:53.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.978+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d81a21b0 con 0x7f73d81065c0 2024-05-10T04:33:53.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.979+0000 7f73ceffd700 1 -- 172.21.15.181:0/768161306 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f73c8014270 con 0x7f73d81065c0 2024-05-10T04:33:53.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.979+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d8104cf0 con 0x7f73d81065c0 2024-05-10T04:33:53.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.979+0000 7f73ceffd700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73bc038c20 0x7f73bc03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:53.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.979+0000 7f73ceffd700 1 -- 172.21.15.181:0/768161306 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73c805ff80 con 0x7f73d81065c0 2024-05-10T04:33:53.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.979+0000 7f73dd568700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73bc038c20 0x7f73bc03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:53.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.980+0000 7f73dd568700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73bc038c20 0x7f73bc03b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f73d8105dd0 tx=0x7f73d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:53.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:53.984+0000 7f73ceffd700 1 -- 172.21.15.181:0/768161306 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f73c801d410 con 0x7f73d81065c0 2024-05-10T04:33:54.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:54.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:54.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:54.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:54.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.273+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73d8062870 con 0x7f73d81065c0 2024-05-10T04:33:54.274 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.274+0000 7f73ceffd700 1 -- 172.21.15.181:0/768161306 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f73c8042020 con 0x7f73d81065c0 2024-05-10T04:33:54.275 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-05-10T04:33:54.275 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","modified":"2024-05-10T04:33:47.614223Z","created":"2024-05-10T04:32:22.444014Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73bc038c20 msgr2=0x7f73bc03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73bc038c20 0x7f73bc03b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f73d8105dd0 tx=0x7f73d4008040 comp rx=0 tx=0).stop 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 msgr2=0x7f73d8194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73d8194fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005cf0 tx=0x7f73c8005320 comp rx=0 tx=0).stop 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 shutdown_connections 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73bc038c20 0x7f73bc03b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d8199d90 0x7f73d819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73d81065c0 0x7f73d8194fb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 --2- 172.21.15.181:0/768161306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73d81954f0 0x7f73d8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 >> 172.21.15.181:0/768161306 conn(0x7f73d8076030 msgr2=0x7f73d80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 shutdown_connections 2024-05-10T04:33:54.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:54.276+0000 7f73dffcd700 1 -- 172.21.15.181:0/768161306 wait complete. 2024-05-10T04:33:54.278 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 3 2024-05-10T04:33:54.653 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-10T04:33:54.654 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-10T04:33:55.076 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:33:55.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: mon.c calling monitor election 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: mon.a calling monitor election 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: fsmap 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: mgrmap e13: x(active, since 41s) 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: mon.b (rank 2) addr [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] is down (out of quorum) 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:55.088 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:55.560 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: mon.b calling monitor election 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: mon.b calling monitor election 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: mon.c calling monitor election 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: mon.a calling monitor election 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: fsmap 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: mgrmap e13: x(active, since 43s) 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: Cluster is now healthy 2024-05-10T04:33:55.561 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:55 smithi150 ceph-mon[42404]: overall HEALTH_OK 2024-05-10T04:33:55.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: mon.b calling monitor election 2024-05-10T04:33:55.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: mon.b calling monitor election 2024-05-10T04:33:55.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: mon.c calling monitor election 2024-05-10T04:33:55.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: mon.a calling monitor election 2024-05-10T04:33:55.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T04:33:55.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:55.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: fsmap 2024-05-10T04:33:55.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:55.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: mgrmap e13: x(active, since 43s) 2024-05-10T04:33:55.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-10T04:33:55.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: Cluster is now healthy 2024-05-10T04:33:55.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:55 smithi161 ceph-mon[46912]: overall HEALTH_OK 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: mon.b calling monitor election 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: mon.b calling monitor election 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: mon.c calling monitor election 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: mon.a calling monitor election 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: monmap e3: 3 mons at {a=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],b=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: fsmap 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: osdmap e4: 0 total, 0 up, 0 in 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: mgrmap e13: x(active, since 43s) 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: Cluster is now healthy 2024-05-10T04:33:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:55 smithi181 ceph-mon[47314]: overall HEALTH_OK 2024-05-10T04:33:56.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 -- 172.21.15.150:0/797089676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 msgr2=0x7f578810e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 --2- 172.21.15.150:0/797089676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f578810e7b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f578000b0a0 tx=0x7f578000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 -- 172.21.15.150:0/797089676 shutdown_connections 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 --2- 172.21.15.150:0/797089676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f578810e7b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 --2- 172.21.15.150:0/797089676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5788102ea0 0x7f57881032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 --2- 172.21.15.150:0/797089676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57881037f0 0x7f578810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 -- 172.21.15.150:0/797089676 >> 172.21.15.150:0/797089676 conn(0x7f5788076720 msgr2=0x7f5788076b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 -- 172.21.15.150:0/797089676 shutdown_connections 2024-05-10T04:33:56.004 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.001+0000 7f578c8b7700 1 -- 172.21.15.150:0/797089676 wait complete. 2024-05-10T04:33:56.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 Processor -- start 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 -- start start 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5788102ea0 0x7f578819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57881037f0 0x7f578819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f5788198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5788198a40 con 0x7f578810c2d0 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5788198bb0 con 0x7f5788102ea0 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578c8b7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5788198d70 con 0x7f57881037f0 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f5788198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5788102ea0 0x7f578819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f5788198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43232/0 (socket says 172.21.15.150:43232) 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f578659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5788102ea0 0x7f578819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:47448/0 (socket says 172.21.15.150:47448) 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 -- 172.21.15.150:0/2449540336 learned_addr learned my addr 172.21.15.150:0/2449540336 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5785d9b700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57881037f0 0x7f578819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:56.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 -- 172.21.15.150:0/2449540336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57881037f0 msgr2=0x7f578819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57881037f0 0x7f578819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 -- 172.21.15.150:0/2449540336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5788102ea0 msgr2=0x7f578819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5788102ea0 0x7f578819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.003+0000 7f5786d9d700 1 -- 172.21.15.150:0/2449540336 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780009d00 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.004+0000 7f5786d9d700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f5788198470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f578000b830 tx=0x7f57800077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.004+0000 7f57777fe700 1 -- 172.21.15.150:0/2449540336 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578000e040 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.004+0000 7f57777fe700 1 -- 172.21.15.150:0/2449540336 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5780004c60 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.004+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5788198ff0 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.004+0000 7f57777fe700 1 -- 172.21.15.150:0/2449540336 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780005280 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.004+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57881a54e0 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.005+0000 7f57777fe700 1 -- 172.21.15.150:0/2449540336 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f5780005420 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.005+0000 7f57777fe700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f57700389d0 0x7f577003aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.005+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5788107490 con 0x7f578810c2d0 2024-05-10T04:33:56.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.005+0000 7f57777fe700 1 -- 172.21.15.150:0/2449540336 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f578001b450 con 0x7f578810c2d0 2024-05-10T04:33:56.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.005+0000 7f578659c700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f57700389d0 0x7f577003aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:56.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.005+0000 7f578659c700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f57700389d0 0x7f577003aee0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f577c005950 tx=0x7f577c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:56.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.010+0000 7f57777fe700 1 -- 172.21.15.150:0/2449540336 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5780020020 con 0x7f578810c2d0 2024-05-10T04:33:56.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.200+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5788077340 con 0x7f578810c2d0 2024-05-10T04:33:56.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.200+0000 7f57777fe700 1 -- 172.21.15.150:0/2449540336 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f5780033090 con 0x7f578810c2d0 2024-05-10T04:33:56.202 INFO:teuthology.orchestra.run.smithi150.stdout:# minimal ceph.conf for 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:56.202 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-05-10T04:33:56.202 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:33:56.202 INFO:teuthology.orchestra.run.smithi150.stdout: mon_host = [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2024-05-10T04:33:56.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f57700389d0 msgr2=0x7f577003aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:56.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f57700389d0 0x7f577003aee0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f577c005950 tx=0x7f577c00a300 comp rx=0 tx=0).stop 2024-05-10T04:33:56.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 msgr2=0x7f5788198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:56.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f5788198470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f578000b830 tx=0x7f57800077f0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 shutdown_connections 2024-05-10T04:33:56.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f578810c2d0 0x7f5788198470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5788102ea0 0x7f578819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f57700389d0 0x7f577003aee0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 --2- 172.21.15.150:0/2449540336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57881037f0 0x7f578819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:56.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.202+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 >> 172.21.15.150:0/2449540336 conn(0x7f5788076720 msgr2=0x7f57880fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:56.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.203+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 shutdown_connections 2024-05-10T04:33:56.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:33:56.203+0000 7f578c8b7700 1 -- 172.21.15.150:0/2449540336 wait complete. 2024-05-10T04:33:56.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:56.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:56.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T04:33:56.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:56.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:56.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2449540336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:56.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2449540336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:56.494 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.569 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-10T04:33:56.569 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:33:56.570 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T04:33:56.600 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:33:56.600 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:56.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T04:33:56.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:56.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:33:56.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2449540336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:56.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:56.667 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:33:56.667 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T04:33:56.698 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:33:56.698 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:56.766 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:33:56.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-05-10T04:33:56.796 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:33:56.796 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-10T04:33:56.864 INFO:tasks.cephadm:Adding mgr.x on smithi150 2024-05-10T04:33:56.865 INFO:tasks.cephadm:Adding mgr.y on smithi161 2024-05-10T04:33:56.865 INFO:tasks.cephadm:Adding mgr.z on smithi181 2024-05-10T04:33:56.865 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi150=x;smithi161=y;smithi181=z' 2024-05-10T04:33:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:33:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: Reconfiguring mon.b (monmap changed)... 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: Reconfiguring daemon mon.b on smithi161 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: Reconfiguring mon.c (monmap changed)... 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: Reconfiguring daemon mon.c on smithi181 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:57.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:57.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:57.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: Reconfiguring mon.b (monmap changed)... 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: Reconfiguring daemon mon.b on smithi161 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: Reconfiguring mon.c (monmap changed)... 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: Reconfiguring daemon mon.c on smithi181 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:57.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: Reconfiguring mon.b (monmap changed)... 2024-05-10T04:33:57.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: Reconfiguring daemon mon.b on smithi161 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: Reconfiguring mon.c (monmap changed)... 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: Reconfiguring daemon mon.c on smithi181 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:57.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 -- 172.21.15.181:0/320772532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac101ce0 msgr2=0x7f3aac1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/320772532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac101ce0 0x7f3aac1020f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009990 tx=0x7f3aa8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 -- 172.21.15.181:0/320772532 shutdown_connections 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/320772532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3aac102680 0x7f3aac108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/320772532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3aac100bb0 0x7f3aac100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/320772532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac101ce0 0x7f3aac1020f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 -- 172.21.15.181:0/320772532 >> 172.21.15.181:0/320772532 conn(0x7f3aac0fc1d0 msgr2=0x7f3aac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 -- 172.21.15.181:0/320772532 shutdown_connections 2024-05-10T04:33:58.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.209+0000 7f3ab3b06700 1 -- 172.21.15.181:0/320772532 wait complete. 2024-05-10T04:33:58.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 Processor -- start 2024-05-10T04:33:58.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 -- start start 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac100bb0 0x7f3aac19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3aac101ce0 0x7f3aac19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3aac102680 0x7f3aac19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac19fe50 con 0x7f3aac102680 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac19ff90 con 0x7f3aac101ce0 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab3b06700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1a00d0 con 0x7f3aac100bb0 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac100bb0 0x7f3aac19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac100bb0 0x7f3aac19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57280/0 (socket says 172.21.15.181:57280) 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 -- 172.21.15.181:0/1272875218 learned_addr learned my addr 172.21.15.181:0/1272875218 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 -- 172.21.15.181:0/1272875218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3aac101ce0 msgr2=0x7f3aac19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3aac101ce0 0x7f3aac19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 -- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3aac102680 msgr2=0x7f3aac19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:58.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3aac102680 0x7f3aac19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 -- 172.21.15.181:0/1272875218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80095f0 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3ab18a2700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac100bb0 0x7f3aac19ae70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d950 tx=0x7f3a9c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3aa2ffd700 1 -- 172.21.15.181:0/1272875218 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c004220 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3aa2ffd700 1 -- 172.21.15.181:0/1272875218 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a9c00de30 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.210+0000 7f3aa2ffd700 1 -- 172.21.15.181:0/1272875218 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c003cb0 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.211+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac1a0330 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.211+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac1a0830 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.211+0000 7f3aa2ffd700 1 -- 172.21.15.181:0/1272875218 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7f3a9c010460 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.212+0000 7f3aa2ffd700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a980389d0 0x7f3a9803aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.212+0000 7f3aa2ffd700 1 -- 172.21.15.181:0/1272875218 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a9c04bd50 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.212+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac107dd0 con 0x7f3aac100bb0 2024-05-10T04:33:58.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.212+0000 7f3ab10a1700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a980389d0 0x7f3a9803aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:33:58.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.212+0000 7f3ab10a1700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a980389d0 0x7f3a9803aee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80098e0 tx=0x7f3aa8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:33:58.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.217+0000 7f3aa2ffd700 1 -- 172.21.15.181:0/1272875218 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3a9c01b400 con 0x7f3aac100bb0 2024-05-10T04:33:58.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.418+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f3aac10c700 con 0x7f3a980389d0 2024-05-10T04:33:58.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.425+0000 7f3aa2ffd700 1 -- 172.21.15.181:0/1272875218 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3aac10c700 con 0x7f3a980389d0 2024-05-10T04:33:58.426 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mgr update... 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a980389d0 msgr2=0x7f3a9803aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a980389d0 0x7f3a9803aee0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80098e0 tx=0x7f3aa8005fb0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac100bb0 msgr2=0x7f3aac19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac100bb0 0x7f3aac19ae70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d950 tx=0x7f3a9c00dd10 comp rx=0 tx=0).stop 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 shutdown_connections 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3aac102680 0x7f3aac19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a980389d0 0x7f3a9803aee0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aac100bb0 0x7f3aac19ae70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 --2- 172.21.15.181:0/1272875218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3aac101ce0 0x7f3aac19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:33:58.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 >> 172.21.15.181:0/1272875218 conn(0x7f3aac0fc1d0 msgr2=0x7f3aac104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:33:58.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 shutdown_connections 2024-05-10T04:33:58.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:33:58.427+0000 7f3ab3b06700 1 -- 172.21.15.181:0/1272875218 wait complete. 2024-05-10T04:33:58.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:58 smithi161 ceph-mon[46912]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:58.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:58.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:58 smithi181 ceph-mon[47314]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:58.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:58 smithi150 ceph-mon[42404]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:58.895 DEBUG:teuthology.orchestra.run.smithi161:mgr.y> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.y.service 2024-05-10T04:33:58.898 DEBUG:teuthology.orchestra.run.smithi181:mgr.z> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.z.service 2024-05-10T04:33:58.900 INFO:tasks.cephadm:Deploying OSDs... 2024-05-10T04:33:58.901 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:33:58.901 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T04:33:58.922 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T04:33:58.922 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-05-10T04:33:58.929 INFO:journalctl@ceph.mgr.z.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:33:58.931 INFO:journalctl@ceph.mgr.y.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:33:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 177075 Links: 1 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 04:33:20.130561839 +0000 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 04:26:23.401317217 +0000 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 04:26:23.401317217 +0000 2024-05-10T04:33:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T04:33:58.985 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T04:33:59.055 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T04:33:59.056 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T04:33:59.056 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000277461 s, 1.8 MB/s 2024-05-10T04:33:59.057 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T04:33:59.118 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-05-10T04:33:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 177121 Links: 1 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 04:33:20.176560956 +0000 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 04:26:23.747310641 +0000 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 04:26:23.747310641 +0000 2024-05-10T04:33:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T04:33:59.178 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T04:33:59.246 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T04:33:59.246 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T04:33:59.246 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000228437 s, 2.2 MB/s 2024-05-10T04:33:59.248 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T04:33:59.308 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 184466 Links: 1 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 04:33:20.219560130 +0000 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 04:26:24.103303875 +0000 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 04:26:24.103303875 +0000 2024-05-10T04:33:59.367 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T04:33:59.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T04:33:59.435 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T04:33:59.435 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T04:33:59.435 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000192828 s, 2.7 MB/s 2024-05-10T04:33:59.437 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T04:33:59.496 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-05-10T04:33:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T04:33:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:33:59.555 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 183563 Links: 1 2024-05-10T04:33:59.555 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:33:59.555 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:33:59.555 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 04:33:20.274559074 +0000 2024-05-10T04:33:59.555 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 04:26:24.445297375 +0000 2024-05-10T04:33:59.555 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 04:26:24.445297375 +0000 2024-05-10T04:33:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-05-10T04:33:59.555 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T04:33:59.621 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-10T04:33:59.621 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-10T04:33:59.621 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000247521 s, 2.1 MB/s 2024-05-10T04:33:59.622 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T04:33:59.681 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-05-10T04:33:59.682 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T04:33:59.704 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T04:33:59.704 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-05-10T04:33:59.759 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: Saving service mgr spec with placement smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: Deploying daemon mgr.z on smithi181 2024-05-10T04:33:59.760 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:33:59 smithi181 ceph-mon[47314]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 177917 Links: 1 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 04:33:47.178691838 +0000 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 04:26:43.204958504 +0000 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 04:26:43.204958504 +0000 2024-05-10T04:33:59.765 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T04:33:59.765 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: Saving service mgr spec with placement smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T04:33:59.832 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:33:59.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:59.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: Deploying daemon mgr.z on smithi181 2024-05-10T04:33:59.833 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:33:59 smithi161 ceph-mon[46912]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:59.834 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T04:33:59.834 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T04:33:59.834 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000227507 s, 2.3 MB/s 2024-05-10T04:33:59.836 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T04:33:59.897 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-05-10T04:33:59.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi150=x;smithi161=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:33:59.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: Saving service mgr spec with placement smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T04:33:59.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:59.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:33:59.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: Deploying daemon mgr.z on smithi181 2024-05-10T04:33:59.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:33:59 smithi150 ceph-mon[42404]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:33:59.956 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T04:33:59.956 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:33:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 181885 Links: 1 2024-05-10T04:33:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:33:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:33:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 04:33:47.178691838 +0000 2024-05-10T04:33:59.957 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 04:26:43.552951162 +0000 2024-05-10T04:33:59.957 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 04:26:43.552951162 +0000 2024-05-10T04:33:59.957 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T04:33:59.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T04:34:00.027 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T04:34:00.027 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T04:34:00.027 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000281181 s, 1.8 MB/s 2024-05-10T04:34:00.028 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T04:34:00.086 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-05-10T04:34:00.096 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:00 smithi181 systemd[1]: Starting Ceph mgr.z for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:34:00.144 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T04:34:00.144 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:34:00.144 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 183450 Links: 1 2024-05-10T04:34:00.144 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:34:00.145 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:34:00.145 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 04:33:47.179691817 +0000 2024-05-10T04:34:00.145 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 04:26:43.891944010 +0000 2024-05-10T04:34:00.145 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 04:26:43.891944010 +0000 2024-05-10T04:34:00.145 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T04:34:00.145 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T04:34:00.210 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T04:34:00.211 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T04:34:00.211 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000187908 s, 2.7 MB/s 2024-05-10T04:34:00.212 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T04:34:00.269 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-05-10T04:34:00.326 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 181992 Links: 1 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-05-10 04:33:47.179691817 +0000 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-05-10 04:26:44.246936520 +0000 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-05-10 04:26:44.246936520 +0000 2024-05-10T04:34:00.327 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-05-10T04:34:00.327 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T04:34:00.393 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-05-10T04:34:00.393 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-05-10T04:34:00.393 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000236546 s, 2.2 MB/s 2024-05-10T04:34:00.394 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T04:34:00.452 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-05-10T04:34:00.452 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-05-10T04:34:00.472 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-10T04:34:00.472 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-05-10T04:34:00.530 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-10T04:34:00.530 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:34:00.531 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 175977 Links: 1 2024-05-10T04:34:00.531 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:34:00.531 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:34:00.531 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 04:34:00.046175280 +0000 2024-05-10T04:34:00.531 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 04:26:30.066863910 +0000 2024-05-10T04:34:00.531 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 04:26:30.066863910 +0000 2024-05-10T04:34:00.531 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T04:34:00.531 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-10T04:34:00.603 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T04:34:00.603 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T04:34:00.603 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000223841 s, 2.3 MB/s 2024-05-10T04:34:00.604 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-10T04:34:00.663 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-05-10T04:34:00.719 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 178956 Links: 1 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 04:34:00.046175280 +0000 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 04:26:30.402857479 +0000 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 04:26:30.402857479 +0000 2024-05-10T04:34:00.720 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T04:34:00.720 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-10T04:34:00.782 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:00 smithi181 ceph-mgr[49204]: -- 172.21.15.181:0/176916759 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5634d956e1a0 con 0x5634d95cb600 2024-05-10T04:34:00.783 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:00 smithi181 bash[49166]: 4acab310f4d26d7ce1f934f83e9ab2f2c12ade710b9c0b177cebaa5ff9e9dfa7 2024-05-10T04:34:00.783 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:00 smithi181 systemd[1]: Started Ceph mgr.z for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:34:00.785 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T04:34:00.785 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T04:34:00.785 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000213049 s, 2.4 MB/s 2024-05-10T04:34:00.787 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-10T04:34:00.845 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 176110 Links: 1 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 04:34:00.047175261 +0000 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 04:26:30.738851048 +0000 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 04:26:30.738851048 +0000 2024-05-10T04:34:00.903 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T04:34:00.903 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-10T04:34:00.967 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:00.905+0000 7f9cfa938200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T04:34:00.970 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T04:34:00.970 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T04:34:00.970 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000173059 s, 3.0 MB/s 2024-05-10T04:34:00.971 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-10T04:34:01.029 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-05-10T04:34:01.087 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-10T04:34:01.087 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-10T04:34:01.088 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 184476 Links: 1 2024-05-10T04:34:01.088 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:34:01.088 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-05-10T04:34:01.088 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-05-10 04:34:00.047175261 +0000 2024-05-10T04:34:01.088 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-05-10 04:26:31.122843698 +0000 2024-05-10T04:34:01.088 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-05-10 04:26:31.122843698 +0000 2024-05-10T04:34:01.088 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-05-10T04:34:01.088 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-10T04:34:01.094 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:01.002+0000 7f9cfa938200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T04:34:01.094 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:01.093+0000 7f9cfa938200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T04:34:01.119 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-05-10T04:34:01.119 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-05-10T04:34:01.119 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000273562 s, 1.9 MB/s 2024-05-10T04:34:01.120 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-10T04:34:01.181 INFO:tasks.cephadm:Deploying osd.0 on smithi150 with /dev/vg_nvme/lv_4... 2024-05-10T04:34:01.181 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T04:34:01.404 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:01.185+0000 7f9cfa938200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T04:34:01.411 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:34:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.660 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:34:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T04:34:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:34:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: Deploying daemon mgr.y on smithi161 2024-05-10T04:34:01.661 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:01 smithi161 ceph-mon[46912]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:01.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:34:01.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T04:34:01.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:34:01.773 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:01.774 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: Deploying daemon mgr.y on smithi161 2024-05-10T04:34:01.774 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:01 smithi150 ceph-mon[42404]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:01.827+0000 7f9cfa938200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: Deploying daemon mgr.y on smithi161 2024-05-10T04:34:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:01 smithi181 ceph-mon[47314]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:02.085 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:01 smithi161 systemd[1]: Starting Ceph mgr.y for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:34:02.404 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:02.078+0000 7f9cfa938200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T04:34:02.405 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:02.164+0000 7f9cfa938200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T04:34:02.405 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:02.272+0000 7f9cfa938200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T04:34:02.585 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:02 smithi161 bash[48387]: ba3a10cc8f2bb3a3d785d926555930242b0b40b4e775fe5b6de31635bbf42713 2024-05-10T04:34:02.585 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:02 smithi161 ceph-mgr[48425]: -- 172.21.15.161:0/968619361 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55df05cac1a0 con 0x55df05d09600 2024-05-10T04:34:02.585 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:02 smithi161 systemd[1]: Started Ceph mgr.y for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:34:02.629 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T04:34:02.629 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T04:34:02.629 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T04:34:02.629 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T04:34:02.629 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127516 s, 822 MB/s 2024-05-10T04:34:02.629 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T04:34:02.787 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:02 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:02.682+0000 7f09f6f48200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-10T04:34:02.905 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:02.432+0000 7f9cfa938200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T04:34:02.905 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:02.568+0000 7f9cfa938200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T04:34:02.905 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:02.657+0000 7f9cfa938200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T04:34:02.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-05-10T04:34:02.970 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:02 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:02.786+0000 7f09f6f48200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-10T04:34:02.970 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:02 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:02.881+0000 7f09f6f48200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-10T04:34:02.971 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:02 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:02.970+0000 7f09f6f48200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-10T04:34:03.187 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:34:03.341 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:03.066+0000 7f9cfa938200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T04:34:03.617 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:03.618 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:03.639 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.654 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:03.484+0000 7f9cfa938200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:03.892 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:03 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:03.638+0000 7f09f6f48200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:34:04.154 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:03.680+0000 7f9cfa938200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T04:34:04.155 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:03.773+0000 7f9cfa938200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T04:34:04.244 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:03 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:03.891+0000 7f09f6f48200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-10T04:34:04.244 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:03 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:03.977+0000 7f09f6f48200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-10T04:34:04.244 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:04 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:04.086+0000 7f09f6f48200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-10T04:34:04.244 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:04 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:04.243+0000 7f09f6f48200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-10T04:34:04.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 -- 172.21.15.150:0/3281598594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774102680 msgr2=0x7fc774108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:04.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/3281598594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774102680 0x7fc774108f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00b0a0 tx=0x7fc76c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:34:04.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 -- 172.21.15.150:0/3281598594 shutdown_connections 2024-05-10T04:34:04.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/3281598594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774102680 0x7fc774108f30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:04.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/3281598594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774100bb0 0x7fc774100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:04.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/3281598594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc774101ce0 0x7fc7741020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:04.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 -- 172.21.15.150:0/3281598594 >> 172.21.15.150:0/3281598594 conn(0x7fc7740fc1d0 msgr2=0x7fc7740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:34:04.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 -- 172.21.15.150:0/3281598594 shutdown_connections 2024-05-10T04:34:04.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.443+0000 7fc77a6d4700 1 -- 172.21.15.150:0/3281598594 wait complete. 2024-05-10T04:34:04.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 Processor -- start 2024-05-10T04:34:04.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 -- start start 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774100bb0 0x7fc7741982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774101ce0 0x7fc774198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc774102680 0x7fc7741995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774199c20 con 0x7fc774100bb0 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774199d60 con 0x7fc774102680 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc77a6d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774199f20 con 0x7fc774101ce0 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.444+0000 7fc773fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774100bb0 0x7fc7741982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774100bb0 0x7fc7741982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44430/0 (socket says 172.21.15.150:44430) 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 -- 172.21.15.150:0/811930961 learned_addr learned my addr 172.21.15.150:0/811930961 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc7737fe700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774101ce0 0x7fc774198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc778c71700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc774102680 0x7fc7741995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:04.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 -- 172.21.15.150:0/811930961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774101ce0 msgr2=0x7fc774198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774101ce0 0x7fc774198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 -- 172.21.15.150:0/811930961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc774102680 msgr2=0x7fc7741995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc774102680 0x7fc7741995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 -- 172.21.15.150:0/811930961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c009d00 con 0x7fc774100bb0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc7737fe700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774101ce0 0x7fc774198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc773fff700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774100bb0 0x7fc7741982d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc76400e9f0 tx=0x7fc76400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764004230 con 0x7fc774100bb0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc764004390 con 0x7fc774100bb0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77419a200 con 0x7fc774100bb0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764003c40 con 0x7fc774100bb0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.445+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7741a09e0 con 0x7fc774100bb0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.446+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc774107dd0 con 0x7fc774100bb0 2024-05-10T04:34:04.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.446+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45598+0+0 (secure 0 0 0) 0x7fc764021440 con 0x7fc774100bb0 2024-05-10T04:34:04.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.447+0000 7fc7717fa700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc7600389d0 0x7fc76003aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:04.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.447+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc764014070 con 0x7fc774100bb0 2024-05-10T04:34:04.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.447+0000 7fc7737fe700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc7600389d0 0x7fc76003aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:04.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.447+0000 7fc7737fe700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc7600389d0 0x7fc76003aee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7741992a0 tx=0x7fc75c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:04.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.452+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc764016960 con 0x7fc774100bb0 2024-05-10T04:34:04.585 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:04 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:04.370+0000 7f09f6f48200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-10T04:34:04.585 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:04 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:04.456+0000 7f09f6f48200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-10T04:34:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:04 smithi161 ceph-mon[46912]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:34:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:34:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:04.641 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:04.541+0000 7f9cfa938200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T04:34:04.642 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:04 smithi181 ceph-mon[47314]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:04.642 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:34:04.642 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:34:04.642 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:04 smithi150 ceph-mon[42404]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-10T04:34:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-10T04:34:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:04.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:04.672+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc77410d7b0 con 0x7fc7600389d0 2024-05-10T04:34:04.904 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:04.640+0000 7f9cfa938200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T04:34:04.904 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:04.790+0000 7f9cfa938200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T04:34:05.085 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:04 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:04.830+0000 7f09f6f48200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-10T04:34:05.504 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:05 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:05.216+0000 7f09f6f48200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-10T04:34:05.504 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:05 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:05.411+0000 7f09f6f48200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-10T04:34:05.504 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:05 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:05.503+0000 7f09f6f48200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-10T04:34:05.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T04:34:05.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: Reconfiguring daemon mgr.x on smithi150 2024-05-10T04:34:05.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.504 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:05.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:05.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:05.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:05.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:05.505 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:05.605 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:05.422+0000 7f9cfa938200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T04:34:05.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: Reconfiguring daemon mgr.x on smithi150 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:05.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:05.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: Reconfiguring mgr.x (unknown last config time)... 2024-05-10T04:34:05.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: Reconfiguring daemon mgr.x on smithi150 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:05.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:05.904 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:05.604+0000 7f9cfa938200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T04:34:05.905 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:05.801+0000 7f9cfa938200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T04:34:06.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:06 smithi161 ceph-mon[46912]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:06.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:06 smithi161 ceph-mon[46912]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:06.586 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:06 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:06.265+0000 7f09f6f48200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-10T04:34:06.586 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:06 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:06.363+0000 7f09f6f48200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-10T04:34:06.586 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:06 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:06.512+0000 7f09f6f48200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-10T04:34:06.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:06 smithi181 ceph-mon[47314]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:06.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:06 smithi181 ceph-mon[47314]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:06 smithi150 ceph-mon[42404]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:06 smithi150 ceph-mon[42404]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:07.023 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:06 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:06.735+0000 7f9cfa938200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T04:34:07.023 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:07 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:07.022+0000 7f9cfa938200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:34:07.303 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:07 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:07.112+0000 7f9cfa938200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T04:34:07.303 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:34:07 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z[49181]: 2024-05-10T04:34:07.302+0000 7f9cfa938200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T04:34:07.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:07.504+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 14) v1 ==== 90708+0+0 (secure 0 0 0) 0x7fc764010430 con 0x7fc774100bb0 2024-05-10T04:34:07.513 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:07 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:07.138+0000 7f09f6f48200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-10T04:34:07.513 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:07 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:07.318+0000 7f09f6f48200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-10T04:34:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1221092526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc48a860-ef1e-4d94-be5a-6c8758a2b7e3"}]: dispatch 2024-05-10T04:34:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1221092526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc48a860-ef1e-4d94-be5a-6c8758a2b7e3"}]': finished 2024-05-10T04:34:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T04:34:07.513 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1269389670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: Standby manager daemon z started 2024-05-10T04:34:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T04:34:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T04:34:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T04:34:07.514 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:07 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1221092526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc48a860-ef1e-4d94-be5a-6c8758a2b7e3"}]: dispatch 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1221092526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc48a860-ef1e-4d94-be5a-6c8758a2b7e3"}]': finished 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1269389670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: Standby manager daemon z started 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T04:34:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:07 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1221092526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc48a860-ef1e-4d94-be5a-6c8758a2b7e3"}]: dispatch 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1221092526' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc48a860-ef1e-4d94-be5a-6c8758a2b7e3"}]': finished 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: osdmap e5: 1 total, 0 up, 1 in 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1269389670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: Standby manager daemon z started 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T04:34:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-10T04:34:07.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:07 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.181:0/2378169649' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T04:34:07.835 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:07 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:07.512+0000 7f09f6f48200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-10T04:34:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:08 smithi181 ceph-mon[47314]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:08 smithi181 ceph-mon[47314]: mgrmap e14: x(active, since 56s), standbys: z 2024-05-10T04:34:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T04:34:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:08 smithi150 ceph-mon[42404]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:08 smithi150 ceph-mon[42404]: mgrmap e14: x(active, since 56s), standbys: z 2024-05-10T04:34:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T04:34:08.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:08 smithi161 ceph-mon[46912]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:08.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:08 smithi161 ceph-mon[46912]: mgrmap e14: x(active, since 56s), standbys: z 2024-05-10T04:34:08.723 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-10T04:34:08.723 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:08 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:08.438+0000 7f09f6f48200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-10T04:34:09.000 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:08 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:08.722+0000 7f09f6f48200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-10T04:34:09.001 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:08 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:08.810+0000 7f09f6f48200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-10T04:34:09.001 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:34:09 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y[48402]: 2024-05-10T04:34:09.000+0000 7f09f6f48200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-10T04:34:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:09.357+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mon.0 v2:172.21.15.150:3300/0 8 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fc764021b10 con 0x7fc774100bb0 2024-05-10T04:34:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:09 smithi181 ceph-mon[47314]: Standby manager daemon y started 2024-05-10T04:34:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:09 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T04:34:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:09 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T04:34:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:09 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T04:34:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:09 smithi181 ceph-mon[47314]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T04:34:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:09 smithi150 ceph-mon[42404]: Standby manager daemon y started 2024-05-10T04:34:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:09 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T04:34:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:09 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T04:34:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:09 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T04:34:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:09 smithi150 ceph-mon[42404]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T04:34:09.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:09 smithi161 ceph-mon[46912]: Standby manager daemon y started 2024-05-10T04:34:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:09 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-10T04:34:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:09 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-10T04:34:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:09 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-10T04:34:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:09 smithi161 ceph-mon[46912]: from='mgr.? 172.21.15.161:0/1186064471' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-10T04:34:10.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:10 smithi181 ceph-mon[47314]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:10 smithi181 ceph-mon[47314]: mgrmap e15: x(active, since 58s), standbys: z, y 2024-05-10T04:34:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T04:34:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:10 smithi150 ceph-mon[42404]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:10 smithi150 ceph-mon[42404]: mgrmap e15: x(active, since 58s), standbys: z, y 2024-05-10T04:34:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T04:34:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:10 smithi161 ceph-mon[46912]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:10 smithi161 ceph-mon[46912]: mgrmap e15: x(active, since 58s), standbys: z, y 2024-05-10T04:34:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-10T04:34:12.620 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:12 smithi150 ceph-mon[42404]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:12.620 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T04:34:12.620 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:12.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:12 smithi181 ceph-mon[47314]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:12.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T04:34:12.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:12 smithi161 ceph-mon[46912]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-10T04:34:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:13.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:13 smithi181 ceph-mon[47314]: Deploying daemon osd.0 on smithi150 2024-05-10T04:34:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:13 smithi150 ceph-mon[42404]: Deploying daemon osd.0 on smithi150 2024-05-10T04:34:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:13 smithi161 ceph-mon[46912]: Deploying daemon osd.0 on smithi150 2024-05-10T04:34:14.642 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:14 smithi150 ceph-mon[42404]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:14.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:14 smithi181 ceph-mon[47314]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:14 smithi161 ceph-mon[46912]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:15.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:15 smithi181 ceph-mon[47314]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:15.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:15 smithi150 ceph-mon[42404]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:15.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:15 smithi161 ceph-mon[46912]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:17.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:17.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:17.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:17.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:17.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:17.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:17.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:17.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:17.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.172+0000 7fc7717fa700 1 -- 172.21.15.150:0/811930961 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc77410d7b0 con 0x7fc7600389d0 2024-05-10T04:34:18.173 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 0 on host 'smithi150' 2024-05-10T04:34:18.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc7600389d0 msgr2=0x7fc76003aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc7600389d0 0x7fc76003aee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7741992a0 tx=0x7fc75c009260 comp rx=0 tx=0).stop 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774100bb0 msgr2=0x7fc7741982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774100bb0 0x7fc7741982d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc76400e9f0 tx=0x7fc76400ed00 comp rx=0 tx=0).stop 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 shutdown_connections 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc774102680 0x7fc7741995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc774100bb0 0x7fc7741982d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc7600389d0 0x7fc76003aee0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 --2- 172.21.15.150:0/811930961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774101ce0 0x7fc774198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 >> 172.21.15.150:0/811930961 conn(0x7fc7740fc1d0 msgr2=0x7fc774104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 shutdown_connections 2024-05-10T04:34:18.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:18.174+0000 7fc77a6d4700 1 -- 172.21.15.150:0/811930961 wait complete. 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.537 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:18.539 DEBUG:teuthology.orchestra.run.smithi150:osd.0> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.0.service 2024-05-10T04:34:18.542 INFO:tasks.cephadm:Deploying osd.1 on smithi161 with /dev/vg_nvme/lv_4... 2024-05-10T04:34:18.542 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T04:34:18.573 INFO:journalctl@ceph.osd.0.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:34:18.576 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.576 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.576 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:18.576 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:18.577 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:18.577 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.577 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.577 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.577 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:18.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:18.771 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:34:20.046 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T04:34:20.046 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T04:34:20.046 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T04:34:20.046 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T04:34:20.046 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126622 s, 828 MB/s 2024-05-10T04:34:20.046 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T04:34:20.264 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:34:19 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-0[60251]: 2024-05-10T04:34:19.953+0000 7f23b6cda7c0 -1 osd.0 0 log_to_monitors true 2024-05-10T04:34:20.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:20 smithi161 ceph-mon[46912]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:20 smithi161 ceph-mon[46912]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T04:34:20.419 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-05-10T04:34:20.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:20 smithi181 ceph-mon[47314]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:20.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:20 smithi181 ceph-mon[47314]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T04:34:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:20 smithi150 ceph-mon[42404]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:20 smithi150 ceph-mon[42404]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-10T04:34:20.677 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:21.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:21.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T04:34:21.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T04:34:21.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:21.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:34:21.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:21.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:21.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:21.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:34:21 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-0[60251]: 2024-05-10T04:34:21.317+0000 7f23ac33e700 -1 osd.0 0 waiting for initial osdmap 2024-05-10T04:34:21.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:34:21 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-0[60251]: 2024-05-10T04:34:21.323+0000 7f23a80f5700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:34:21.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-10T04:34:21.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: osdmap e6: 1 total, 0 up, 1 in 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:21.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:21.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 -- 172.21.15.161:0/1807343587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 msgr2=0x7fdad0109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 --2- 172.21.15.161:0/1807343587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad0109eb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b0a0 tx=0x7fdac800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:34:21.689 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 -- 172.21.15.161:0/1807343587 shutdown_connections 2024-05-10T04:34:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 --2- 172.21.15.161:0/1807343587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad0109eb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 --2- 172.21.15.161:0/1807343587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad01027d0 0x7fdad0104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 --2- 172.21.15.161:0/1807343587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad0105130 0x7fdad0107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 -- 172.21.15.161:0/1807343587 >> 172.21.15.161:0/1807343587 conn(0x7fdad00fc230 msgr2=0x7fdad00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:34:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 -- 172.21.15.161:0/1807343587 shutdown_connections 2024-05-10T04:34:21.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.684+0000 7fdad5747700 1 -- 172.21.15.161:0/1807343587 wait complete. 2024-05-10T04:34:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 Processor -- start 2024-05-10T04:34:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 -- start start 2024-05-10T04:34:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad01027d0 0x7fdad01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad0105130 0x7fdad0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad01994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:21.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0199b20 con 0x7fdad0105130 2024-05-10T04:34:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0199c60 con 0x7fdad0107a90 2024-05-10T04:34:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdad5747700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0199e20 con 0x7fdad01027d0 2024-05-10T04:34:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdacf7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad01994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdacf7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad01994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56976/0 (socket says 172.21.15.161:56976) 2024-05-10T04:34:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.685+0000 7fdacf7fe700 1 -- 172.21.15.161:0/3948428152 learned_addr learned my addr 172.21.15.161:0/3948428152 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T04:34:21.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdaceffd700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad01027d0 0x7fdad01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdace7fc700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad0105130 0x7fdad0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdacf7fe700 1 -- 172.21.15.161:0/3948428152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad01027d0 msgr2=0x7fdad01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdacf7fe700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad01027d0 0x7fdad01981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdacf7fe700 1 -- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad0105130 msgr2=0x7fdad0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdacf7fe700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad0105130 0x7fdad0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:21.693 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdacf7fe700 1 -- 172.21.15.161:0/3948428152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8009d00 con 0x7fdad0107a90 2024-05-10T04:34:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdacf7fe700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad01994c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006070 tx=0x7fdac8004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdab7fff700 1 -- 172.21.15.161:0/3948428152 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800e040 con 0x7fdad0107a90 2024-05-10T04:34:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdab7fff700 1 -- 172.21.15.161:0/3948428152 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdac800cec0 con 0x7fdad0107a90 2024-05-10T04:34:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad019a0a0 con 0x7fdad0107a90 2024-05-10T04:34:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdab7fff700 1 -- 172.21.15.161:0/3948428152 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac801dd70 con 0x7fdad0107a90 2024-05-10T04:34:21.694 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.686+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad01a08e0 con 0x7fdad0107a90 2024-05-10T04:34:21.695 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.687+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad00fdde0 con 0x7fdad0107a90 2024-05-10T04:34:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.689+0000 7fdab7fff700 1 -- 172.21.15.161:0/3948428152 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fdac8003a00 con 0x7fdad0107a90 2024-05-10T04:34:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.689+0000 7fdab7fff700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdab80a1090 0x7fdab80a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:21.696 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.689+0000 7fdab7fff700 1 -- 172.21.15.161:0/3948428152 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7fdac80ccbd0 con 0x7fdad0107a90 2024-05-10T04:34:21.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.689+0000 7fdaceffd700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdab80a1090 0x7fdab80a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:21.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.690+0000 7fdaceffd700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdab80a1090 0x7fdab80a35a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdac4005950 tx=0x7fdac400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:21.697 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.693+0000 7fdab7fff700 1 -- 172.21.15.161:0/3948428152 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdac8097f80 con 0x7fdad0107a90 2024-05-10T04:34:21.895 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:21.893+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdad010d5d0 con 0x7fdab80a1090 2024-05-10T04:34:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi150 2024-05-10T04:34:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi150 to 17114M 2024-05-10T04:34:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:22.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:34:22.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T04:34:22.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:22.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:22.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:22.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:22.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:22 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:22.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi150 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi150 to 17114M 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:22 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi150 2024-05-10T04:34:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi150 to 17114M 2024-05-10T04:34:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: from='osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:34:22.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: osdmap e7: 1 total, 0 up, 1 in 2024-05-10T04:34:22.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:22.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:22.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:22.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:22.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:23.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:23 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:34:23.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:23 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:34:23.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:23 smithi161 ceph-mon[46912]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:23.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:23.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:23 smithi161 ceph-mon[46912]: osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] boot 2024-05-10T04:34:23.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:23 smithi161 ceph-mon[46912]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T04:34:23.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:23.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:23 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:34:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:23 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:34:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:23 smithi181 ceph-mon[47314]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:23 smithi181 ceph-mon[47314]: osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] boot 2024-05-10T04:34:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:23 smithi181 ceph-mon[47314]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T04:34:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:23.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:23 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:34:23.749 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:23 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:34:23.749 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:23 smithi150 ceph-mon[42404]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:23.749 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:23.749 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:23 smithi150 ceph-mon[42404]: osd.0 [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] boot 2024-05-10T04:34:23.749 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:23 smithi150 ceph-mon[42404]: osdmap e8: 1 total, 1 up, 1 in 2024-05-10T04:34:23.749 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-10T04:34:24.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:24 smithi161 ceph-mon[46912]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:24.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:24 smithi161 ceph-mon[46912]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T04:34:24.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:24 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/1786112146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]: dispatch 2024-05-10T04:34:24.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:24 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]: dispatch 2024-05-10T04:34:24.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:24 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]': finished 2024-05-10T04:34:24.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:24 smithi161 ceph-mon[46912]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T04:34:24.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:24 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:24.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:24 smithi181 ceph-mon[47314]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:24 smithi181 ceph-mon[47314]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T04:34:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:24 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/1786112146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]: dispatch 2024-05-10T04:34:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:24 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]: dispatch 2024-05-10T04:34:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:24 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]': finished 2024-05-10T04:34:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:24 smithi181 ceph-mon[47314]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T04:34:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:24 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:24 smithi150 ceph-mon[42404]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-10T04:34:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:24 smithi150 ceph-mon[42404]: osdmap e9: 1 total, 1 up, 1 in 2024-05-10T04:34:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:24 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/1786112146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]: dispatch 2024-05-10T04:34:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:24 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]: dispatch 2024-05-10T04:34:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:24 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a6ad73-ce28-44f3-bdf4-178f42703543"}]': finished 2024-05-10T04:34:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:24 smithi150 ceph-mon[42404]: osdmap e10: 2 total, 1 up, 2 in 2024-05-10T04:34:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:24 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:25.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:25 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/3778222035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:25 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/3778222035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:25.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:25 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/3778222035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:26.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:26 smithi181 ceph-mon[47314]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:26 smithi150 ceph-mon[42404]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:26.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:26 smithi161 ceph-mon[46912]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:28.554 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:28 smithi161 ceph-mon[46912]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:28.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:28 smithi181 ceph-mon[47314]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:28.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:28 smithi150 ceph-mon[42404]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:30.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:30 smithi181 ceph-mon[47314]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T04:34:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:30 smithi150 ceph-mon[42404]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T04:34:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:30.763 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:30 smithi161 ceph-mon[46912]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:30.763 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-10T04:34:30.763 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:31.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:31 smithi181 ceph-mon[47314]: Deploying daemon osd.1 on smithi161 2024-05-10T04:34:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:31 smithi150 ceph-mon[42404]: Deploying daemon osd.1 on smithi161 2024-05-10T04:34:31.709 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:31 smithi161 ceph-mon[46912]: Deploying daemon osd.1 on smithi161 2024-05-10T04:34:32.620 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:32 smithi161 ceph-mon[46912]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:32.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:32 smithi181 ceph-mon[47314]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:32 smithi150 ceph-mon[42404]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:34 smithi161 ceph-mon[46912]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:34.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:34 smithi181 ceph-mon[47314]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:34 smithi150 ceph-mon[42404]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:35.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.752+0000 7fdab7fff700 1 -- 172.21.15.161:0/3948428152 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdad010d5d0 con 0x7fdab80a1090 2024-05-10T04:34:35.755 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 1 on host 'smithi161' 2024-05-10T04:34:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.754+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdab80a1090 msgr2=0x7fdab80a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.754+0000 7fdad5747700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdab80a1090 0x7fdab80a35a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdac4005950 tx=0x7fdac400b410 comp rx=0 tx=0).stop 2024-05-10T04:34:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 msgr2=0x7fdad01994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad01994c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006070 tx=0x7fdac8004c80 comp rx=0 tx=0).stop 2024-05-10T04:34:35.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 shutdown_connections 2024-05-10T04:34:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad0107a90 0x7fdad01994c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdab80a1090 0x7fdab80a35a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad01027d0 0x7fdad01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 --2- 172.21.15.161:0/3948428152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdad0105130 0x7fdad0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 >> 172.21.15.161:0/3948428152 conn(0x7fdad00fc230 msgr2=0x7fdad0103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:34:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 shutdown_connections 2024-05-10T04:34:35.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:34:35.755+0000 7fdad5747700 1 -- 172.21.15.161:0/3948428152 wait complete. 2024-05-10T04:34:35.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:35.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:35.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:35 smithi181 ceph-mon[47314]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:35.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:35.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:35.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:35 smithi150 ceph-mon[42404]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:35.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:35.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:35.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:35.920 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:35 smithi161 ceph-mon[46912]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:36.141 DEBUG:teuthology.orchestra.run.smithi161:osd.1> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.1.service 2024-05-10T04:34:36.144 INFO:tasks.cephadm:Deploying osd.2 on smithi181 with /dev/vg_nvme/lv_4... 2024-05-10T04:34:36.144 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-10T04:34:36.174 INFO:journalctl@ceph.osd.1.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:34:36.370 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:34:36.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:36.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:36.771 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:36.849 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:36.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:36.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:37.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:37.586 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:34:37 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-1[51874]: 2024-05-10T04:34:37.325+0000 7f514b29a7c0 -1 osd.1 0 log_to_monitors true 2024-05-10T04:34:37.762 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-10T04:34:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-10T04:34:37.763 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T04:34:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T04:34:37.763 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112259 s, 934 MB/s 2024-05-10T04:34:37.763 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T04:34:37.974 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:37 smithi181 ceph-mon[47314]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:37.974 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:37 smithi181 ceph-mon[47314]: from='osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T04:34:37.974 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:37 smithi181 ceph-mon[47314]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T04:34:38.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:37 smithi161 ceph-mon[46912]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:38.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:37 smithi161 ceph-mon[46912]: from='osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T04:34:38.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:37 smithi161 ceph-mon[46912]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T04:34:38.130 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-05-10T04:34:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:37 smithi150 ceph-mon[42404]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:37 smithi150 ceph-mon[42404]: from='osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T04:34:38.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:37 smithi150 ceph-mon[42404]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-10T04:34:38.347 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:34:38.849 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T04:34:38.849 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T04:34:38.849 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi161 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi161 to 17114M 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:38.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:38 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi161 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:39.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi161 to 17114M 2024-05-10T04:34:39.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:39.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:39.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:38 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: osdmap e11: 2 total, 1 up, 2 in 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi161 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi161 to 17114M 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:39.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:39.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:38 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:39.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 -- 172.21.15.181:0/276045515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 msgr2=0x7f1d20109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 --2- 172.21.15.181:0/276045515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20109f50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b0a0 tx=0x7f1d1800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 -- 172.21.15.181:0/276045515 shutdown_connections 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 --2- 172.21.15.181:0/276045515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20109f50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 --2- 172.21.15.181:0/276045515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d201000a0 0x7f1d20100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 --2- 172.21.15.181:0/276045515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20105140 0x7f1d20107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 -- 172.21.15.181:0/276045515 >> 172.21.15.181:0/276045515 conn(0x7f1d200fb950 msgr2=0x7f1d200fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 -- 172.21.15.181:0/276045515 shutdown_connections 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.395+0000 7f1d25d90700 1 -- 172.21.15.181:0/276045515 wait complete. 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 Processor -- start 2024-05-10T04:34:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 -- start start 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201000a0 0x7f1d20198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d20105140 0x7f1d201987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20199b20 con 0x7f1d20105140 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20199c60 con 0x7f1d201000a0 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d25d90700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20199e20 con 0x7f1d20107aa0 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.396+0000 7f1d1ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49034/0 (socket says 172.21.15.181:49034) 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 -- 172.21.15.181:0/2922800505 learned_addr learned my addr 172.21.15.181:0/2922800505 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:34:39.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1f7fe700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201000a0 0x7f1d20198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1effd700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d20105140 0x7f1d201987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 -- 172.21.15.181:0/2922800505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201000a0 msgr2=0x7f1d20198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201000a0 0x7f1d20198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 -- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d20105140 msgr2=0x7f1d201987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d20105140 0x7f1d201987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 -- 172.21.15.181:0/2922800505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18009d00 con 0x7f1d20107aa0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1ffff700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20199550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b070 tx=0x7f1d1800bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1cff9700 1 -- 172.21.15.181:0/2922800505 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800e040 con 0x7f1d20107aa0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1cff9700 1 -- 172.21.15.181:0/2922800505 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d1800be30 con 0x7f1d20107aa0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d1cff9700 1 -- 172.21.15.181:0/2922800505 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1801b680 con 0x7f1d20107aa0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2019a0a0 con 0x7f1d20107aa0 2024-05-10T04:34:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.397+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d201a07a0 con 0x7f1d20107aa0 2024-05-10T04:34:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.398+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d200fd500 con 0x7f1d20107aa0 2024-05-10T04:34:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.399+0000 7f1d1cff9700 1 -- 172.21.15.181:0/2922800505 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f1d1801d470 con 0x7f1d20107aa0 2024-05-10T04:34:39.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.400+0000 7f1d1cff9700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d0c0a1110 0x7f1d0c0a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.400+0000 7f1d1cff9700 1 -- 172.21.15.181:0/2922800505 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f1d180cbfd0 con 0x7f1d20107aa0 2024-05-10T04:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.400+0000 7f1d1f7fe700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d0c0a1110 0x7f1d0c0a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:39.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.401+0000 7f1d1f7fe700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d0c0a1110 0x7f1d0c0a3620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d08005ea0 tx=0x7f1d08005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.405+0000 7f1d1cff9700 1 -- 172.21.15.181:0/2922800505 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1d180ffd60 con 0x7f1d20107aa0 2024-05-10T04:34:39.585 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:34:39 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-1[51874]: 2024-05-10T04:34:39.214+0000 7f51410ff700 -1 osd.1 0 waiting for initial osdmap 2024-05-10T04:34:39.585 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:34:39 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-1[51874]: 2024-05-10T04:34:39.220+0000 7f51386ad700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:34:39.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:39.609+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d2010d720 con 0x7f1d0c0a1110 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: from='osd.1 ' entity='osd.1' 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:40.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:40.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: from='osd.1 ' entity='osd.1' 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:40.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:40.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:34:40.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: osdmap e12: 2 total, 1 up, 2 in 2024-05-10T04:34:40.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:40.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:40.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: from='osd.1 ' entity='osd.1' 2024-05-10T04:34:40.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-10T04:34:40.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:40.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:40.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:41.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:40 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:34:41.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:40 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:34:41.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:40 smithi161 ceph-mon[46912]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:41.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:40 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:41.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:40 smithi161 ceph-mon[46912]: osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] boot 2024-05-10T04:34:41.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:40 smithi161 ceph-mon[46912]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T04:34:41.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:40 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:40 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:34:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:40 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:34:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:40 smithi181 ceph-mon[47314]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:40 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:40 smithi181 ceph-mon[47314]: osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] boot 2024-05-10T04:34:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:40 smithi181 ceph-mon[47314]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T04:34:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:40 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:41.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:40 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:34:41.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:40 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:34:41.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:40 smithi150 ceph-mon[42404]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:41.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:40 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:41.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:40 smithi150 ceph-mon[42404]: osd.1 [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] boot 2024-05-10T04:34:41.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:40 smithi150 ceph-mon[42404]: osdmap e13: 2 total, 2 up, 2 in 2024-05-10T04:34:41.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:40 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-10T04:34:42.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:42 smithi161 ceph-mon[46912]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T04:34:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:42 smithi161 ceph-mon[46912]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:42 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/873766322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]: dispatch 2024-05-10T04:34:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:42 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]: dispatch 2024-05-10T04:34:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:42 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]': finished 2024-05-10T04:34:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:42 smithi161 ceph-mon[46912]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T04:34:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:42.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:42 smithi181 ceph-mon[47314]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T04:34:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:42 smithi181 ceph-mon[47314]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:42 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/873766322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]: dispatch 2024-05-10T04:34:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:42 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]: dispatch 2024-05-10T04:34:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:42 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]': finished 2024-05-10T04:34:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:42 smithi181 ceph-mon[47314]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T04:34:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:42 smithi150 ceph-mon[42404]: osdmap e14: 2 total, 2 up, 2 in 2024-05-10T04:34:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:42 smithi150 ceph-mon[42404]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:42 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/873766322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]: dispatch 2024-05-10T04:34:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:42 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]: dispatch 2024-05-10T04:34:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:42 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1c4f641-65de-4cd3-8a02-b1694aa6e086"}]': finished 2024-05-10T04:34:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:42 smithi150 ceph-mon[42404]: osdmap e15: 3 total, 2 up, 3 in 2024-05-10T04:34:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:43.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:43 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/2268943901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:43.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:43 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/2268943901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:43 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/2268943901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:34:44.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:44 smithi161 ceph-mon[46912]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:44.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:44 smithi181 ceph-mon[47314]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:44 smithi150 ceph-mon[42404]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:46.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:46 smithi161 ceph-mon[46912]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:46 smithi181 ceph-mon[47314]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:46 smithi150 ceph-mon[42404]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:48.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:48 smithi161 ceph-mon[46912]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:48.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T04:34:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:48.598 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:48 smithi181 ceph-mon[47314]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:48.598 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T04:34:48.598 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:48.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:48 smithi150 ceph-mon[42404]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:48.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-10T04:34:48.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:49.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:49 smithi161 ceph-mon[46912]: Deploying daemon osd.2 on smithi181 2024-05-10T04:34:49.630 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:49 smithi181 ceph-mon[47314]: Deploying daemon osd.2 on smithi181 2024-05-10T04:34:49.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:49 smithi150 ceph-mon[42404]: Deploying daemon osd.2 on smithi181 2024-05-10T04:34:50.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:50 smithi161 ceph-mon[46912]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:50.586 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:50 smithi181 ceph-mon[47314]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:50.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:50 smithi150 ceph-mon[42404]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:52.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:52.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:52 smithi181 ceph-mon[47314]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:52.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:52.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:52.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:52 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:52.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:52 smithi161 ceph-mon[46912]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:52.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:52 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:52.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:52 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:52 smithi150 ceph-mon[42404]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:53.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.763+0000 7f1d1cff9700 1 -- 172.21.15.181:0/2922800505 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1d2010d720 con 0x7f1d0c0a1110 2024-05-10T04:34:53.764 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 2 on host 'smithi181' 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d0c0a1110 msgr2=0x7f1d0c0a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d0c0a1110 0x7f1d0c0a3620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d08005ea0 tx=0x7f1d08005e30 comp rx=0 tx=0).stop 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 msgr2=0x7f1d20199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20199550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b070 tx=0x7f1d1800bb60 comp rx=0 tx=0).stop 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 shutdown_connections 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d0c0a1110 0x7f1d0c0a3620 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d20107aa0 0x7f1d20199550 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:53.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d201000a0 0x7f1d20198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 --2- 172.21.15.181:0/2922800505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d20105140 0x7f1d201987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 >> 172.21.15.181:0/2922800505 conn(0x7f1d200fb950 msgr2=0x7f1d20105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:34:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 shutdown_connections 2024-05-10T04:34:53.768 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:34:53.766+0000 7f1d25d90700 1 -- 172.21.15.181:0/2922800505 wait complete. 2024-05-10T04:34:53.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:53.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:53.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:53.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:53.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:53.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:53 smithi181 ceph-mon[47314]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:54.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:54.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:54.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:53 smithi161 ceph-mon[46912]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:54.143 DEBUG:teuthology.orchestra.run.smithi181:osd.2> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.2.service 2024-05-10T04:34:54.145 INFO:tasks.cephadm:Deploying osd.3 on smithi150 with /dev/vg_nvme/lv_3... 2024-05-10T04:34:54.145 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T04:34:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:54.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:54.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:53 smithi150 ceph-mon[42404]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:54.178 INFO:journalctl@ceph.osd.2.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:34:54.380 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:34:54.783 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.837 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:54.837 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:55.006 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:55.006 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:55.006 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:55.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:55.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:55.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:34:55.651 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T04:34:55.651 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T04:34:55.651 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T04:34:55.651 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T04:34:55.651 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117116 s, 895 MB/s 2024-05-10T04:34:55.651 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T04:34:55.655 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:34:55 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-2[53066]: 2024-05-10T04:34:55.278+0000 7f3f53fb07c0 -1 osd.2 0 log_to_monitors true 2024-05-10T04:34:56.002 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:55 smithi150 ceph-mon[42404]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:56.002 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:55 smithi150 ceph-mon[42404]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T04:34:56.004 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-05-10T04:34:56.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:55 smithi161 ceph-mon[46912]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:56.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:55 smithi161 ceph-mon[46912]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T04:34:56.113 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:55 smithi181 ceph-mon[47314]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:56.113 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:55 smithi181 ceph-mon[47314]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-10T04:34:56.263 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:34:57.058 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T04:34:57.058 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi181 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi181 to 17113M 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:57.059 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T04:34:57.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T04:34:57.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:57.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:34:57.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi181 2024-05-10T04:34:57.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:57.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi181 to 17113M 2024-05-10T04:34:57.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:57.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:57.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.155 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:34:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-2[53066]: 2024-05-10T04:34:56.730+0000 7f3f49614700 -1 osd.2 0 waiting for initial osdmap 2024-05-10T04:34:57.155 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:34:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-2[53066]: 2024-05-10T04:34:56.739+0000 7f3f453cb700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: osdmap e16: 3 total, 2 up, 3 in 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi181 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:34:57.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi181 to 17113M 2024-05-10T04:34:57.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:57.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:34:57.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 -- 172.21.15.150:0/3350160698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 msgr2=0x7f0f30103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 --2- 172.21.15.150:0/3350160698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30103290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009990 tx=0x7f0f20009ca0 comp rx=0 tx=0).stop 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 -- 172.21.15.150:0/3350160698 shutdown_connections 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 --2- 172.21.15.150:0/3350160698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f30103820 0x7f0f3010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 --2- 172.21.15.150:0/3350160698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f30101d50 0x7f0f30102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 --2- 172.21.15.150:0/3350160698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30103290 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 -- 172.21.15.150:0/3350160698 >> 172.21.15.150:0/3350160698 conn(0x7f0f300fd390 msgr2=0x7f0f300ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 -- 172.21.15.150:0/3350160698 shutdown_connections 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.228+0000 7f0f37f97700 1 -- 172.21.15.150:0/3350160698 wait complete. 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 Processor -- start 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 -- start start 2024-05-10T04:34:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f30101d50 0x7f0f30198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f30103820 0x7f0f30199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30199ba0 con 0x7f0f30102e80 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30199ce0 con 0x7f0f30101d50 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f37f97700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30199ea0 con 0x7f0f30103820 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f35532700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f35532700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45960/0 (socket says 172.21.15.150:45960) 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.229+0000 7f0f35532700 1 -- 172.21.15.150:0/2755967939 learned_addr learned my addr 172.21.15.150:0/2755967939 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f36534700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f30103820 0x7f0f30199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35d33700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f30101d50 0x7f0f30198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35532700 1 -- 172.21.15.150:0/2755967939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f30103820 msgr2=0x7f0f30199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35532700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f30103820 0x7f0f30199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35532700 1 -- 172.21.15.150:0/2755967939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f30101d50 msgr2=0x7f0f30198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:34:57.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35532700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f30101d50 0x7f0f30198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35532700 1 -- 172.21.15.150:0/2755967939 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200095f0 con 0x7f0f30102e80 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35d33700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f30101d50 0x7f0f30198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f35532700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30198790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0f20003800 tx=0x7f0f2000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2001d070 con 0x7f0f30102e80 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3019a120 con 0x7f0f30102e80 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2000fc70 con 0x7f0f30102e80 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200176e0 con 0x7f0f30102e80 2024-05-10T04:34:57.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.230+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f301a0960 con 0x7f0f30102e80 2024-05-10T04:34:57.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.231+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f30108f70 con 0x7f0f30102e80 2024-05-10T04:34:57.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.232+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f0f200229c0 con 0x7f0f30102e80 2024-05-10T04:34:57.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.233+0000 7f0f26ffd700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0f1c0a1090 0x7f0f1c0a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:34:57.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.233+0000 7f0f35d33700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0f1c0a1090 0x7f0f1c0a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:34:57.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.233+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7f0f200cc6f0 con 0x7f0f30102e80 2024-05-10T04:34:57.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.234+0000 7f0f35d33700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0f1c0a1090 0x7f0f1c0a35a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f30102d80 tx=0x7f0f2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:34:57.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.237+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0f2005dd60 con 0x7f0f30102e80 2024-05-10T04:34:57.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:34:57.440+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3010d6d0 con 0x7f0f1c0a1090 2024-05-10T04:34:58.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:34:58.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T04:34:58.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:58.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:58.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:58.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:58.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.107 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:58.108 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: from='osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: osdmap e17: 3 total, 2 up, 3 in 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:34:58.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:59.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:58 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:34:59.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:58 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:34:59.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:58 smithi161 ceph-mon[46912]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:59.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:58 smithi161 ceph-mon[46912]: osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] boot 2024-05-10T04:34:59.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:58 smithi161 ceph-mon[46912]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T04:34:59.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:59.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:58 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:34:59.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:58 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:34:59.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:58 smithi181 ceph-mon[47314]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:59.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:58 smithi181 ceph-mon[47314]: osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] boot 2024-05-10T04:34:59.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:58 smithi181 ceph-mon[47314]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T04:34:59.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:34:59.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:58 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:34:59.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:58 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:34:59.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:58 smithi150 ceph-mon[42404]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:34:59.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:58 smithi150 ceph-mon[42404]: osd.2 [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] boot 2024-05-10T04:34:59.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:58 smithi150 ceph-mon[42404]: osdmap e18: 3 total, 3 up, 3 in 2024-05-10T04:34:59.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-10T04:35:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:59 smithi161 ceph-mon[46912]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T04:35:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:59 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/826349408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "589765c5-703f-4ac8-b3d4-07fdbc0de50e"}]: dispatch 2024-05-10T04:35:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:59 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/826349408' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "589765c5-703f-4ac8-b3d4-07fdbc0de50e"}]': finished 2024-05-10T04:35:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:59 smithi161 ceph-mon[46912]: osdmap e20: 4 total, 3 up, 4 in 2024-05-10T04:35:00.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:00.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:59 smithi161 ceph-mon[46912]: pgmap v64: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:00.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:34:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:35:00.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:59 smithi181 ceph-mon[47314]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T04:35:00.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:59 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/826349408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "589765c5-703f-4ac8-b3d4-07fdbc0de50e"}]: dispatch 2024-05-10T04:35:00.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:59 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/826349408' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "589765c5-703f-4ac8-b3d4-07fdbc0de50e"}]': finished 2024-05-10T04:35:00.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:59 smithi181 ceph-mon[47314]: osdmap e20: 4 total, 3 up, 4 in 2024-05-10T04:35:00.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:00.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:59 smithi181 ceph-mon[47314]: pgmap v64: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:00.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:34:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:35:00.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:59 smithi150 ceph-mon[42404]: osdmap e19: 3 total, 3 up, 3 in 2024-05-10T04:35:00.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:59 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/826349408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "589765c5-703f-4ac8-b3d4-07fdbc0de50e"}]: dispatch 2024-05-10T04:35:00.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:59 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/826349408' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "589765c5-703f-4ac8-b3d4-07fdbc0de50e"}]': finished 2024-05-10T04:35:00.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:59 smithi150 ceph-mon[42404]: osdmap e20: 4 total, 3 up, 4 in 2024-05-10T04:35:00.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:00.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:59 smithi150 ceph-mon[42404]: pgmap v64: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:00.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:34:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:35:01.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:00 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3145402217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:01.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T04:35:01.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:00 smithi161 ceph-mon[46912]: osdmap e21: 4 total, 3 up, 4 in 2024-05-10T04:35:01.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:01.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:35:01.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:00 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3145402217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:01.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T04:35:01.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:00 smithi181 ceph-mon[47314]: osdmap e21: 4 total, 3 up, 4 in 2024-05-10T04:35:01.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:01.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:35:01.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:00 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3145402217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:01.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-10T04:35:01.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:00 smithi150 ceph-mon[42404]: osdmap e21: 4 total, 3 up, 4 in 2024-05-10T04:35:01.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:01.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:35:02.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:02.266+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0f20097990 con 0x7f0f30102e80 2024-05-10T04:35:02.366 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:35:02.366 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:02 smithi150 ceph-mon[42404]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T04:35:02.366 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:02.366 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:02 smithi150 ceph-mon[42404]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:02.366 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T04:35:02.367 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:02.367 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67558]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:02.367 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67558]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:02.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:35:02.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:02 smithi161 ceph-mon[46912]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T04:35:02.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:02.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:02 smithi161 ceph-mon[46912]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:02.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:35:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:02 smithi181 ceph-mon[47314]: osdmap e22: 4 total, 3 up, 4 in 2024-05-10T04:35:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:02 smithi181 ceph-mon[47314]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:02.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67751]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T04:35:02.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67751]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:02.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67751]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:02.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:35:02 smithi150 sudo[67751]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:02.886 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56694]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T04:35:02.886 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56694]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:02.886 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56694]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:02.886 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56694]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:02.886 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56697]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T04:35:02.887 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56697]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:02.887 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56697]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:02.887 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:35:02 smithi161 sudo[56697]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:03.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:03 smithi181 ceph-mon[47314]: mgrmap e16: x(active, since 111s), standbys: z, y 2024-05-10T04:35:03.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:03 smithi181 ceph-mon[47314]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T04:35:03.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:03.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:03.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:03.398 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57847]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T04:35:03.398 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57847]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:03.399 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57847]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:03.399 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57847]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:03.399 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57850]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T04:35:03.399 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57850]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:03.399 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57850]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:03.399 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:35:03 smithi181 sudo[57850]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:03 smithi161 ceph-mon[46912]: mgrmap e16: x(active, since 111s), standbys: z, y 2024-05-10T04:35:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:03 smithi161 ceph-mon[46912]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T04:35:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:03.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:03.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:03.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 ceph-mon[42404]: mgrmap e16: x(active, since 111s), standbys: z, y 2024-05-10T04:35:03.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 ceph-mon[42404]: osdmap e23: 4 total, 3 up, 4 in 2024-05-10T04:35:03.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:03.648 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:03.648 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:03.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T04:35:03.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:03.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68029]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:03.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68029]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:04.306 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68052]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T04:35:04.306 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68052]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:04.307 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68052]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:04.307 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:03 smithi150 sudo[68052]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:04.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57853]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T04:35:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57853]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57853]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57856]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T04:35:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57856]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 sudo[57856]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56700]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56700]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56700]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56703]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56703]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56703]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-10T04:35:04.911 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 sudo[56703]: pam_unix(sudo:session): session closed for user root 2024-05-10T04:35:05.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:05.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:05.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:05.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:05.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:05.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:05.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:05.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:05.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:05.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:05.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:05.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:05.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:05.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:05.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:05.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:05.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:05.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:05.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:05.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:05.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T04:35:06.133 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:06.134 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: Deploying daemon osd.3 on smithi150 2024-05-10T04:35:06.134 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:05 smithi150 ceph-mon[42404]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:06.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:06.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: Deploying daemon osd.3 on smithi150 2024-05-10T04:35:06.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:05 smithi161 ceph-mon[46912]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:06.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: Deploying daemon osd.3 on smithi150 2024-05-10T04:35:06.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:05 smithi181 ceph-mon[47314]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:08.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:08 smithi150 ceph-mon[42404]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:08.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:08 smithi161 ceph-mon[46912]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:08.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:08 smithi181 ceph-mon[47314]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:09.836 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:09 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:09.836 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:09 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:09.836 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:09 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:09.836 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:09 smithi150 ceph-mon[42404]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:10.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:09 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:10.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:09 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:10.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:09 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:10.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:09 smithi161 ceph-mon[46912]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:10.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:09 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:10.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:09 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:10.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:09 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:10.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:09 smithi181 ceph-mon[47314]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:10.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.831+0000 7f0f26ffd700 1 -- 172.21.15.150:0/2755967939 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0f3010d6d0 con 0x7f0f1c0a1090 2024-05-10T04:35:10.833 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 3 on host 'smithi150' 2024-05-10T04:35:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0f1c0a1090 msgr2=0x7f0f1c0a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:10.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0f1c0a1090 0x7f0f1c0a35a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f30102d80 tx=0x7f0f2c008040 comp rx=0 tx=0).stop 2024-05-10T04:35:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 msgr2=0x7f0f30198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30198790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0f20003800 tx=0x7f0f2000f740 comp rx=0 tx=0).stop 2024-05-10T04:35:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 shutdown_connections 2024-05-10T04:35:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0f1c0a1090 0x7f0f1c0a35a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f30103820 0x7f0f30199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f30101d50 0x7f0f30198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 --2- 172.21.15.150:0/2755967939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f30102e80 0x7f0f30198790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.835+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 >> 172.21.15.150:0/2755967939 conn(0x7f0f300fd390 msgr2=0x7f0f301060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:35:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.836+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 shutdown_connections 2024-05-10T04:35:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:10.836+0000 7f0f37f97700 1 -- 172.21.15.150:0/2755967939 wait complete. 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.006 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:11.201 DEBUG:teuthology.orchestra.run.smithi150:osd.3> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.3.service 2024-05-10T04:35:11.203 INFO:tasks.cephadm:Deploying osd.4 on smithi161 with /dev/vg_nvme/lv_3... 2024-05-10T04:35:11.203 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T04:35:11.236 INFO:journalctl@ceph.osd.3.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:35:11.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.237 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:11.238 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:11.238 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.238 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.238 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.238 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:11.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:11.438 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:35:12.215 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:11 smithi161 ceph-mon[46912]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:12.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:11 smithi181 ceph-mon[47314]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:12.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:11 smithi150 ceph-mon[42404]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:12.731 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T04:35:12.731 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T04:35:12.731 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T04:35:12.731 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T04:35:12.732 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134719 s, 778 MB/s 2024-05-10T04:35:12.732 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T04:35:12.997 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:12 smithi150 ceph-mon[42404]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T04:35:12.998 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:35:12 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-3[68766]: 2024-05-10T04:35:12.720+0000 7f60a39837c0 -1 osd.3 0 log_to_monitors true 2024-05-10T04:35:13.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:12 smithi161 ceph-mon[46912]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T04:35:13.119 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-05-10T04:35:13.380 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:35:13.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:12 smithi181 ceph-mon[47314]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-10T04:35:14.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi150 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi150 to 8557M 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:13 smithi161 ceph-mon[46912]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:14.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 -- 172.21.15.161:0/1534900756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8102650 msgr2=0x7f44b8108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 --2- 172.21.15.161:0/1534900756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8102650 0x7f44b8108f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f44b000b0a0 tx=0x7f44b000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 -- 172.21.15.161:0/1534900756 shutdown_connections 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 --2- 172.21.15.161:0/1534900756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8102650 0x7f44b8108f00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 --2- 172.21.15.161:0/1534900756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b8100b80 0x7f44b8100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 --2- 172.21.15.161:0/1534900756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b8101cb0 0x7f44b81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 -- 172.21.15.161:0/1534900756 >> 172.21.15.161:0/1534900756 conn(0x7f44b80fc1c0 msgr2=0x7f44b80fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 -- 172.21.15.161:0/1534900756 shutdown_connections 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.268+0000 7f44be8bd700 1 -- 172.21.15.161:0/1534900756 wait complete. 2024-05-10T04:35:14.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.269+0000 7f44be8bd700 1 Processor -- start 2024-05-10T04:35:14.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.269+0000 7f44be8bd700 1 -- start start 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.269+0000 7f44be8bd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8100b80 0x7f44b8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.269+0000 7f44be8bd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b8101cb0 0x7f44b8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44be8bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b8102650 0x7f44b81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44be8bd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b8198a50 con 0x7f44b8101cb0 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44be8bd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b8198b90 con 0x7f44b8100b80 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44be8bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b8198d50 con 0x7f44b8102650 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8100b80 0x7f44b8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8100b80 0x7f44b8197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:33394/0 (socket says 172.21.15.161:33394) 2024-05-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 -- 172.21.15.161:0/3214253605 learned_addr learned my addr 172.21.15.161:0/3214253605 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44affff700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b8101cb0 0x7f44b8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44bce5a700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b8102650 0x7f44b81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 -- 172.21.15.161:0/3214253605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b8102650 msgr2=0x7f44b81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b8102650 0x7f44b81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 -- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b8101cb0 msgr2=0x7f44b8197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b8101cb0 0x7f44b8197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 -- 172.21.15.161:0/3214253605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0009d00 con 0x7f44b8100b80 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44bce5a700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b8102650 0x7f44b81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b7fff700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8100b80 0x7f44b8197100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f44a800e940 tx=0x7f44a800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b5ffb700 1 -- 172.21.15.161:0/3214253605 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a800cbd0 con 0x7f44b8100b80 2024-05-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b5ffb700 1 -- 172.21.15.161:0/3214253605 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a80044e0 con 0x7f44b8100b80 2024-05-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b8199030 con 0x7f44b8100b80 2024-05-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44b5ffb700 1 -- 172.21.15.161:0/3214253605 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8003dc0 con 0x7f44b8100b80 2024-05-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.270+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b819f810 con 0x7f44b8100b80 2024-05-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.271+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b8107da0 con 0x7f44b8100b80 2024-05-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.272+0000 7f44b5ffb700 1 -- 172.21.15.161:0/3214253605 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f44a8004020 con 0x7f44b8100b80 2024-05-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.273+0000 7f44b5ffb700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44a40a1120 0x7f44a40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:14.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.273+0000 7f44b5ffb700 1 -- 172.21.15.161:0/3214253605 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3389+0+0 (secure 0 0 0) 0x7f44a8014070 con 0x7f44b8100b80 2024-05-10T04:35:14.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.273+0000 7f44affff700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44a40a1120 0x7f44a40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:14.277 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.274+0000 7f44affff700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44a40a1120 0x7f44a40a3630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44a0005950 tx=0x7f44a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:35:14.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.278+0000 7f44b5ffb700 1 -- 172.21.15.161:0/3214253605 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f44a8096400 con 0x7f44b8100b80 2024-05-10T04:35:14.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T04:35:14.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T04:35:14.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:14.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:35:14.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi150 2024-05-10T04:35:14.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi150 to 8557M 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:13 smithi181 ceph-mon[47314]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:14.411 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:35:13 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-3[68766]: 2024-05-10T04:35:13.962+0000 7f6098fe7700 -1 osd.3 0 waiting for initial osdmap 2024-05-10T04:35:14.411 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:35:13 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-3[68766]: 2024-05-10T04:35:13.969+0000 7f6094d9e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:35:14.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: osdmap e24: 4 total, 3 up, 4 in 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi150 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi150 to 8557M 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:14.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:14.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:13 smithi150 ceph-mon[42404]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-10T04:35:14.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:14.478+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f44b810c530 con 0x7f44a40a1120 2024-05-10T04:35:15.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:14 smithi161 ceph-mon[46912]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:35:15.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:14 smithi161 ceph-mon[46912]: osdmap e25: 4 total, 3 up, 4 in 2024-05-10T04:35:15.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:14 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:15.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:14 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:15.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:14 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:15.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:14 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:15.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:14 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:15.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:14 smithi181 ceph-mon[47314]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:35:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:14 smithi181 ceph-mon[47314]: osdmap e25: 4 total, 3 up, 4 in 2024-05-10T04:35:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:14 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:14 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:14 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:14 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:14 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:15.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:14 smithi150 ceph-mon[42404]: from='osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:35:15.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:14 smithi150 ceph-mon[42404]: osdmap e25: 4 total, 3 up, 4 in 2024-05-10T04:35:15.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:14 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:15.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:14 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:15.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:14 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:15.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:14 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:15.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:14 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:16.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:15 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:35:16.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:15 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:35:16.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:15 smithi161 ceph-mon[46912]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:16.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:15 smithi161 ceph-mon[46912]: osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] boot 2024-05-10T04:35:16.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:15 smithi161 ceph-mon[46912]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T04:35:16.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:15 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:16.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:15 smithi161 ceph-mon[46912]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:16.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:15 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:35:16.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:15 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:35:16.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:15 smithi181 ceph-mon[47314]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:16.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:15 smithi181 ceph-mon[47314]: osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] boot 2024-05-10T04:35:16.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:15 smithi181 ceph-mon[47314]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T04:35:16.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:15 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:16.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:15 smithi181 ceph-mon[47314]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:16.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:15 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:35:16.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:15 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:35:16.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:15 smithi150 ceph-mon[42404]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:16.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:15 smithi150 ceph-mon[42404]: osd.3 [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] boot 2024-05-10T04:35:16.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:15 smithi150 ceph-mon[42404]: osdmap e26: 4 total, 4 up, 4 in 2024-05-10T04:35:16.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:15 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-10T04:35:16.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:15 smithi150 ceph-mon[42404]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:17.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:16 smithi161 ceph-mon[46912]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T04:35:17.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:16 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/2730893186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]: dispatch 2024-05-10T04:35:17.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:16 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]: dispatch 2024-05-10T04:35:17.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:16 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]': finished 2024-05-10T04:35:17.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:16 smithi161 ceph-mon[46912]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T04:35:17.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:17.338 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:16 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/815833129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:17.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:16 smithi181 ceph-mon[47314]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T04:35:17.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:16 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/2730893186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]: dispatch 2024-05-10T04:35:17.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:16 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]: dispatch 2024-05-10T04:35:17.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:16 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]': finished 2024-05-10T04:35:17.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:16 smithi181 ceph-mon[47314]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T04:35:17.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:17.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:16 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/815833129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:17.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:16 smithi150 ceph-mon[42404]: osdmap e27: 4 total, 4 up, 4 in 2024-05-10T04:35:17.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:16 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/2730893186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]: dispatch 2024-05-10T04:35:17.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:16 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]: dispatch 2024-05-10T04:35:17.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:16 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8507ef53-8650-4e6f-b8a2-482d3b632548"}]': finished 2024-05-10T04:35:17.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:16 smithi150 ceph-mon[42404]: osdmap e28: 5 total, 4 up, 5 in 2024-05-10T04:35:17.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:17.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:16 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/815833129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:18 smithi161 ceph-mon[46912]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T04:35:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:18 smithi161 ceph-mon[46912]: pgmap v82: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:18.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:18 smithi181 ceph-mon[47314]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T04:35:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:18 smithi181 ceph-mon[47314]: pgmap v82: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:18 smithi150 ceph-mon[42404]: osdmap e29: 5 total, 4 up, 5 in 2024-05-10T04:35:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:18 smithi150 ceph-mon[42404]: pgmap v82: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:19.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:19 smithi161 ceph-mon[46912]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T04:35:19.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:19 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:19.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:19 smithi181 ceph-mon[47314]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T04:35:19.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:19 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:19.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:19 smithi150 ceph-mon[42404]: osdmap e30: 5 total, 4 up, 5 in 2024-05-10T04:35:19.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:19 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:20.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:20 smithi161 ceph-mon[46912]: pgmap v84: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:20 smithi181 ceph-mon[47314]: pgmap v84: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:20.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:20 smithi150 ceph-mon[42404]: pgmap v84: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:22 smithi161 ceph-mon[46912]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-10T04:35:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:22 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T04:35:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:22 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:22.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:22 smithi181 ceph-mon[47314]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-10T04:35:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:22 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T04:35:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:22 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:22 smithi150 ceph-mon[42404]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-10T04:35:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-10T04:35:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:22 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:23.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:23 smithi161 ceph-mon[46912]: Deploying daemon osd.4 on smithi161 2024-05-10T04:35:23.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:23 smithi181 ceph-mon[47314]: Deploying daemon osd.4 on smithi161 2024-05-10T04:35:23.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:23 smithi150 ceph-mon[42404]: Deploying daemon osd.4 on smithi161 2024-05-10T04:35:24.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:24 smithi161 ceph-mon[46912]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-10T04:35:24.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:24 smithi181 ceph-mon[47314]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-10T04:35:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:24 smithi150 ceph-mon[42404]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-10T04:35:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:26 smithi161 ceph-mon[46912]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-10T04:35:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:26 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:26 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:26 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:26.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:26 smithi181 ceph-mon[47314]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-10T04:35:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:26 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:26 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:26 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:26 smithi150 ceph-mon[42404]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-10T04:35:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:27.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.733+0000 7f44b5ffb700 1 -- 172.21.15.161:0/3214253605 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f44b810c530 con 0x7f44a40a1120 2024-05-10T04:35:27.735 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 4 on host 'smithi161' 2024-05-10T04:35:27.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44a40a1120 msgr2=0x7f44a40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:27.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44a40a1120 0x7f44a40a3630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44a0005950 tx=0x7f44a00058e0 comp rx=0 tx=0).stop 2024-05-10T04:35:27.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8100b80 msgr2=0x7f44b8197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8100b80 0x7f44b8197100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f44a800e940 tx=0x7f44a800ed00 comp rx=0 tx=0).stop 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 shutdown_connections 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44a40a1120 0x7f44a40a3630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b8102650 0x7f44b81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b8100b80 0x7f44b8197100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 --2- 172.21.15.161:0/3214253605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b8101cb0 0x7f44b8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 >> 172.21.15.161:0/3214253605 conn(0x7f44b80fc1c0 msgr2=0x7f44b8104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 shutdown_connections 2024-05-10T04:35:27.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:35:27.736+0000 7f44be8bd700 1 -- 172.21.15.161:0/3214253605 wait complete. 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:27.937 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:28.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:28.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:35:28.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:28.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:28.178 DEBUG:teuthology.orchestra.run.smithi161:osd.4> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.4.service 2024-05-10T04:35:28.180 INFO:tasks.cephadm:Deploying osd.5 on smithi181 with /dev/vg_nvme/lv_3... 2024-05-10T04:35:28.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-10T04:35:28.210 INFO:journalctl@ceph.osd.4.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:35:28.413 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:35:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-10T04:35:29.731 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-10T04:35:29.732 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T04:35:29.732 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T04:35:29.732 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0170634 s, 615 MB/s 2024-05-10T04:35:29.732 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T04:35:29.798 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:35:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-4[59826]: 2024-05-10T04:35:29.544+0000 7f7938b8d7c0 -1 osd.4 0 log_to_monitors true 2024-05-10T04:35:30.089 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-05-10T04:35:30.308 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:35:30.327 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-10T04:35:30.328 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T04:35:30.328 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T04:35:30.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-10T04:35:30.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T04:35:30.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T04:35:30.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:30.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:30.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:30.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:30.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:30.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-10T04:35:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:30.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3202796869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 msgr2=0x7fb698108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3202796869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb698108f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb69000b0a0 tx=0x7fb69000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3202796869 shutdown_connections 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3202796869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb698108f00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3202796869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb698100b80 0x7fb698100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3202796869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698101cb0 0x7fb6981020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3202796869 >> 172.21.15.181:0/3202796869 conn(0x7fb6980fc1c0 msgr2=0x7fb6980fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3202796869 shutdown_connections 2024-05-10T04:35:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.304+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3202796869 wait complete. 2024-05-10T04:35:31.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 Processor -- start 2024-05-10T04:35:31.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 -- start start 2024-05-10T04:35:31.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb698100b80 0x7fb69819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:31.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698101cb0 0x7fb69819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb69819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69819bd80 con 0x7fb698100b80 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69819bec0 con 0x7fb698101cb0 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69f3f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69819c080 con 0x7fb698102650 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb69819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb69819b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59406/0 (socket says 172.21.15.181:59406) 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 -- 172.21.15.181:0/3992542898 learned_addr learned my addr 172.21.15.181:0/3992542898 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d18d700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb698100b80 0x7fb69819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69c98c700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698101cb0 0x7fb69819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 -- 172.21.15.181:0/3992542898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698101cb0 msgr2=0x7fb69819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698101cb0 0x7fb69819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 -- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb698100b80 msgr2=0x7fb69819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb698100b80 0x7fb69819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 -- 172.21.15.181:0/3992542898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690009d00 con 0x7fb698102650 2024-05-10T04:35:31.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69c98c700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698101cb0 0x7fb69819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:35:31.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.305+0000 7fb69d98e700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb69819b720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb690006070 tx=0x7fb690004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:35:31.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.306+0000 7fb68e7fc700 1 -- 172.21.15.181:0/3992542898 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69000e040 con 0x7fb698102650 2024-05-10T04:35:31.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.306+0000 7fb68e7fc700 1 -- 172.21.15.181:0/3992542898 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb69000cec0 con 0x7fb698102650 2024-05-10T04:35:31.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.306+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69819c300 con 0x7fb698102650 2024-05-10T04:35:31.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.306+0000 7fb68e7fc700 1 -- 172.21.15.181:0/3992542898 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69001de80 con 0x7fb698102650 2024-05-10T04:35:31.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.306+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6981a2b40 con 0x7fb698102650 2024-05-10T04:35:31.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.307+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6980ffe80 con 0x7fb698102650 2024-05-10T04:35:31.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.308+0000 7fb68e7fc700 1 -- 172.21.15.181:0/3992542898 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb690003a00 con 0x7fb698102650 2024-05-10T04:35:31.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.308+0000 7fb68e7fc700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6840a1120 0x7fb6840a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:31.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.308+0000 7fb68e7fc700 1 -- 172.21.15.181:0/3992542898 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3805+0+0 (secure 0 0 0) 0x7fb6900cd5f0 con 0x7fb698102650 2024-05-10T04:35:31.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.309+0000 7fb69d18d700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6840a1120 0x7fb6840a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:31.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.309+0000 7fb69d18d700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6840a1120 0x7fb6840a3630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb698101bb0 tx=0x7fb69400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:35:31.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.312+0000 7fb68e7fc700 1 -- 172.21.15.181:0/3992542898 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb690097fb0 con 0x7fb698102650 2024-05-10T04:35:31.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:31 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi161 2024-05-10T04:35:31.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:31 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi161 to 8557M 2024-05-10T04:35:31.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:31 smithi181 ceph-mon[47314]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T04:35:31.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:31 smithi181 ceph-mon[47314]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T04:35:31.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:31.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:31 smithi181 ceph-mon[47314]: from='osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:35:31.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:31 smithi181 ceph-mon[47314]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:35:31.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:31.515+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6980fe230 con 0x7fb6840a1120 2024-05-10T04:35:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:31 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi161 2024-05-10T04:35:31.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:31 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi161 to 8557M 2024-05-10T04:35:31.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:31 smithi150 ceph-mon[42404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T04:35:31.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:31 smithi150 ceph-mon[42404]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T04:35:31.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:31.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:31 smithi150 ceph-mon[42404]: from='osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:35:31.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:31 smithi150 ceph-mon[42404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:35:31.835 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:35:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-4[59826]: 2024-05-10T04:35:31.339+0000 7f792e1f1700 -1 osd.4 0 waiting for initial osdmap 2024-05-10T04:35:31.835 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:35:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-4[59826]: 2024-05-10T04:35:31.347+0000 7f7929fa8700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:35:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:31 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi161 2024-05-10T04:35:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:31 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi161 to 8557M 2024-05-10T04:35:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:31 smithi161 ceph-mon[46912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-10T04:35:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:31 smithi161 ceph-mon[46912]: osdmap e31: 5 total, 4 up, 5 in 2024-05-10T04:35:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:31 smithi161 ceph-mon[46912]: from='osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:35:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:31 smithi161 ceph-mon[46912]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: osdmap e32: 5 total, 4 up, 5 in 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:32.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:32 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:35:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: osdmap e32: 5 total, 4 up, 5 in 2024-05-10T04:35:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:32.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:32.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:32.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:35:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: osdmap e32: 5 total, 4 up, 5 in 2024-05-10T04:35:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:32 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:33 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:35:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:33 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:35:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:33 smithi181 ceph-mon[47314]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:33 smithi181 ceph-mon[47314]: osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] boot 2024-05-10T04:35:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:33 smithi181 ceph-mon[47314]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T04:35:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:33 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:33 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:35:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:33 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:35:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:33 smithi150 ceph-mon[42404]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:33 smithi150 ceph-mon[42404]: osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] boot 2024-05-10T04:35:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:33 smithi150 ceph-mon[42404]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T04:35:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:33 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:33.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:33 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:35:33.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:33 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:35:33.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:33 smithi161 ceph-mon[46912]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:33.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:33 smithi161 ceph-mon[46912]: osd.4 [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] boot 2024-05-10T04:35:33.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:33 smithi161 ceph-mon[46912]: osdmap e33: 5 total, 5 up, 5 in 2024-05-10T04:35:33.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:33 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-10T04:35:34.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: osdmap e34: 5 total, 5 up, 5 in 2024-05-10T04:35:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/2666210126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]: dispatch 2024-05-10T04:35:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]: dispatch 2024-05-10T04:35:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]': finished 2024-05-10T04:35:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T04:35:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:34 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/3707377664' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: osdmap e34: 5 total, 5 up, 5 in 2024-05-10T04:35:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/2666210126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]: dispatch 2024-05-10T04:35:34.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]: dispatch 2024-05-10T04:35:34.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]': finished 2024-05-10T04:35:34.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T04:35:34.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:34.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/3707377664' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-10T04:35:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: osdmap e34: 5 total, 5 up, 5 in 2024-05-10T04:35:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/2666210126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]: dispatch 2024-05-10T04:35:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]: dispatch 2024-05-10T04:35:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4632c68c-e68e-44c9-9f14-8f5a2c836c06"}]': finished 2024-05-10T04:35:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: osdmap e35: 6 total, 5 up, 6 in 2024-05-10T04:35:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:34 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/3707377664' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:36.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:36 smithi181 ceph-mon[47314]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:36.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:36 smithi150 ceph-mon[42404]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:36.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:36 smithi161 ceph-mon[46912]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:38.549 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:38 smithi181 ceph-mon[47314]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:38.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:38 smithi150 ceph-mon[42404]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:38.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:38 smithi161 ceph-mon[46912]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:39.613 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T04:35:39.613 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T04:35:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-10T04:35:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:40.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:40 smithi181 ceph-mon[47314]: Deploying daemon osd.5 on smithi181 2024-05-10T04:35:40.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:40 smithi181 ceph-mon[47314]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:40.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:40 smithi150 ceph-mon[42404]: Deploying daemon osd.5 on smithi181 2024-05-10T04:35:40.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:40 smithi150 ceph-mon[42404]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:40 smithi161 ceph-mon[46912]: Deploying daemon osd.5 on smithi181 2024-05-10T04:35:40.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:40 smithi161 ceph-mon[46912]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:42 smithi181 ceph-mon[47314]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:42.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:42 smithi150 ceph-mon[42404]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:42 smithi161 ceph-mon[46912]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:44.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:44.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:44.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:44.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:44 smithi161 ceph-mon[46912]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:44 smithi181 ceph-mon[47314]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:44.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:44.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:44.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:44.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:44 smithi150 ceph-mon[42404]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:45.520 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.519+0000 7fb68e7fc700 1 -- 172.21.15.181:0/3992542898 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb6980fe230 con 0x7fb6840a1120 2024-05-10T04:35:45.521 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 5 on host 'smithi181' 2024-05-10T04:35:45.522 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6840a1120 msgr2=0x7fb6840a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:45.522 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6840a1120 0x7fb6840a3630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb698101bb0 tx=0x7fb69400b410 comp rx=0 tx=0).stop 2024-05-10T04:35:45.522 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 msgr2=0x7fb69819b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:45.522 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb69819b720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb690006070 tx=0x7fb690004c80 comp rx=0 tx=0).stop 2024-05-10T04:35:45.522 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 shutdown_connections 2024-05-10T04:35:45.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6840a1120 0x7fb6840a3630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:45.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb698102650 0x7fb69819b720 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:45.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb698100b80 0x7fb69819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:45.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 --2- 172.21.15.181:0/3992542898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb698101cb0 0x7fb69819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:45.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 >> 172.21.15.181:0/3992542898 conn(0x7fb6980fc1c0 msgr2=0x7fb6980fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:35:45.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 shutdown_connections 2024-05-10T04:35:45.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:35:45.521+0000 7fb69f3f1700 1 -- 172.21.15.181:0/3992542898 wait complete. 2024-05-10T04:35:45.938 DEBUG:teuthology.orchestra.run.smithi181:osd.5> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.5.service 2024-05-10T04:35:45.940 INFO:tasks.cephadm:Deploying osd.6 on smithi150 with /dev/vg_nvme/lv_2... 2024-05-10T04:35:45.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T04:35:45.966 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.966 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.966 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:45.966 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:45.966 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.967 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:45.967 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.967 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.967 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:45.973 INFO:journalctl@ceph.osd.5.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:45.973 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:46.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:46.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:46.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:46.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:46.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:46.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:46.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:46.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:46.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:35:46.173 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:35:47.373 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T04:35:47.373 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T04:35:47.373 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T04:35:47.373 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T04:35:47.373 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112188 s, 935 MB/s 2024-05-10T04:35:47.374 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T04:35:47.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:47 smithi181 ceph-mon[47314]: from='osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:35:47.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:47 smithi181 ceph-mon[47314]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:35:47.405 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:35:47 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:35:47.018+0000 7f86d2c8a7c0 -1 osd.5 0 log_to_monitors true 2024-05-10T04:35:47.410 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:47 smithi150 ceph-mon[42404]: from='osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:35:47.410 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:47 smithi150 ceph-mon[42404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:35:47.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:47 smithi161 ceph-mon[46912]: from='osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:35:47.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:47 smithi161 ceph-mon[46912]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-10T04:35:47.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-05-10T04:35:48.010 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:35:48.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:48.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-10T04:35:48.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: osdmap e36: 6 total, 5 up, 6 in 2024-05-10T04:35:48.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:35:48.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:35:48.904 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:35:48 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:35:48.605+0000 7f86c82ee700 -1 osd.5 0 waiting for initial osdmap 2024-05-10T04:35:48.904 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:35:48 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:35:48.615+0000 7f86c009d700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:35:48.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 -- 172.21.15.150:0/2734029579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c104fe0 msgr2=0x7f029c1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:48.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 --2- 172.21.15.150:0/2734029579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c104fe0 0x7f029c1053f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0290009990 tx=0x7f0290009ca0 comp rx=0 tx=0).stop 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 -- 172.21.15.150:0/2734029579 shutdown_connections 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 --2- 172.21.15.150:0/2734029579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f029c105980 0x7f029c10c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 --2- 172.21.15.150:0/2734029579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c103eb0 0x7f029c1042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 --2- 172.21.15.150:0/2734029579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c104fe0 0x7f029c1053f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 -- 172.21.15.150:0/2734029579 >> 172.21.15.150:0/2734029579 conn(0x7f029c0ff4f0 msgr2=0x7f029c1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 -- 172.21.15.150:0/2734029579 shutdown_connections 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 -- 172.21.15.150:0/2734029579 wait complete. 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.989+0000 7f02a2a14700 1 Processor -- start 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a2a14700 1 -- start start 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a2a14700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c103eb0 0x7f029c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a2a14700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c104fe0 0x7f029c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a2a14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f029c105980 0x7f029c19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:48.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a2a14700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19bc50 con 0x7f029c103eb0 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a2a14700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19bd90 con 0x7f029c104fe0 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a2a14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19bf50 con 0x7f029c105980 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c103eb0 0x7f029c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c103eb0 0x7f029c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37508/0 (socket says 172.21.15.150:37508) 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 -- 172.21.15.150:0/1426423249 learned_addr learned my addr 172.21.15.150:0/1426423249 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029b7fe700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c104fe0 0x7f029c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a0fb1700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f029c105980 0x7f029c19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 -- 172.21.15.150:0/1426423249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f029c105980 msgr2=0x7f029c19b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f029c105980 0x7f029c19b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 -- 172.21.15.150:0/1426423249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c104fe0 msgr2=0x7f029c19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c104fe0 0x7f029c19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f029bfff700 1 -- 172.21.15.150:0/1426423249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900095f0 con 0x7f029c103eb0 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.990+0000 7f02a0fb1700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f029c105980 0x7f029c19b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:35:48.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.991+0000 7f029bfff700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c103eb0 0x7f029c19a300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d950 tx=0x7f028c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:35:48.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.991+0000 7f02997fa700 1 -- 172.21.15.150:0/1426423249 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c004220 con 0x7f029c103eb0 2024-05-10T04:35:48.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.991+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c19c230 con 0x7f029c103eb0 2024-05-10T04:35:48.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.991+0000 7f02997fa700 1 -- 172.21.15.150:0/1426423249 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028c004380 con 0x7f029c103eb0 2024-05-10T04:35:48.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.991+0000 7f02997fa700 1 -- 172.21.15.150:0/1426423249 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c003bf0 con 0x7f029c103eb0 2024-05-10T04:35:48.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.991+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c1a2a10 con 0x7f029c103eb0 2024-05-10T04:35:48.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.992+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c10b0d0 con 0x7f029c103eb0 2024-05-10T04:35:48.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.993+0000 7f02997fa700 1 -- 172.21.15.150:0/1426423249 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f028c003d50 con 0x7f029c103eb0 2024-05-10T04:35:48.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.994+0000 7f02997fa700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02840a1120 0x7f02840a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:35:48.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.994+0000 7f029b7fe700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02840a1120 0x7f02840a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:35:48.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.994+0000 7f02997fa700 1 -- 172.21.15.150:0/1426423249 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4253+0+0 (secure 0 0 0) 0x7f028c0cbbf0 con 0x7f029c103eb0 2024-05-10T04:35:48.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.994+0000 7f029b7fe700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02840a1120 0x7f02840a3630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0290003730 tx=0x7f029001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:35:48.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:48.998+0000 7f02997fa700 1 -- 172.21.15.150:0/1426423249 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f028c096300 con 0x7f029c103eb0 2024-05-10T04:35:49.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:35:49.202+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f029c10f8e0 con 0x7f02840a1120 2024-05-10T04:35:49.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:49 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi181 2024-05-10T04:35:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:49 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi181 to 8556M 2024-05-10T04:35:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:49 smithi181 ceph-mon[47314]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:35:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:49 smithi181 ceph-mon[47314]: osdmap e37: 6 total, 5 up, 6 in 2024-05-10T04:35:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:49 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:49 smithi181 ceph-mon[47314]: from='osd.5 ' entity='osd.5' 2024-05-10T04:35:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:49 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi181 2024-05-10T04:35:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:49 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi181 to 8556M 2024-05-10T04:35:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:49 smithi150 ceph-mon[42404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:35:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:49 smithi150 ceph-mon[42404]: osdmap e37: 6 total, 5 up, 6 in 2024-05-10T04:35:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:49 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:49 smithi150 ceph-mon[42404]: from='osd.5 ' entity='osd.5' 2024-05-10T04:35:49.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:49 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi181 2024-05-10T04:35:49.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:49 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi181 to 8556M 2024-05-10T04:35:49.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:49 smithi161 ceph-mon[46912]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:35:49.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:49 smithi161 ceph-mon[46912]: osdmap e37: 6 total, 5 up, 6 in 2024-05-10T04:35:49.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:49 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:49.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:49 smithi161 ceph-mon[46912]: from='osd.5 ' entity='osd.5' 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] boot 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:50 smithi181 ceph-mon[47314]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:35:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:35:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] boot 2024-05-10T04:35:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T04:35:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:50.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:50.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:50 smithi150 ceph-mon[42404]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:50.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:35:50.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: osd.5 [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] boot 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: osdmap e38: 6 total, 6 up, 6 in 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:50.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:50 smithi161 ceph-mon[46912]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-05-10T04:35:51.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:51 smithi150 ceph-mon[42404]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T04:35:51.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:51 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/858640236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89f0a895-52f5-4cb3-97df-71df296938c8"}]: dispatch 2024-05-10T04:35:51.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:51 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/858640236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89f0a895-52f5-4cb3-97df-71df296938c8"}]': finished 2024-05-10T04:35:51.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:51 smithi150 ceph-mon[42404]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T04:35:51.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:51.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:51 smithi161 ceph-mon[46912]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T04:35:51.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:51 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/858640236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89f0a895-52f5-4cb3-97df-71df296938c8"}]: dispatch 2024-05-10T04:35:51.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:51 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/858640236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89f0a895-52f5-4cb3-97df-71df296938c8"}]': finished 2024-05-10T04:35:51.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:51 smithi161 ceph-mon[46912]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T04:35:51.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:51.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:51 smithi181 ceph-mon[47314]: osdmap e39: 6 total, 6 up, 6 in 2024-05-10T04:35:51.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:51 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/858640236' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89f0a895-52f5-4cb3-97df-71df296938c8"}]: dispatch 2024-05-10T04:35:51.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:51 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/858640236' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89f0a895-52f5-4cb3-97df-71df296938c8"}]': finished 2024-05-10T04:35:51.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:51 smithi181 ceph-mon[47314]: osdmap e40: 7 total, 6 up, 7 in 2024-05-10T04:35:51.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:52.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:52 smithi161 ceph-mon[46912]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:52.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:52 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1859742988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:52.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:52 smithi161 ceph-mon[46912]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T04:35:52.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:52 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:52.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:52 smithi181 ceph-mon[47314]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:52.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:52 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1859742988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:52.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:52 smithi181 ceph-mon[47314]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T04:35:52.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:52 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:52 smithi150 ceph-mon[42404]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:52 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1859742988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:35:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:52 smithi150 ceph-mon[42404]: osdmap e41: 7 total, 6 up, 7 in 2024-05-10T04:35:52.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:52 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:54.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:54 smithi161 ceph-mon[46912]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T04:35:54.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:54.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:54 smithi161 ceph-mon[46912]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:54.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:54 smithi181 ceph-mon[47314]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T04:35:54.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:54.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:54 smithi181 ceph-mon[47314]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:54.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:54 smithi150 ceph-mon[42404]: osdmap e42: 7 total, 6 up, 7 in 2024-05-10T04:35:54.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:35:54.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:54 smithi150 ceph-mon[42404]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:56.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:56 smithi150 ceph-mon[42404]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:56.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:56 smithi161 ceph-mon[46912]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:56 smithi181 ceph-mon[47314]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-10T04:35:57.541 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T04:35:57.541 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T04:35:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:57.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-10T04:35:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:35:58.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:58 smithi161 ceph-mon[46912]: Deploying daemon osd.6 on smithi150 2024-05-10T04:35:58.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:35:58 smithi161 ceph-mon[46912]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-10T04:35:58.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:58 smithi181 ceph-mon[47314]: Deploying daemon osd.6 on smithi150 2024-05-10T04:35:58.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:35:58 smithi181 ceph-mon[47314]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-10T04:35:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:58 smithi150 ceph-mon[42404]: Deploying daemon osd.6 on smithi150 2024-05-10T04:35:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:35:58 smithi150 ceph-mon[42404]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-10T04:36:00.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:00 smithi181 ceph-mon[47314]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:36:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:00 smithi150 ceph-mon[42404]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:36:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:00 smithi161 ceph-mon[46912]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:36:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:01 smithi150 ceph-mon[42404]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T04:36:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:01 smithi161 ceph-mon[46912]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T04:36:01.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:01.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:01 smithi181 ceph-mon[47314]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-10T04:36:02.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.746+0000 7f02997fa700 1 -- 172.21.15.150:0/1426423249 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f029c10f8e0 con 0x7f02840a1120 2024-05-10T04:36:02.748 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 6 on host 'smithi150' 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02840a1120 msgr2=0x7f02840a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02840a1120 0x7f02840a3630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0290003730 tx=0x7f029001a040 comp rx=0 tx=0).stop 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c103eb0 msgr2=0x7f029c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c103eb0 0x7f029c19a300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d950 tx=0x7f028c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 shutdown_connections 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02840a1120 0x7f02840a3630 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f029c105980 0x7f029c19b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f029c103eb0 0x7f029c19a300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 --2- 172.21.15.150:0/1426423249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f029c104fe0 0x7f029c19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.748+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 >> 172.21.15.150:0/1426423249 conn(0x7f029c0ff4f0 msgr2=0x7f029c108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:02.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.749+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 shutdown_connections 2024-05-10T04:36:02.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:02.749+0000 7f02a2a14700 1 -- 172.21.15.150:0/1426423249 wait complete. 2024-05-10T04:36:03.049 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.049 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.049 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:03.050 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:03.050 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.050 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.050 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.050 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:03.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:03.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:03.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:03.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:03.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:03.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:03.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:03.304 DEBUG:teuthology.orchestra.run.smithi150:osd.6> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.6.service 2024-05-10T04:36:03.306 INFO:tasks.cephadm:Deploying osd.7 on smithi161 with /dev/vg_nvme/lv_2... 2024-05-10T04:36:03.307 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T04:36:03.337 INFO:journalctl@ceph.osd.6.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:36:03.540 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:36:03.940 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:03 smithi161 ceph-mon[46912]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T04:36:03.967 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:03 smithi150 ceph-mon[42404]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T04:36:04.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:03 smithi181 ceph-mon[47314]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T04:36:04.775 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T04:36:04.775 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T04:36:04.775 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T04:36:04.776 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T04:36:04.776 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.016627 s, 631 MB/s 2024-05-10T04:36:04.776 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T04:36:04.812 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:36:04 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:36:04.517+0000 7f06fefe57c0 -1 osd.6 0 log_to_monitors true 2024-05-10T04:36:05.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:04 smithi161 ceph-mon[46912]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T04:36:05.099 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:04 smithi150 ceph-mon[42404]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T04:36:05.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:04 smithi181 ceph-mon[47314]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-10T04:36:05.211 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-05-10T04:36:05.436 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:36:06.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi150 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi150 to 5704M 2024-05-10T04:36:06.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:06.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:06.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi150 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi150 to 5704M 2024-05-10T04:36:06.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:06.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:06.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:36:05 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:36:05.837+0000 7f06f4649700 -1 osd.6 0 waiting for initial osdmap 2024-05-10T04:36:06.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:36:05 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:36:05.849+0000 7f06f0400700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:36:06.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: osdmap e43: 7 total, 6 up, 7 in 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi150 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi150 to 5704M 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:06.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:06.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 -- 172.21.15.161:0/1221737080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4101cb0 msgr2=0x7fdda41020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:06.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 --2- 172.21.15.161:0/1221737080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4101cb0 0x7fdda41020c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009990 tx=0x7fdd94009ca0 comp rx=0 tx=0).stop 2024-05-10T04:36:06.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 -- 172.21.15.161:0/1221737080 shutdown_connections 2024-05-10T04:36:06.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 --2- 172.21.15.161:0/1221737080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda4102650 0x7fdda4108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:06.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 --2- 172.21.15.161:0/1221737080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda4100b80 0x7fdda4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:06.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 --2- 172.21.15.161:0/1221737080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4101cb0 0x7fdda41020c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:06.390 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 -- 172.21.15.161:0/1221737080 >> 172.21.15.161:0/1221737080 conn(0x7fdda40fc1c0 msgr2=0x7fdda40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 -- 172.21.15.161:0/1221737080 shutdown_connections 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.386+0000 7fddac6ae700 1 -- 172.21.15.161:0/1221737080 wait complete. 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 Processor -- start 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 -- start start 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda4100b80 0x7fdda41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda4101cb0 0x7fdda41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4102650 0x7fdda4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4199aa0 con 0x7fdda4100b80 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4199be0 con 0x7fdda4102650 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddac6ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda4199da0 con 0x7fdda4101cb0 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaac4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4102650 0x7fdda4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaac4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4102650 0x7fdda4199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44520/0 (socket says 172.21.15.161:44520) 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaac4b700 1 -- 172.21.15.161:0/939281759 learned_addr learned my addr 172.21.15.161:0/939281759 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T04:36:06.391 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaa44a700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda4100b80 0x7fdda41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fdda9c49700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda4101cb0 0x7fdda41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaac4b700 1 -- 172.21.15.161:0/939281759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda4101cb0 msgr2=0x7fdda41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaac4b700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda4101cb0 0x7fdda41986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaac4b700 1 -- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda4100b80 msgr2=0x7fdda41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.387+0000 7fddaac4b700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda4100b80 0x7fdda41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.388+0000 7fddaac4b700 1 -- 172.21.15.161:0/939281759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940095f0 con 0x7fdda4102650 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.388+0000 7fddaac4b700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4102650 0x7fdda4199440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c007ee0 tx=0x7fdd9c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.388+0000 7fdd9b7fe700 1 -- 172.21.15.161:0/939281759 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c017070 con 0x7fdda4102650 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.388+0000 7fdd9b7fe700 1 -- 172.21.15.161:0/939281759 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd9c00f040 con 0x7fdda4102650 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.388+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda419a080 con 0x7fdda4102650 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.388+0000 7fdd9b7fe700 1 -- 172.21.15.161:0/939281759 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c005740 con 0x7fdda4102650 2024-05-10T04:36:06.392 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.388+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda41a0860 con 0x7fdda4102650 2024-05-10T04:36:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.389+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda40ffe80 con 0x7fdda4102650 2024-05-10T04:36:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.390+0000 7fdd9b7fe700 1 -- 172.21.15.161:0/939281759 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdd9c00dc60 con 0x7fdda4102650 2024-05-10T04:36:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.391+0000 7fdd9b7fe700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdd900a11a0 0x7fdd900a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:06.393 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.391+0000 7fdd9b7fe700 1 -- 172.21.15.161:0/939281759 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4685+0+0 (secure 0 0 0) 0x7fdd9c0cb280 con 0x7fdda4102650 2024-05-10T04:36:06.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.391+0000 7fddaa44a700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdd900a11a0 0x7fdd900a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:06.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.391+0000 7fddaa44a700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdd900a11a0 0x7fdd900a36b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdda4101bb0 tx=0x7fdda0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:06.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.395+0000 7fdd9b7fe700 1 -- 172.21.15.161:0/939281759 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdd9c0957a0 con 0x7fdda4102650 2024-05-10T04:36:06.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:06.594+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdda4069430 con 0x7fdd900a11a0 2024-05-10T04:36:07.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:06 smithi181 ceph-mon[47314]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:36:07.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:06 smithi181 ceph-mon[47314]: osdmap e44: 7 total, 6 up, 7 in 2024-05-10T04:36:07.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:07.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:07.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:07.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:07.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:07.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:06 smithi150 ceph-mon[42404]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:36:07.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:06 smithi150 ceph-mon[42404]: osdmap e44: 7 total, 6 up, 7 in 2024-05-10T04:36:07.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:07.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:07.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:07.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:07.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:07.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:06 smithi161 ceph-mon[46912]: from='osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:36:07.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:06 smithi161 ceph-mon[46912]: osdmap e44: 7 total, 6 up, 7 in 2024-05-10T04:36:07.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:07.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:07.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:07.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:07.256 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:08.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:07 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:36:08.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:07 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:36:08.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:07 smithi181 ceph-mon[47314]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:08.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:07 smithi181 ceph-mon[47314]: osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] boot 2024-05-10T04:36:08.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:07 smithi181 ceph-mon[47314]: osdmap e45: 7 total, 7 up, 7 in 2024-05-10T04:36:08.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:08.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:07 smithi181 ceph-mon[47314]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:08.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:07 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:36:08.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:07 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:36:08.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:07 smithi150 ceph-mon[42404]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:08.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:07 smithi150 ceph-mon[42404]: osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] boot 2024-05-10T04:36:08.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:07 smithi150 ceph-mon[42404]: osdmap e45: 7 total, 7 up, 7 in 2024-05-10T04:36:08.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:08.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:07 smithi150 ceph-mon[42404]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:08.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:07 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:36:08.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:07 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:36:08.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:07 smithi161 ceph-mon[46912]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:08.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:07 smithi161 ceph-mon[46912]: osd.6 [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] boot 2024-05-10T04:36:08.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:07 smithi161 ceph-mon[46912]: osdmap e45: 7 total, 7 up, 7 in 2024-05-10T04:36:08.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-10T04:36:08.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:07 smithi161 ceph-mon[46912]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:09.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:08 smithi181 ceph-mon[47314]: osdmap e46: 7 total, 7 up, 7 in 2024-05-10T04:36:09.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:08 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/71320969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]: dispatch 2024-05-10T04:36:09.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:08 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]: dispatch 2024-05-10T04:36:09.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:08 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]': finished 2024-05-10T04:36:09.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:08 smithi181 ceph-mon[47314]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T04:36:09.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:09.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:08 smithi150 ceph-mon[42404]: osdmap e46: 7 total, 7 up, 7 in 2024-05-10T04:36:09.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:08 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/71320969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]: dispatch 2024-05-10T04:36:09.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:08 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]: dispatch 2024-05-10T04:36:09.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:08 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]': finished 2024-05-10T04:36:09.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:08 smithi150 ceph-mon[42404]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T04:36:09.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:09.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:08 smithi161 ceph-mon[46912]: osdmap e46: 7 total, 7 up, 7 in 2024-05-10T04:36:09.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:08 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/71320969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]: dispatch 2024-05-10T04:36:09.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:08 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]: dispatch 2024-05-10T04:36:09.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:08 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09b5c539-71b8-42c8-97c4-3a3de4864a5a"}]': finished 2024-05-10T04:36:09.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:08 smithi161 ceph-mon[46912]: osdmap e47: 8 total, 7 up, 8 in 2024-05-10T04:36:09.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:10.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:09 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/2405901317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:10.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:09 smithi181 ceph-mon[47314]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:10.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:09 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/2405901317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:10.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:09 smithi150 ceph-mon[42404]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:10.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:09 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/2405901317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:10.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:09 smithi161 ceph-mon[46912]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:12.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:12 smithi161 ceph-mon[46912]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:12.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:12 smithi181 ceph-mon[47314]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:12 smithi150 ceph-mon[42404]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:14.611 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:14 smithi161 ceph-mon[46912]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:14.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:14 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T04:36:14.612 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:14 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:14.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:14 smithi181 ceph-mon[47314]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:14.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:14 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T04:36:14.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:14 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:14 smithi150 ceph-mon[42404]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:14 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-10T04:36:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:14 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:15.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:15 smithi181 ceph-mon[47314]: Deploying daemon osd.7 on smithi161 2024-05-10T04:36:15.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:15 smithi150 ceph-mon[42404]: Deploying daemon osd.7 on smithi161 2024-05-10T04:36:15.687 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:15 smithi161 ceph-mon[46912]: Deploying daemon osd.7 on smithi161 2024-05-10T04:36:16.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:16 smithi161 ceph-mon[46912]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:16.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:16 smithi181 ceph-mon[47314]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:16.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:16 smithi150 ceph-mon[42404]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:18.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:18 smithi161 ceph-mon[46912]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:18.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:18.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:18 smithi181 ceph-mon[47314]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:18 smithi150 ceph-mon[42404]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:19.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.995+0000 7fdd9b7fe700 1 -- 172.21.15.161:0/939281759 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdda4069430 con 0x7fdd900a11a0 2024-05-10T04:36:19.996 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 7 on host 'smithi161' 2024-05-10T04:36:19.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdd900a11a0 msgr2=0x7fdd900a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdd900a11a0 0x7fdd900a36b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdda4101bb0 tx=0x7fdda0007400 comp rx=0 tx=0).stop 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4102650 msgr2=0x7fdda4199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4102650 0x7fdda4199440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c007ee0 tx=0x7fdd9c00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 shutdown_connections 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdda4102650 0x7fdda4199440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdda4100b80 0x7fdda41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdd900a11a0 0x7fdd900a36b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 --2- 172.21.15.161:0/939281759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdda4101cb0 0x7fdda41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 >> 172.21.15.161:0/939281759 conn(0x7fdda40fc1c0 msgr2=0x7fdda4068d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 shutdown_connections 2024-05-10T04:36:19.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:19.997+0000 7fddac6ae700 1 -- 172.21.15.161:0/939281759 wait complete. 2024-05-10T04:36:20.372 DEBUG:teuthology.orchestra.run.smithi161:osd.7> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.7.service 2024-05-10T04:36:20.375 INFO:tasks.cephadm:Deploying osd.8 on smithi181 with /dev/vg_nvme/lv_2... 2024-05-10T04:36:20.375 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-10T04:36:20.403 INFO:journalctl@ceph.osd.7.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:36:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.602 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.603 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:20.607 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:36:20.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:20.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:21.976 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-10T04:36:21.976 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-10T04:36:21.976 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T04:36:21.976 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T04:36:21.976 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112692 s, 930 MB/s 2024-05-10T04:36:21.976 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T04:36:22.213 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:36:21 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:36:21.880+0000 7fda0f46c7c0 -1 osd.7 0 log_to_monitors true 2024-05-10T04:36:22.374 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-05-10T04:36:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:22 smithi161 ceph-mon[46912]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:22.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:22 smithi161 ceph-mon[46912]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:36:22.595 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:36:22.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:22 smithi181 ceph-mon[47314]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:22 smithi181 ceph-mon[47314]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:36:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:22 smithi150 ceph-mon[42404]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:22 smithi150 ceph-mon[42404]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-10T04:36:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.603+0000 7ff7884a5700 1 -- 172.21.15.181:0/1367909074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 msgr2=0x7ff780102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 --2- 172.21.15.181:0/1367909074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780102160 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff774009990 tx=0x7ff774009ca0 comp rx=0 tx=0).stop 2024-05-10T04:36:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 -- 172.21.15.181:0/1367909074 shutdown_connections 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 --2- 172.21.15.181:0/1367909074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff780103820 0x7ff78010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 --2- 172.21.15.181:0/1367909074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780102160 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 --2- 172.21.15.181:0/1367909074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff780102e80 0x7ff780103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 -- 172.21.15.181:0/1367909074 >> 172.21.15.181:0/1367909074 conn(0x7ff7800fd390 msgr2=0x7ff7800ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 -- 172.21.15.181:0/1367909074 shutdown_connections 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 -- 172.21.15.181:0/1367909074 wait complete. 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 Processor -- start 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.604+0000 7ff7884a5700 1 -- start start 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff780102e80 0x7ff7801986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff780103820 0x7ff780199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff780199a40 con 0x7ff780102e80 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff780199b80 con 0x7ff780103820 2024-05-10T04:36:23.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff780199d40 con 0x7ff780101d50 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51656/0 (socket says 172.21.15.181:51656) 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 -- 172.21.15.181:0/79942934 learned_addr learned my addr 172.21.15.181:0/79942934 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786a42700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff780103820 0x7ff780199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff785a40700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff780102e80 0x7ff7801986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 -- 172.21.15.181:0/79942934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff780103820 msgr2=0x7ff780199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff780103820 0x7ff780199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 -- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff780102e80 msgr2=0x7ff7801986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff780102e80 0x7ff7801986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 -- 172.21.15.181:0/79942934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740095f0 con 0x7ff780101d50 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff786241700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780198180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098e0 tx=0x7ff774009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7737fe700 1 -- 172.21.15.181:0/79942934 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77401d070 con 0x7ff780101d50 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7737fe700 1 -- 172.21.15.181:0/79942934 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff774021410 con 0x7ff780101d50 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7737fe700 1 -- 172.21.15.181:0/79942934 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774017650 con 0x7ff780101d50 2024-05-10T04:36:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff780199fc0 con 0x7ff780101d50 2024-05-10T04:36:23.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.605+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7801a06c0 con 0x7ff780101d50 2024-05-10T04:36:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.606+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff780108f70 con 0x7ff780101d50 2024-05-10T04:36:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.608+0000 7ff7737fe700 1 -- 172.21.15.181:0/79942934 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff774021580 con 0x7ff780101d50 2024-05-10T04:36:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.608+0000 7ff7737fe700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff76c0a1120 0x7ff76c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.608+0000 7ff7737fe700 1 -- 172.21.15.181:0/79942934 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5117+0+0 (secure 0 0 0) 0x7ff7740cd6d0 con 0x7ff780101d50 2024-05-10T04:36:23.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.608+0000 7ff785a40700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff76c0a1120 0x7ff76c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:23.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.609+0000 7ff785a40700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff76c0a1120 0x7ff76c0a3630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff780199150 tx=0x7ff77c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:23.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.612+0000 7ff7737fe700 1 -- 172.21.15.181:0/79942934 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff774097b70 con 0x7ff780101d50 2024-05-10T04:36:23.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:36:23.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: osdmap e49: 8 total, 7 up, 8 in 2024-05-10T04:36:23.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:23 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:23.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T04:36:23.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T04:36:23.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: osdmap e49: 8 total, 7 up, 8 in 2024-05-10T04:36:23.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:23 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:23.813 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:23.812+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff78010d690 con 0x7ff76c0a1120 2024-05-10T04:36:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-10T04:36:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: osdmap e48: 8 total, 7 up, 8 in 2024-05-10T04:36:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: osdmap e49: 8 total, 7 up, 8 in 2024-05-10T04:36:23.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:23 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:24.335 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:36:23 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:36:23.867+0000 7fda04ad0700 -1 osd.7 0 waiting for initial osdmap 2024-05-10T04:36:24.335 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:36:23 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:36:23.882+0000 7fd9fc87f700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:36:24.608 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi161 2024-05-10T04:36:24.608 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi161 to 5704M 2024-05-10T04:36:24.608 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:24.608 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:24.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:24.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:24.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:24.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' 2024-05-10T04:36:24.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] boot 2024-05-10T04:36:24.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T04:36:24.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:24 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi161 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi161 to 5704M 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] boot 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T04:36:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:24 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi161 2024-05-10T04:36:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi161 to 5704M 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: from='osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973]' entity='osd.7' 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: osd.7 [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] boot 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: osdmap e50: 8 total, 8 up, 8 in 2024-05-10T04:36:24.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:24 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-10T04:36:25.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:25 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:36:25.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:25 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:36:25.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:25 smithi181 ceph-mon[47314]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:25.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:25 smithi181 ceph-mon[47314]: osdmap e51: 8 total, 8 up, 8 in 2024-05-10T04:36:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:25 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:36:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:25 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:36:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:25 smithi150 ceph-mon[42404]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:25.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:25 smithi150 ceph-mon[42404]: osdmap e51: 8 total, 8 up, 8 in 2024-05-10T04:36:25.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:25 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:36:25.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:25 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:36:25.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:25 smithi161 ceph-mon[46912]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:25.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:25 smithi161 ceph-mon[46912]: osdmap e51: 8 total, 8 up, 8 in 2024-05-10T04:36:26.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:26 smithi181 ceph-mon[47314]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:26 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/3659442770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]: dispatch 2024-05-10T04:36:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:26 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]: dispatch 2024-05-10T04:36:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:26 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]': finished 2024-05-10T04:36:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:26 smithi181 ceph-mon[47314]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T04:36:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:26 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:26 smithi150 ceph-mon[42404]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:26.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:26 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/3659442770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]: dispatch 2024-05-10T04:36:26.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:26 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]: dispatch 2024-05-10T04:36:26.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:26 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]': finished 2024-05-10T04:36:26.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:26 smithi150 ceph-mon[42404]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T04:36:26.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:26 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:26.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:26 smithi161 ceph-mon[46912]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:26.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:26 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/3659442770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]: dispatch 2024-05-10T04:36:26.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:26 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]: dispatch 2024-05-10T04:36:26.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:26 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aceac05-7393-4887-8b99-8ed52159a6a0"}]': finished 2024-05-10T04:36:26.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:26 smithi161 ceph-mon[46912]: osdmap e52: 9 total, 8 up, 9 in 2024-05-10T04:36:26.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:26 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:27.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:27 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/2127679484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:27.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:27 smithi181 ceph-mon[47314]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T04:36:27.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:27 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:27.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:27 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/2127679484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:27.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:27 smithi150 ceph-mon[42404]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T04:36:27.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:27 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:27.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:27 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/2127679484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:27.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:27 smithi161 ceph-mon[46912]: osdmap e53: 9 total, 8 up, 9 in 2024-05-10T04:36:27.854 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:27 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:28.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:28 smithi181 ceph-mon[47314]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:28.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:28 smithi181 ceph-mon[47314]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T04:36:28.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:28 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:28.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:28 smithi150 ceph-mon[42404]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:28.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:28 smithi150 ceph-mon[42404]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T04:36:28.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:28 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:28.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:28 smithi161 ceph-mon[46912]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:28.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:28 smithi161 ceph-mon[46912]: osdmap e54: 9 total, 8 up, 9 in 2024-05-10T04:36:28.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:28 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:30.640 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:30 smithi181 ceph-mon[47314]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:30 smithi150 ceph-mon[42404]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:30.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:30 smithi161 ceph-mon[46912]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 267 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:32.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:32 smithi181 ceph-mon[47314]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:32.606 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:32 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T04:36:32.607 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:32 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:32 smithi150 ceph-mon[42404]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T04:36:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:32 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:32 smithi161 ceph-mon[46912]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:32 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-10T04:36:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:32 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:33.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:33 smithi181 ceph-mon[47314]: Deploying daemon osd.8 on smithi181 2024-05-10T04:36:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:33 smithi150 ceph-mon[42404]: Deploying daemon osd.8 on smithi181 2024-05-10T04:36:33.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:33 smithi161 ceph-mon[46912]: Deploying daemon osd.8 on smithi181 2024-05-10T04:36:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:34 smithi181 ceph-mon[47314]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:34 smithi150 ceph-mon[42404]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:34 smithi161 ceph-mon[46912]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:35 smithi181 ceph-mon[47314]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:35 smithi150 ceph-mon[42404]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:35 smithi161 ceph-mon[46912]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:36.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:36.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:36.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:37.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:38.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:37 smithi161 ceph-mon[46912]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:38.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:37 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:37 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:37 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:38.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:37 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:38.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:37 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.109 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:37 smithi181 ceph-mon[47314]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:38.109 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:37 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.109 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:37 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.109 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:37 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:38.110 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:37 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:38.110 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:37 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:37 smithi150 ceph-mon[42404]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:37 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:37 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:37 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:38.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:37 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:38.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:37 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:38.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.166+0000 7ff7737fe700 1 -- 172.21.15.181:0/79942934 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff78010d690 con 0x7ff76c0a1120 2024-05-10T04:36:38.167 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 8 on host 'smithi181' 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff76c0a1120 msgr2=0x7ff76c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff76c0a1120 0x7ff76c0a3630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff780199150 tx=0x7ff77c009260 comp rx=0 tx=0).stop 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 msgr2=0x7ff780198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780198180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098e0 tx=0x7ff774009580 comp rx=0 tx=0).stop 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 shutdown_connections 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff780103820 0x7ff780199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff76c0a1120 0x7ff76c0a3630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:38.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff780101d50 0x7ff780198180 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 --2- 172.21.15.181:0/79942934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff780102e80 0x7ff7801986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 >> 172.21.15.181:0/79942934 conn(0x7ff7800fd390 msgr2=0x7ff7801060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 shutdown_connections 2024-05-10T04:36:38.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:36:38.168+0000 7ff7884a5700 1 -- 172.21.15.181:0/79942934 wait complete. 2024-05-10T04:36:38.540 DEBUG:teuthology.orchestra.run.smithi181:osd.8> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.8.service 2024-05-10T04:36:38.543 INFO:tasks.cephadm:Deploying osd.9 on smithi150 with /dev/vg_nvme/lv_1... 2024-05-10T04:36:38.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T04:36:38.589 INFO:journalctl@ceph.osd.8.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:36:38.790 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:36:39.228 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:39.228 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:39.228 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:39 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:39.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:39.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:39.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:39 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:39.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:39.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:39.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:39 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T04:36:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T04:36:40.114 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-10T04:36:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-10T04:36:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113548 s, 923 MB/s 2024-05-10T04:36:40.114 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-10T04:36:40.143 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:36:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:36:39.746+0000 7f09e2a4b7c0 -1 osd.8 0 log_to_monitors true 2024-05-10T04:36:40.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:40 smithi181 ceph-mon[47314]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:40.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:40 smithi181 ceph-mon[47314]: from='osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:36:40.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:40 smithi181 ceph-mon[47314]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:36:40.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:40 smithi150 ceph-mon[42404]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:40.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:40 smithi150 ceph-mon[42404]: from='osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:36:40.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:40 smithi150 ceph-mon[42404]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:36:40.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-05-10T04:36:40.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:40 smithi161 ceph-mon[46912]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:40.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:40 smithi161 ceph-mon[46912]: from='osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:36:40.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:40 smithi161 ceph-mon[46912]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-10T04:36:40.705 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:41.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:41.156 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:41 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:41.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:41.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:41 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: osdmap e55: 9 total, 8 up, 9 in 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:41.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:41.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:41.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:41 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:41.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 -- 172.21.15.150:0/447715463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 msgr2=0x7f8868100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:41.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 --2- 172.21.15.150:0/447715463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868100720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8858009990 tx=0x7f8858009ca0 comp rx=0 tx=0).stop 2024-05-10T04:36:41.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 -- 172.21.15.150:0/447715463 shutdown_connections 2024-05-10T04:36:41.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 --2- 172.21.15.150:0/447715463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8868101de0 0x7f8868108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:41.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 --2- 172.21.15.150:0/447715463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868100720 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 --2- 172.21.15.150:0/447715463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8868101440 0x7f8868101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 -- 172.21.15.150:0/447715463 >> 172.21.15.150:0/447715463 conn(0x7f88680fb950 msgr2=0x7f88680fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 -- 172.21.15.150:0/447715463 shutdown_connections 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.591+0000 7f886e3a9700 1 -- 172.21.15.150:0/447715463 wait complete. 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 Processor -- start 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 -- start start 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8868101440 0x7f88681986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8868101de0 0x7f8868199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:41.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868199ab0 con 0x7f8868100310 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868199bf0 con 0x7f8868101de0 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f886e3a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868199db0 con 0x7f8868101440 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.592+0000 7f8867fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38796/0 (socket says 172.21.15.150:38796) 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 -- 172.21.15.150:0/1823093980 learned_addr learned my addr 172.21.15.150:0/1823093980 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f885ffff700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8868101440 0x7f88681986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f886c946700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8868101de0 0x7f8868199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 -- 172.21.15.150:0/1823093980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8868101440 msgr2=0x7f88681986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8868101440 0x7f88681986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:41.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 -- 172.21.15.150:0/1823093980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8868101de0 msgr2=0x7f8868199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8868101de0 0x7f8868199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 -- 172.21.15.150:0/1823093980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88580095f0 con 0x7f8868100310 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8867fff700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868198160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8858000c00 tx=0x7f8858009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8865ffb700 1 -- 172.21.15.150:0/1823093980 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885801d070 con 0x7f8868100310 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8865ffb700 1 -- 172.21.15.150:0/1823093980 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8858021410 con 0x7f8868100310 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f8865ffb700 1 -- 172.21.15.150:0/1823093980 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858017650 con 0x7f8868100310 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886819a030 con 0x7f8868100310 2024-05-10T04:36:41.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.593+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88681a0870 con 0x7f8868100310 2024-05-10T04:36:41.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.595+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8868107530 con 0x7f8868100310 2024-05-10T04:36:41.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.596+0000 7f8865ffb700 1 -- 172.21.15.150:0/1823093980 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8858021580 con 0x7f8868100310 2024-05-10T04:36:41.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.597+0000 7f8865ffb700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88540a11a0 0x7f88540a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:41.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.597+0000 7f885ffff700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88540a11a0 0x7f88540a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:41.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.597+0000 7f885ffff700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88540a11a0 0x7f88540a36b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8868199130 tx=0x7f885000b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:41.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.599+0000 7f8865ffb700 1 -- 172.21.15.150:0/1823093980 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f885809b7a0 con 0x7f8868100310 2024-05-10T04:36:41.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.604+0000 7f8865ffb700 1 -- 172.21.15.150:0/1823093980 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f88580973e0 con 0x7f8868100310 2024-05-10T04:36:41.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:41.805+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f886810b330 con 0x7f88540a11a0 2024-05-10T04:36:41.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:36:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:36:41.590+0000 7f09d80af700 -1 osd.8 0 waiting for initial osdmap 2024-05-10T04:36:41.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:36:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:36:41.607+0000 7f09cfe5e700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:36:42.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi181 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi181 to 5704M 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: osdmap e56: 9 total, 8 up, 9 in 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: from='osd.8 ' entity='osd.8' 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:42.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi181 2024-05-10T04:36:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi181 to 5704M 2024-05-10T04:36:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:36:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: osdmap e56: 9 total, 8 up, 9 in 2024-05-10T04:36:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:42.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:42.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:42.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: from='osd.8 ' entity='osd.8' 2024-05-10T04:36:42.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:42.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:42.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:42.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi181 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi181 to 5704M 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: osdmap e56: 9 total, 8 up, 9 in 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: from='osd.8 ' entity='osd.8' 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:42.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:43.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:36:43.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:36:43.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:43.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:43.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] boot 2024-05-10T04:36:43.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T04:36:43.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:43.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:43 smithi181 ceph-mon[47314]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T04:36:43.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:36:43.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:36:43.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:43.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:43.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] boot 2024-05-10T04:36:43.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T04:36:43.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:43.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:43 smithi150 ceph-mon[42404]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T04:36:43.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:36:43.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:36:43.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:36:43.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:43.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: osd.8 [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] boot 2024-05-10T04:36:43.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: osdmap e57: 9 total, 9 up, 9 in 2024-05-10T04:36:43.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-10T04:36:43.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:43 smithi161 ceph-mon[46912]: osdmap e58: 9 total, 9 up, 9 in 2024-05-10T04:36:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:44 smithi181 ceph-mon[47314]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:44 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3668728816' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84f2137-43f9-4072-8828-316f6a5f3582"}]: dispatch 2024-05-10T04:36:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:44 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3668728816' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f84f2137-43f9-4072-8828-316f6a5f3582"}]': finished 2024-05-10T04:36:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:44 smithi181 ceph-mon[47314]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T04:36:44.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:44.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:44 smithi150 ceph-mon[42404]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:44.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:44 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3668728816' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84f2137-43f9-4072-8828-316f6a5f3582"}]: dispatch 2024-05-10T04:36:44.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:44 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3668728816' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f84f2137-43f9-4072-8828-316f6a5f3582"}]': finished 2024-05-10T04:36:44.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:44 smithi150 ceph-mon[42404]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T04:36:44.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:44.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:44 smithi161 ceph-mon[46912]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:44.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:44 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3668728816' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f84f2137-43f9-4072-8828-316f6a5f3582"}]: dispatch 2024-05-10T04:36:44.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:44 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3668728816' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f84f2137-43f9-4072-8828-316f6a5f3582"}]': finished 2024-05-10T04:36:44.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:44 smithi161 ceph-mon[46912]: osdmap e59: 10 total, 9 up, 10 in 2024-05-10T04:36:44.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:45.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:45 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4110638049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:45.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:45 smithi161 ceph-mon[46912]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T04:36:45.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:45.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:45 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4110638049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:45.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:45 smithi181 ceph-mon[47314]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T04:36:45.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:45.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:45 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4110638049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:36:45.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:45 smithi150 ceph-mon[42404]: osdmap e60: 10 total, 9 up, 10 in 2024-05-10T04:36:45.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:46.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:46 smithi161 ceph-mon[46912]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T04:36:46.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:46 smithi161 ceph-mon[46912]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T04:36:46.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:46 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:46.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:46 smithi181 ceph-mon[47314]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T04:36:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:46 smithi181 ceph-mon[47314]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T04:36:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:46 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:46 smithi150 ceph-mon[42404]: pgmap v157: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-10T04:36:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:46 smithi150 ceph-mon[42404]: osdmap e61: 10 total, 9 up, 10 in 2024-05-10T04:36:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:46 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:48.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:48 smithi181 ceph-mon[47314]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-10T04:36:48.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:48 smithi150 ceph-mon[42404]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-10T04:36:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:48 smithi161 ceph-mon[46912]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-05-10T04:36:50.593 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:50 smithi150 ceph-mon[42404]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T04:36:50.593 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T04:36:50.593 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:50.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:50 smithi181 ceph-mon[47314]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T04:36:50.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T04:36:50.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:50.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:50 smithi161 ceph-mon[46912]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-10T04:36:50.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-10T04:36:50.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:51.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:51 smithi181 ceph-mon[47314]: Deploying daemon osd.9 on smithi150 2024-05-10T04:36:51.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:51 smithi150 ceph-mon[42404]: Deploying daemon osd.9 on smithi150 2024-05-10T04:36:51.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:51 smithi161 ceph-mon[46912]: Deploying daemon osd.9 on smithi150 2024-05-10T04:36:52.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:52 smithi181 ceph-mon[47314]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-10T04:36:52.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:52 smithi150 ceph-mon[42404]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-10T04:36:52.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:52 smithi161 ceph-mon[46912]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-10T04:36:54.371 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:54.371 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:54.371 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:54.371 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:54 smithi150 ceph-mon[42404]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:54.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:54.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:54.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:54.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:54 smithi161 ceph-mon[46912]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:54 smithi181 ceph-mon[47314]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:55.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.501+0000 7f8865ffb700 1 -- 172.21.15.150:0/1823093980 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f886810b330 con 0x7f88540a11a0 2024-05-10T04:36:55.503 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 9 on host 'smithi150' 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88540a11a0 msgr2=0x7f88540a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88540a11a0 0x7f88540a36b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8868199130 tx=0x7f885000b510 comp rx=0 tx=0).stop 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 msgr2=0x7f8868198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868198160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8858000c00 tx=0x7f8858009580 comp rx=0 tx=0).stop 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 shutdown_connections 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8868101de0 0x7f8868199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8868100310 0x7f8868198160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88540a11a0 0x7f88540a36b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:55.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 --2- 172.21.15.150:0/1823093980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8868101440 0x7f88681986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.503+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 >> 172.21.15.150:0/1823093980 conn(0x7f88680fb950 msgr2=0x7f8868104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.504+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 shutdown_connections 2024-05-10T04:36:55.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:36:55.504+0000 7f886e3a9700 1 -- 172.21.15.150:0/1823093980 wait complete. 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.835 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.836 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:55.894 DEBUG:teuthology.orchestra.run.smithi150:osd.9> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.9.service 2024-05-10T04:36:55.897 INFO:tasks.cephadm:Deploying osd.10 on smithi161 with /dev/vg_nvme/lv_1... 2024-05-10T04:36:55.897 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T04:36:55.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:55.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:55.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:55.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:55.930 INFO:journalctl@ceph.osd.9.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:36:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:55.944 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:55.945 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:36:56.141 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:36:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T04:36:57.469 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T04:36:57.469 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-05-10T04:36:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-05-10T04:36:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011202 s, 936 MB/s 2024-05-10T04:36:57.470 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-05-10T04:36:57.584 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:57 smithi150 ceph-mon[42404]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:36:57.584 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:36:57 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:36:57.259+0000 7f022dbf97c0 -1 osd.9 0 log_to_monitors true 2024-05-10T04:36:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:57 smithi161 ceph-mon[46912]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:36:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:57 smithi181 ceph-mon[47314]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-10T04:36:57.861 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-05-10T04:36:58.085 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.473 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:58.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:58.474 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:58.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:58.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:36:58 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:36:58.324+0000 7f022325d700 -1 osd.9 0 waiting for initial osdmap 2024-05-10T04:36:58.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:36:58 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:36:58.341+0000 7f021f014700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:36:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:36:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-10T04:36:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: osdmap e62: 10 total, 9 up, 10 in 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:36:58.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:36:58.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 -- 172.21.15.161:0/4013808060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c102e10 msgr2=0x7f175c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:58.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 --2- 172.21.15.161:0/4013808060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c102e10 0x7f175c103220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f174c009990 tx=0x7f174c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:36:58.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 -- 172.21.15.161:0/4013808060 shutdown_connections 2024-05-10T04:36:58.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 --2- 172.21.15.161:0/4013808060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f175c1037b0 0x7f175c10a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:58.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 --2- 172.21.15.161:0/4013808060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f175c101ce0 0x7f175c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:58.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 --2- 172.21.15.161:0/4013808060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c102e10 0x7f175c103220 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:58.998 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 -- 172.21.15.161:0/4013808060 >> 172.21.15.161:0/4013808060 conn(0x7f175c0fd320 msgr2=0x7f175c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 -- 172.21.15.161:0/4013808060 shutdown_connections 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.994+0000 7f1761e52700 1 -- 172.21.15.161:0/4013808060 wait complete. 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 Processor -- start 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 -- start start 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c101ce0 0x7f175c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f175c102e10 0x7f175c074670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f175c1037b0 0x7f175c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c075400 con 0x7f175c1037b0 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1a2100 con 0x7f175c101ce0 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f1761e52700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1a2240 con 0x7f175c102e10 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f175b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c101ce0 0x7f175c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f175b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c101ce0 0x7f175c0761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54312/0 (socket says 172.21.15.161:54312) 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f175b7fe700 1 -- 172.21.15.161:0/152989812 learned_addr learned my addr 172.21.15.161:0/152989812 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-05-10T04:36:58.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.995+0000 7f175bfff700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f175c1037b0 0x7f175c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175affd700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f175c102e10 0x7f175c074670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175b7fe700 1 -- 172.21.15.161:0/152989812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f175c102e10 msgr2=0x7f175c074670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175b7fe700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f175c102e10 0x7f175c074670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175b7fe700 1 -- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f175c1037b0 msgr2=0x7f175c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175b7fe700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f175c1037b0 0x7f175c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175b7fe700 1 -- 172.21.15.161:0/152989812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c0095f0 con 0x7f175c101ce0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175affd700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f175c102e10 0x7f175c074670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f175b7fe700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c101ce0 0x7f175c0761a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f174400d980 tx=0x7f174400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f1758ff9700 1 -- 172.21.15.161:0/152989812 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400b950 con 0x7f175c101ce0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f1758ff9700 1 -- 172.21.15.161:0/152989812 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1744007620 con 0x7f175c101ce0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c1a24a0 con 0x7f175c101ce0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f1758ff9700 1 -- 172.21.15.161:0/152989812 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400f3c0 con 0x7f175c101ce0 2024-05-10T04:36:59.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.996+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c1a29a0 con 0x7f175c101ce0 2024-05-10T04:36:59.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.997+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c108f00 con 0x7f175c101ce0 2024-05-10T04:36:59.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.998+0000 7f1758ff9700 1 -- 172.21.15.161:0/152989812 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f17440105d0 con 0x7f175c101ce0 2024-05-10T04:36:59.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.999+0000 7f1758ff9700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17480a1120 0x7f17480a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:36:59.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.999+0000 7f1758ff9700 1 -- 172.21.15.161:0/152989812 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f17440cc6c0 con 0x7f175c101ce0 2024-05-10T04:36:59.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.999+0000 7f175affd700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17480a1120 0x7f17480a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:36:59.002 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:58.999+0000 7f175affd700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17480a1120 0x7f17480a3630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f174c000c00 tx=0x7f174c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:36:59.004 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:59.002+0000 7f1758ff9700 1 -- 172.21.15.161:0/152989812 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1744096770 con 0x7f175c101ce0 2024-05-10T04:36:59.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:36:59.203+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f175c10d710 con 0x7f17480a1120 2024-05-10T04:36:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi150 2024-05-10T04:36:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi150 to 4278M 2024-05-10T04:36:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:36:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: osdmap e63: 10 total, 9 up, 10 in 2024-05-10T04:36:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:59.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:59.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:59.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:59.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:36:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:59.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi150 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi150 to 4278M 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: osdmap e63: 10 total, 9 up, 10 in 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:36:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:36:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi150 2024-05-10T04:36:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi150 to 4278M 2024-05-10T04:36:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: from='osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-10T04:36:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: osdmap e63: 10 total, 9 up, 10 in 2024-05-10T04:36:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:36:59.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:36:59.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:36:59.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:36:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:00.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:37:00.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:37:00.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:00.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:37:00.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:37:00.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] boot 2024-05-10T04:37:00.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T04:37:00.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:37:00.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:37:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:37:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:37:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:37:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] boot 2024-05-10T04:37:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T04:37:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:37:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:37:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:37:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-10T04:37:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:37:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: osd.9 [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] boot 2024-05-10T04:37:00.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: osdmap e64: 10 total, 10 up, 10 in 2024-05-10T04:37:00.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-10T04:37:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:01 smithi181 ceph-mon[47314]: osdmap e65: 10 total, 10 up, 10 in 2024-05-10T04:37:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:01 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/2934110993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]: dispatch 2024-05-10T04:37:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:01 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]: dispatch 2024-05-10T04:37:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:01 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]': finished 2024-05-10T04:37:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:01 smithi181 ceph-mon[47314]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T04:37:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:01.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:01 smithi150 ceph-mon[42404]: osdmap e65: 10 total, 10 up, 10 in 2024-05-10T04:37:01.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:01 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/2934110993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]: dispatch 2024-05-10T04:37:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:01 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]: dispatch 2024-05-10T04:37:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:01 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]': finished 2024-05-10T04:37:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:01 smithi150 ceph-mon[42404]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T04:37:01.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:01 smithi161 ceph-mon[46912]: osdmap e65: 10 total, 10 up, 10 in 2024-05-10T04:37:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:01 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/2934110993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]: dispatch 2024-05-10T04:37:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:01 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]: dispatch 2024-05-10T04:37:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:01 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0228ed-9ae2-4aa7-bd86-db1412ad4969"}]': finished 2024-05-10T04:37:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:01 smithi161 ceph-mon[46912]: osdmap e66: 11 total, 10 up, 11 in 2024-05-10T04:37:01.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:02 smithi181 ceph-mon[47314]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:02 smithi181 ceph-mon[47314]: from='client.? 172.21.15.161:0/3477495118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:37:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:02 smithi150 ceph-mon[42404]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:02 smithi150 ceph-mon[42404]: from='client.? 172.21.15.161:0/3477495118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:37:02.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:02 smithi161 ceph-mon[46912]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:02.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:02 smithi161 ceph-mon[46912]: from='client.? 172.21.15.161:0/3477495118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:37:04.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:04 smithi181 ceph-mon[47314]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:04 smithi150 ceph-mon[42404]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:04.828 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:04 smithi161 ceph-mon[46912]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:06 smithi161 ceph-mon[46912]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:06.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:06 smithi181 ceph-mon[47314]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:06 smithi150 ceph-mon[42404]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:07.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T04:37:07.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T04:37:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:07.721 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-10T04:37:07.722 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:08.452 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:08 smithi161 ceph-mon[46912]: Deploying daemon osd.10 on smithi161 2024-05-10T04:37:08.452 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:08 smithi161 ceph-mon[46912]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:08.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:08 smithi181 ceph-mon[47314]: Deploying daemon osd.10 on smithi161 2024-05-10T04:37:08.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:08 smithi181 ceph-mon[47314]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:08 smithi150 ceph-mon[42404]: Deploying daemon osd.10 on smithi161 2024-05-10T04:37:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:08 smithi150 ceph-mon[42404]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:10.645 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:10 smithi161 ceph-mon[46912]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:10.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:10 smithi181 ceph-mon[47314]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:10 smithi150 ceph-mon[42404]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:12.031 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:12.031 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:12.031 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:12.031 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:11 smithi161 ceph-mon[46912]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:12.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:12.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:12.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:12.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:11 smithi181 ceph-mon[47314]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:12.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:12.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:12.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:12.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:11 smithi150 ceph-mon[42404]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:13.061 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 10 on host 'smithi161' 2024-05-10T04:37:13.061 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.059+0000 7f1758ff9700 1 -- 172.21.15.161:0/152989812 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f175c10d710 con 0x7f17480a1120 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17480a1120 msgr2=0x7f17480a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17480a1120 0x7f17480a3630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f174c000c00 tx=0x7f174c005fb0 comp rx=0 tx=0).stop 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c101ce0 msgr2=0x7f175c0761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c101ce0 0x7f175c0761a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f174400d980 tx=0x7f174400dc90 comp rx=0 tx=0).stop 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 shutdown_connections 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f175c1037b0 0x7f175c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f175c101ce0 0x7f175c0761a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17480a1120 0x7f17480a3630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 --2- 172.21.15.161:0/152989812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f175c102e10 0x7f175c074670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 >> 172.21.15.161:0/152989812 conn(0x7f175c0fd320 msgr2=0x7f175c106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:13.063 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 shutdown_connections 2024-05-10T04:37:13.064 INFO:teuthology.orchestra.run.smithi161.stderr:2024-05-10T04:37:13.062+0000 7f1761e52700 1 -- 172.21.15.161:0/152989812 wait complete. 2024-05-10T04:37:13.429 DEBUG:teuthology.orchestra.run.smithi161:osd.10> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.10.service 2024-05-10T04:37:13.431 INFO:tasks.cephadm:Deploying osd.11 on smithi181 with /dev/vg_nvme/lv_1... 2024-05-10T04:37:13.431 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-10T04:37:13.463 INFO:journalctl@ceph.osd.10.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:37:13.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:13.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:13.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:13.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:13.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:13.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:13.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:13.663 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:37:14.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:14 smithi181 ceph-mon[47314]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:14.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:14 smithi161 ceph-mon[46912]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:14 smithi150 ceph-mon[42404]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-10T04:37:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-10T04:37:15.074 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2024-05-10T04:37:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2024-05-10T04:37:15.074 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112309 s, 934 MB/s 2024-05-10T04:37:15.074 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2024-05-10T04:37:15.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:37:14 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:37:14.689+0000 7f224fe8f7c0 -1 osd.10 0 log_to_monitors true 2024-05-10T04:37:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:15 smithi181 ceph-mon[47314]: from='osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:37:15.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:15 smithi181 ceph-mon[47314]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:37:15.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2024-05-10T04:37:15.557 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:15 smithi161 ceph-mon[46912]: from='osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:37:15.557 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:15 smithi161 ceph-mon[46912]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:37:15.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:15 smithi150 ceph-mon[42404]: from='osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:37:15.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:15 smithi150 ceph-mon[42404]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-10T04:37:15.699 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:37:16.585 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:37:16 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:37:16.317+0000 7f22454f3700 -1 osd.10 0 waiting for initial osdmap 2024-05-10T04:37:16.586 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:37:16 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:37:16.333+0000 7f22412aa700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:16.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:16.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:16 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:16.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T04:37:16.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T04:37:16.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:16.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:37:16.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:37:16.609 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:16.610 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:16 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:16.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-10T04:37:16.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: osdmap e67: 11 total, 10 up, 11 in 2024-05-10T04:37:16.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:16.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:16 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:16.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.805+0000 7f75828fc700 1 -- 172.21.15.181:0/106829429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 msgr2=0x7f757c102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:16.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.805+0000 7f75828fc700 1 --2- 172.21.15.181:0/106829429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c102060 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7570009990 tx=0x7f7570009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 -- 172.21.15.181:0/106829429 shutdown_connections 2024-05-10T04:37:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 --2- 172.21.15.181:0/106829429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757c1025f0 0x7f757c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 --2- 172.21.15.181:0/106829429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c100b20 0x7f757c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 --2- 172.21.15.181:0/106829429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c102060 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 -- 172.21.15.181:0/106829429 >> 172.21.15.181:0/106829429 conn(0x7f757c0fc160 msgr2=0x7f757c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 -- 172.21.15.181:0/106829429 shutdown_connections 2024-05-10T04:37:16.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 -- 172.21.15.181:0/106829429 wait complete. 2024-05-10T04:37:16.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 Processor -- start 2024-05-10T04:37:16.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.806+0000 7f75828fc700 1 -- start start 2024-05-10T04:37:16.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75828fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757c100b20 0x7f757c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:16.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75828fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c19d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75828fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c1025f0 0x7f757c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75828fc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c19fd80 con 0x7f757c100b20 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75828fc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c19fec0 con 0x7f757c1025f0 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75828fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c1a0000 con 0x7f757c101c50 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c19d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c19d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44946/0 (socket says 172.21.15.181:44946) 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 -- 172.21.15.181:0/1306197880 learned_addr learned my addr 172.21.15.181:0/1306197880 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757bfff700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757c100b20 0x7f757c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f7580e99700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c1025f0 0x7f757c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 -- 172.21.15.181:0/1306197880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c1025f0 msgr2=0x7f757c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c1025f0 0x7f757c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:16.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 -- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757c100b20 msgr2=0x7f757c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757c100b20 0x7f757c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 -- 172.21.15.181:0/1306197880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700095f0 con 0x7f757c101c50 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f757b7fe700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c19d2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75700095c0 tx=0x7f757000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75797fa700 1 -- 172.21.15.181:0/1306197880 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757001d070 con 0x7f757c101c50 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75797fa700 1 -- 172.21.15.181:0/1306197880 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7570005030 con 0x7f757c101c50 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c1a0280 con 0x7f757c101c50 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.807+0000 7f75797fa700 1 -- 172.21.15.181:0/1306197880 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570004ae0 con 0x7f757c101c50 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.808+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c1a0690 con 0x7f757c101c50 2024-05-10T04:37:16.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.808+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c107d40 con 0x7f757c101c50 2024-05-10T04:37:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.810+0000 7f75797fa700 1 -- 172.21.15.181:0/1306197880 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f75700051a0 con 0x7f757c101c50 2024-05-10T04:37:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.810+0000 7f75797fa700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f75640a1120 0x7f75640a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.810+0000 7f75797fa700 1 -- 172.21.15.181:0/1306197880 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f75700cd4d0 con 0x7f757c101c50 2024-05-10T04:37:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.810+0000 7f757bfff700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f75640a1120 0x7f75640a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:16.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.811+0000 7f757bfff700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f75640a1120 0x7f75640a3630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f757c069690 tx=0x7f756c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:16.814 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:16.814+0000 7f75797fa700 1 -- 172.21.15.181:0/1306197880 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f757009aef0 con 0x7f757c101c50 2024-05-10T04:37:17.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:17.016+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f757c10c4e0 con 0x7f75640a1120 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi161 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi161 to 4278M 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: osdmap e68: 11 total, 10 up, 11 in 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:37:17.580 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:17 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:17.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi161 2024-05-10T04:37:17.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi161 to 4278M 2024-05-10T04:37:17.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:37:17.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: osdmap e68: 11 total, 10 up, 11 in 2024-05-10T04:37:17.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:17.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:17.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:37:17.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:37:17.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:17 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:17.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi161 2024-05-10T04:37:17.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi161 to 4278M 2024-05-10T04:37:17.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-05-10T04:37:17.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: osdmap e68: 11 total, 10 up, 11 in 2024-05-10T04:37:17.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:17.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:17.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-10T04:37:17.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-10T04:37:17.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:17 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:18.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:37:18.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:37:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] boot 2024-05-10T04:37:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T04:37:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:18.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:18 smithi161 ceph-mon[46912]: osdmap e70: 11 total, 11 up, 11 in 2024-05-10T04:37:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:37:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:37:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:18.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:18.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] boot 2024-05-10T04:37:18.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T04:37:18.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:18.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:18 smithi181 ceph-mon[47314]: osdmap e70: 11 total, 11 up, 11 in 2024-05-10T04:37:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:37:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:37:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-10T04:37:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: osd.10 [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] boot 2024-05-10T04:37:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: osdmap e69: 11 total, 11 up, 11 in 2024-05-10T04:37:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-10T04:37:18.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:18 smithi150 ceph-mon[42404]: osdmap e70: 11 total, 11 up, 11 in 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: osdmap e71: 11 total, 11 up, 11 in 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/168872895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]: dispatch 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]: dispatch 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]': finished 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:37:20.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:19 smithi181 ceph-mon[47314]: from='client.? 172.21.15.181:0/2139211048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:37:20.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: osdmap e71: 11 total, 11 up, 11 in 2024-05-10T04:37:20.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/168872895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]: dispatch 2024-05-10T04:37:20.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]: dispatch 2024-05-10T04:37:20.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]': finished 2024-05-10T04:37:20.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T04:37:20.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:20.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:37:20.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:19 smithi150 ceph-mon[42404]: from='client.? 172.21.15.181:0/2139211048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:37:20.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: osdmap e71: 11 total, 11 up, 11 in 2024-05-10T04:37:20.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/168872895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]: dispatch 2024-05-10T04:37:20.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]: dispatch 2024-05-10T04:37:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "815f00ef-e11c-42c7-a1b4-52bbfefc253e"}]': finished 2024-05-10T04:37:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: osdmap e72: 12 total, 11 up, 12 in 2024-05-10T04:37:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:37:20.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:19 smithi161 ceph-mon[46912]: from='client.? 172.21.15.181:0/2139211048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-10T04:37:21.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:20 smithi161 ceph-mon[46912]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T04:37:21.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:20 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:21.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:20 smithi181 ceph-mon[47314]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T04:37:21.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:20 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:21.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:20 smithi150 ceph-mon[42404]: osdmap e73: 12 total, 11 up, 12 in 2024-05-10T04:37:21.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:20 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:22.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:21 smithi161 ceph-mon[46912]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T04:37:22.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:21 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:22.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:21 smithi161 ceph-mon[46912]: pgmap v189: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T04:37:22.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:21 smithi181 ceph-mon[47314]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T04:37:22.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:21 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:22.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:21 smithi181 ceph-mon[47314]: pgmap v189: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T04:37:22.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:21 smithi150 ceph-mon[42404]: osdmap e74: 12 total, 11 up, 12 in 2024-05-10T04:37:22.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:21 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:22.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:21 smithi150 ceph-mon[42404]: pgmap v189: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T04:37:23.068 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:22 smithi181 ceph-mon[47314]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-10T04:37:23.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:22 smithi161 ceph-mon[46912]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-10T04:37:23.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:22 smithi150 ceph-mon[42404]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-10T04:37:24.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:23 smithi181 ceph-mon[47314]: pgmap v190: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T04:37:24.339 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:23 smithi161 ceph-mon[46912]: pgmap v190: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T04:37:24.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:23 smithi150 ceph-mon[42404]: pgmap v190: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-10T04:37:25.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:25 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T04:37:25.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:25 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:25.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:25 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T04:37:25.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:25 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:25.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:25 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-10T04:37:25.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:25 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:26.342 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:26 smithi181 ceph-mon[47314]: Deploying daemon osd.11 on smithi181 2024-05-10T04:37:26.342 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:26 smithi181 ceph-mon[47314]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-10T04:37:26.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:26 smithi150 ceph-mon[42404]: Deploying daemon osd.11 on smithi181 2024-05-10T04:37:26.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:26 smithi150 ceph-mon[42404]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-10T04:37:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:26 smithi161 ceph-mon[46912]: Deploying daemon osd.11 on smithi181 2024-05-10T04:37:26.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:26 smithi161 ceph-mon[46912]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-10T04:37:27.300 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:27 smithi181 ceph-mon[47314]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-10T04:37:27.300 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:27 smithi181 ceph-mon[47314]: Cluster is now healthy 2024-05-10T04:37:27.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:27 smithi150 ceph-mon[42404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-10T04:37:27.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:27 smithi150 ceph-mon[42404]: Cluster is now healthy 2024-05-10T04:37:27.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:27 smithi161 ceph-mon[46912]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-10T04:37:27.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:27 smithi161 ceph-mon[46912]: Cluster is now healthy 2024-05-10T04:37:28.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:28 smithi181 ceph-mon[47314]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:37:28.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:28 smithi150 ceph-mon[42404]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:37:28.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:28 smithi161 ceph-mon[46912]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-10T04:37:30.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:30.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:30.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:30 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:30.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:30 smithi181 ceph-mon[47314]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-10T04:37:30.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:30.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:30.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:30 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:30.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:30 smithi161 ceph-mon[46912]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-10T04:37:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:30 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:30.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:30 smithi150 ceph-mon[42404]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-10T04:37:31.673 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 11 on host 'smithi181' 2024-05-10T04:37:31.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.672+0000 7f75797fa700 1 -- 172.21.15.181:0/1306197880 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f757c10c4e0 con 0x7f75640a1120 2024-05-10T04:37:31.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.674+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f75640a1120 msgr2=0x7f75640a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:31.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.674+0000 7f75828fc700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f75640a1120 0x7f75640a3630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f757c069690 tx=0x7f756c00b410 comp rx=0 tx=0).stop 2024-05-10T04:37:31.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.674+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 msgr2=0x7f757c19d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:31.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.674+0000 7f75828fc700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c19d2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75700095c0 tx=0x7f757000f690 comp rx=0 tx=0).stop 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 shutdown_connections 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f757c1025f0 0x7f757c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f757c100b20 0x7f757c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f75640a1120 0x7f75640a3630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 --2- 172.21.15.181:0/1306197880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f757c101c50 0x7f757c19d2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 >> 172.21.15.181:0/1306197880 conn(0x7f757c0fc160 msgr2=0x7f757c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 shutdown_connections 2024-05-10T04:37:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:31.675+0000 7f75828fc700 1 -- 172.21.15.181:0/1306197880 wait complete. 2024-05-10T04:37:32.064 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.064 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.064 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:32.064 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:32.064 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.064 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T04:37:32.064 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.065 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.065 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:31 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:32.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:32.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:32.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T04:37:32.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:31 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:32.102 DEBUG:teuthology.orchestra.run.smithi181:osd.11> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.11.service 2024-05-10T04:37:32.104 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-10T04:37:32.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T04:37:32.135 INFO:journalctl@ceph.osd.11.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:37:32.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:32.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:32.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-10T04:37:32.138 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:32.139 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:31 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:32.342 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:37:33.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:33 smithi181 ceph-mon[47314]: from='osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:37:33.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:33 smithi181 ceph-mon[47314]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:37:33.155 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:37:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:37:33.018+0000 7ff5d41337c0 -1 osd.11 0 log_to_monitors true 2024-05-10T04:37:33.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 -- 172.21.15.150:0/1178460293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4100bd0 msgr2=0x7f87b4100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 --2- 172.21.15.150:0/1178460293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4100bd0 0x7f87b4100fe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009990 tx=0x7f87a8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 -- 172.21.15.150:0/1178460293 shutdown_connections 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 --2- 172.21.15.150:0/1178460293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b41026a0 0x7f87b4108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 --2- 172.21.15.150:0/1178460293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4100bd0 0x7f87b4100fe0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 --2- 172.21.15.150:0/1178460293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b4101d00 0x7f87b4102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 -- 172.21.15.150:0/1178460293 >> 172.21.15.150:0/1178460293 conn(0x7f87b40fc230 msgr2=0x7f87b40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 -- 172.21.15.150:0/1178460293 shutdown_connections 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.290+0000 7f87bb009700 1 -- 172.21.15.150:0/1178460293 wait complete. 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 Processor -- start 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 -- start start 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4100bd0 0x7f87b4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:33.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4101d00 0x7f87b4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b41026a0 0x7f87b4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4199a90 con 0x7f87b4101d00 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4199bd0 con 0x7f87b41026a0 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87bb009700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4199d90 con 0x7f87b4100bd0 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b8da5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4100bd0 0x7f87b4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4101d00 0x7f87b4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b8da5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4100bd0 0x7f87b4198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:52960/0 (socket says 172.21.15.150:52960) 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b8da5700 1 -- 172.21.15.150:0/1209997678 learned_addr learned my addr 172.21.15.150:0/1209997678 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b95a6700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b41026a0 0x7f87b4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b3fff700 1 -- 172.21.15.150:0/1209997678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4100bd0 msgr2=0x7f87b4198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b3fff700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4100bd0 0x7f87b4198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b3fff700 1 -- 172.21.15.150:0/1209997678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b41026a0 msgr2=0x7f87b4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:33.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b3fff700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b41026a0 0x7f87b4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b3fff700 1 -- 172.21.15.150:0/1209997678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80095f0 con 0x7f87b4101d00 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b8da5700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4100bd0 0x7f87b4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.291+0000 7f87b3fff700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4101d00 0x7f87b4198680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87a000e9f0 tx=0x7f87a000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.292+0000 7f87b1ffb700 1 -- 172.21.15.150:0/1209997678 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000c9c0 con 0x7f87b4101d00 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.292+0000 7f87b1ffb700 1 -- 172.21.15.150:0/1209997678 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87a0010430 con 0x7f87b4101d00 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.292+0000 7f87b1ffb700 1 -- 172.21.15.150:0/1209997678 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0018680 con 0x7f87b4101d00 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.292+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b419a070 con 0x7f87b4101d00 2024-05-10T04:37:33.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.292+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b41a0850 con 0x7f87b4101d00 2024-05-10T04:37:33.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.293+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b40ffed0 con 0x7f87b4101d00 2024-05-10T04:37:33.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.294+0000 7f87b1ffb700 1 -- 172.21.15.150:0/1209997678 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f87a0010860 con 0x7f87b4101d00 2024-05-10T04:37:33.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.295+0000 7f87b1ffb700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87a40a11a0 0x7f87a40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:33.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.295+0000 7f87b8da5700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87a40a11a0 0x7f87a40a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:33.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.295+0000 7f87b1ffb700 1 -- 172.21.15.150:0/1209997678 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f87a0014070 con 0x7f87b4101d00 2024-05-10T04:37:33.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.295+0000 7f87b8da5700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87a40a11a0 0x7f87a40a36b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87a800b2e0 tx=0x7f87a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:33.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.298+0000 7f87b1ffb700 1 -- 172.21.15.150:0/1209997678 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f87a0096320 con 0x7f87b4101d00 2024-05-10T04:37:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:33 smithi150 ceph-mon[42404]: from='osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:37:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:33 smithi150 ceph-mon[42404]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:37:33.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.493+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f87b404edc0 con 0x7f87b4101d00 2024-05-10T04:37:33.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.493+0000 7f87b1ffb700 1 -- 172.21.15.150:0/1209997678 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f87a0099940 con 0x7f87b4101d00 2024-05-10T04:37:33.495 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:33.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87a40a11a0 msgr2=0x7f87a40a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:33.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87a40a11a0 0x7f87a40a36b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87a800b2e0 tx=0x7f87a801a040 comp rx=0 tx=0).stop 2024-05-10T04:37:33.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4101d00 msgr2=0x7f87b4198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:33.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4101d00 0x7f87b4198680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87a000e9f0 tx=0x7f87a000ed00 comp rx=0 tx=0).stop 2024-05-10T04:37:33.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 shutdown_connections 2024-05-10T04:37:33.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87b41026a0 0x7f87b4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87a40a11a0 0x7f87a40a36b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b4100bd0 0x7f87b4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 --2- 172.21.15.150:0/1209997678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b4101d00 0x7f87b4198680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 >> 172.21.15.150:0/1209997678 conn(0x7f87b40fc230 msgr2=0x7f87b40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 shutdown_connections 2024-05-10T04:37:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:33.496+0000 7f87bb009700 1 -- 172.21.15.150:0/1209997678 wait complete. 2024-05-10T04:37:33.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:33 smithi161 ceph-mon[46912]: from='osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:37:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:33 smithi161 ceph-mon[46912]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-10T04:37:33.876 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1715315837,"num_in_osds":12,"osd_in_since":1715315838,"num_remapped_pgs":0} 2024-05-10T04:37:34.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:34 smithi150 ceph-mon[42404]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T04:37:34.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:34 smithi150 ceph-mon[42404]: osdmap e75: 12 total, 11 up, 12 in 2024-05-10T04:37:34.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:34.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:34 smithi150 ceph-mon[42404]: from='osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:37:34.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:34 smithi150 ceph-mon[42404]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:37:34.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:34 smithi150 ceph-mon[42404]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:37:34.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1209997678' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:37:34.192 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:34 smithi181 ceph-mon[47314]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T04:37:34.193 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:34 smithi181 ceph-mon[47314]: osdmap e75: 12 total, 11 up, 12 in 2024-05-10T04:37:34.193 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:34.193 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:34 smithi181 ceph-mon[47314]: from='osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:37:34.193 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:34 smithi181 ceph-mon[47314]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:37:34.193 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:34 smithi181 ceph-mon[47314]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:37:34.193 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:34 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1209997678' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:37:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:34 smithi161 ceph-mon[46912]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-10T04:37:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:34 smithi161 ceph-mon[46912]: osdmap e75: 12 total, 11 up, 12 in 2024-05-10T04:37:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:34 smithi161 ceph-mon[46912]: from='osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:37:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:34 smithi161 ceph-mon[46912]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-05-10T04:37:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:34 smithi161 ceph-mon[46912]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-10T04:37:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:34 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1209997678' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:37:34.878 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-10T04:37:34.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:37:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:37:34.576+0000 7ff5c9f98700 -1 osd.11 0 waiting for initial osdmap 2024-05-10T04:37:34.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:37:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:37:34.594+0000 7ff5c1546700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-10T04:37:35.106 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: osdmap e76: 12 total, 11 up, 12 in 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: Detected new or changed devices on smithi181 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.162 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: Adjusting osd_memory_target on smithi181 to 4278M 2024-05-10T04:37:35.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:35.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:35.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.163 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:35 smithi150 ceph-mon[42404]: from='osd.11 ' entity='osd.11' 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: osdmap e76: 12 total, 11 up, 12 in 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: Detected new or changed devices on smithi181 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: Adjusting osd_memory_target on smithi181 to 4278M 2024-05-10T04:37:35.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:35.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:35.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:35 smithi181 ceph-mon[47314]: from='osd.11 ' entity='osd.11' 2024-05-10T04:37:35.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-05-10T04:37:35.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: osdmap e76: 12 total, 11 up, 12 in 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: Detected new or changed devices on smithi181 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: Adjusting osd_memory_target on smithi181 to 4278M 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:35.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:35.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:35 smithi161 ceph-mon[46912]: from='osd.11 ' entity='osd.11' 2024-05-10T04:37:35.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 -- 172.21.15.150:0/3274476261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc100bb0 msgr2=0x7fb3dc100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/3274476261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc100bb0 0x7fb3dc100fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009990 tx=0x7fb3cc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 -- 172.21.15.150:0/3274476261 shutdown_connections 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/3274476261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc102680 0x7fb3dc108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/3274476261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc100bb0 0x7fb3dc100fc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/3274476261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3dc101ce0 0x7fb3dc1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 -- 172.21.15.150:0/3274476261 >> 172.21.15.150:0/3274476261 conn(0x7fb3dc0fc210 msgr2=0x7fb3dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 -- 172.21.15.150:0/3274476261 shutdown_connections 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 -- 172.21.15.150:0/3274476261 wait complete. 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.989+0000 7fb3e1adb700 1 Processor -- start 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3e1adb700 1 -- start start 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3e1adb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc100bb0 0x7fb3dc104b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3e1adb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc101ce0 0x7fb3dc103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:35.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3e1adb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3dc102680 0x7fb3dc103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3e1adb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc103db0 con 0x7fb3dc101ce0 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3e1adb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc1a21b0 con 0x7fb3dc102680 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3e1adb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc1a22f0 con 0x7fb3dc100bb0 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc101ce0 0x7fb3dc103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc101ce0 0x7fb3dc103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53570/0 (socket says 172.21.15.150:53570) 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 -- 172.21.15.150:0/4131018643 learned_addr learned my addr 172.21.15.150:0/4131018643 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3db7fe700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc100bb0 0x7fb3dc104b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3dbfff700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3dc102680 0x7fb3dc103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 -- 172.21.15.150:0/4131018643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc100bb0 msgr2=0x7fb3dc104b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc100bb0 0x7fb3dc104b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 -- 172.21.15.150:0/4131018643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3dc102680 msgr2=0x7fb3dc103680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3dc102680 0x7fb3dc103680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 -- 172.21.15.150:0/4131018643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0095f0 con 0x7fb3dc101ce0 2024-05-10T04:37:35.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.990+0000 7fb3daffd700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc101ce0 0x7fb3dc103020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000e9d0 tx=0x7fb3d000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.991+0000 7fb3d8ff9700 1 -- 172.21.15.150:0/4131018643 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0009790 con 0x7fb3dc101ce0 2024-05-10T04:37:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.991+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3dc1a2550 con 0x7fb3dc101ce0 2024-05-10T04:37:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.991+0000 7fb3d8ff9700 1 -- 172.21.15.150:0/4131018643 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3d000ee70 con 0x7fb3dc101ce0 2024-05-10T04:37:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.991+0000 7fb3d8ff9700 1 -- 172.21.15.150:0/4131018643 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0010ea0 con 0x7fb3dc101ce0 2024-05-10T04:37:35.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.991+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3dc1a2a50 con 0x7fb3dc101ce0 2024-05-10T04:37:35.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.992+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3dc0ffeb0 con 0x7fb3dc101ce0 2024-05-10T04:37:35.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.993+0000 7fb3d8ff9700 1 -- 172.21.15.150:0/4131018643 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb3d000cbd0 con 0x7fb3dc101ce0 2024-05-10T04:37:35.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.993+0000 7fb3d8ff9700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3c40a1120 0x7fb3c40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:35.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.994+0000 7fb3db7fe700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3c40a1120 0x7fb3c40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:35.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.994+0000 7fb3d8ff9700 1 -- 172.21.15.150:0/4131018643 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb3d0014070 con 0x7fb3dc101ce0 2024-05-10T04:37:35.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.994+0000 7fb3db7fe700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3c40a1120 0x7fb3c40a3630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc104890 tx=0x7fb3cc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:35.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:35.997+0000 7fb3d8ff9700 1 -- 172.21.15.150:0/4131018643 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3d00966a0 con 0x7fb3dc101ce0 2024-05-10T04:37:36.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.192+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb3dc10c380 con 0x7fb3dc101ce0 2024-05-10T04:37:36.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.192+0000 7fb3d8ff9700 1 -- 172.21.15.150:0/4131018643 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7fb3d0099cc0 con 0x7fb3dc101ce0 2024-05-10T04:37:36.194 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:36.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.194+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3c40a1120 msgr2=0x7fb3c40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:36.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.194+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3c40a1120 0x7fb3c40a3630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc104890 tx=0x7fb3cc01a040 comp rx=0 tx=0).stop 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.194+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc101ce0 msgr2=0x7fb3dc103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.194+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc101ce0 0x7fb3dc103020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000e9d0 tx=0x7fb3d000ece0 comp rx=0 tx=0).stop 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 shutdown_connections 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3dc102680 0x7fb3dc103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3c40a1120 0x7fb3c40a3630 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc100bb0 0x7fb3dc104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 --2- 172.21.15.150:0/4131018643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3dc101ce0 0x7fb3dc103020 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:36.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 >> 172.21.15.150:0/4131018643 conn(0x7fb3dc0fc210 msgr2=0x7fb3dc0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:36.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 shutdown_connections 2024-05-10T04:37:36.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:36.195+0000 7fb3e1adb700 1 -- 172.21.15.150:0/4131018643 wait complete. 2024-05-10T04:37:36.378 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:36 smithi150 ceph-mon[42404]: purged_snaps scrub starts 2024-05-10T04:37:36.379 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:36 smithi150 ceph-mon[42404]: purged_snaps scrub ok 2024-05-10T04:37:36.379 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:36.379 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:36 smithi150 ceph-mon[42404]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:37:36.379 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:36 smithi150 ceph-mon[42404]: osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] boot 2024-05-10T04:37:36.379 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:36 smithi150 ceph-mon[42404]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T04:37:36.379 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:36 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:36.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:36 smithi181 ceph-mon[47314]: purged_snaps scrub starts 2024-05-10T04:37:36.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:36 smithi181 ceph-mon[47314]: purged_snaps scrub ok 2024-05-10T04:37:36.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:36.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:36 smithi181 ceph-mon[47314]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:37:36.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:36 smithi181 ceph-mon[47314]: osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] boot 2024-05-10T04:37:36.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:36 smithi181 ceph-mon[47314]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T04:37:36.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:36 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:36.558 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1715315855,"num_in_osds":12,"osd_in_since":1715315838,"num_remapped_pgs":0} 2024-05-10T04:37:36.559 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T04:37:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:36 smithi161 ceph-mon[46912]: purged_snaps scrub starts 2024-05-10T04:37:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:36 smithi161 ceph-mon[46912]: purged_snaps scrub ok 2024-05-10T04:37:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:36 smithi161 ceph-mon[46912]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-05-10T04:37:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:36 smithi161 ceph-mon[46912]: osd.11 [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] boot 2024-05-10T04:37:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:36 smithi161 ceph-mon[46912]: osdmap e77: 12 total, 12 up, 12 in 2024-05-10T04:37:36.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:36 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-10T04:37:36.781 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:37:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:37 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4131018643' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:37:37.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:37 smithi150 ceph-mon[42404]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T04:37:37.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:37 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4131018643' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:37:37.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:37 smithi181 ceph-mon[47314]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T04:37:37.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:37 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4131018643' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-10T04:37:37.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:37 smithi161 ceph-mon[46912]: osdmap e78: 12 total, 12 up, 12 in 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 -- 172.21.15.150:0/162143148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 msgr2=0x7f0b90105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 --2- 172.21.15.150:0/162143148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90105b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009990 tx=0x7f0b7c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 -- 172.21.15.150:0/162143148 shutdown_connections 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 --2- 172.21.15.150:0/162143148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b90106080 0x7f0b9010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 --2- 172.21.15.150:0/162143148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b90100e00 0x7f0b90101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 --2- 172.21.15.150:0/162143148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90105b40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 -- 172.21.15.150:0/162143148 >> 172.21.15.150:0/162143148 conn(0x7f0b900fc7a0 msgr2=0x7f0b900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 -- 172.21.15.150:0/162143148 shutdown_connections 2024-05-10T04:37:37.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.696+0000 7f0b96c99700 1 -- 172.21.15.150:0/162143148 wait complete. 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 Processor -- start 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 -- start start 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b90100e00 0x7f0b901973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b90106080 0x7f0b9019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9019c3d0 con 0x7f0b90105730 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9019c540 con 0x7f0b90100e00 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b96c99700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9019c700 con 0x7f0b90106080 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b8ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b8ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59038/0 (socket says 172.21.15.150:59038) 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b8ffff700 1 -- 172.21.15.150:0/2164196334 learned_addr learned my addr 172.21.15.150:0/2164196334 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b94a35700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b90100e00 0x7f0b901973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.697+0000 7f0b95236700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b90106080 0x7f0b9019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8ffff700 1 -- 172.21.15.150:0/2164196334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b90106080 msgr2=0x7f0b9019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:37.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8ffff700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b90106080 0x7f0b9019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8ffff700 1 -- 172.21.15.150:0/2164196334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b90100e00 msgr2=0x7f0b901973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8ffff700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b90100e00 0x7f0b901973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8ffff700 1 -- 172.21.15.150:0/2164196334 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0095f0 con 0x7f0b90105730 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b95236700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b90106080 0x7f0b9019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8ffff700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90197950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c000c00 tx=0x7f0b7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8dffb700 1 -- 172.21.15.150:0/2164196334 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c01f070 con 0x7f0b90105730 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8dffb700 1 -- 172.21.15.150:0/2164196334 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b7c00f460 con 0x7f0b90105730 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9019c980 con 0x7f0b90105730 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b8dffb700 1 -- 172.21.15.150:0/2164196334 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c025680 con 0x7f0b90105730 2024-05-10T04:37:37.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.698+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b90068d20 con 0x7f0b90105730 2024-05-10T04:37:37.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.699+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b901022b0 con 0x7f0b90105730 2024-05-10T04:37:37.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.701+0000 7f0b8dffb700 1 -- 172.21.15.150:0/2164196334 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0b7c023070 con 0x7f0b90105730 2024-05-10T04:37:37.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.702+0000 7f0b8dffb700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0b800a1120 0x7f0b800a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:37.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.702+0000 7f0b94a35700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0b800a1120 0x7f0b800a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:37.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.702+0000 7f0b8dffb700 1 -- 172.21.15.150:0/2164196334 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0b7c0cfb90 con 0x7f0b90105730 2024-05-10T04:37:37.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.702+0000 7f0b94a35700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0b800a1120 0x7f0b800a3630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005950 tx=0x7f0b8400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:37.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.705+0000 7f0b8dffb700 1 -- 172.21.15.150:0/2164196334 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0b7c0997b0 con 0x7f0b90105730 2024-05-10T04:37:37.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.900+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b9004edc0 con 0x7f0b90105730 2024-05-10T04:37:37.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.901+0000 7f0b8dffb700 1 -- 172.21.15.150:0/2164196334 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18455 (secure 0 0 0) 0x7f0b7c09cdd0 con 0x7f0b90105730 2024-05-10T04:37:37.903 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:37.903 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":78,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:37:36.584832+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:37:37.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0b800a1120 msgr2=0x7f0b800a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:37.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0b800a1120 0x7f0b800a3630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005950 tx=0x7f0b8400a300 comp rx=0 tx=0).stop 2024-05-10T04:37:37.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 msgr2=0x7f0b90197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:37.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90197950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c000c00 tx=0x7f0b7c004730 comp rx=0 tx=0).stop 2024-05-10T04:37:37.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 shutdown_connections 2024-05-10T04:37:37.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0b800a1120 0x7f0b800a3630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b90106080 0x7f0b9019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b90100e00 0x7f0b901973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 --2- 172.21.15.150:0/2164196334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b90105730 0x7f0b90197950 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:37.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 >> 172.21.15.150:0/2164196334 conn(0x7f0b900fc7a0 msgr2=0x7f0b90106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:37.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 shutdown_connections 2024-05-10T04:37:37.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:37.904+0000 7f0b96c99700 1 -- 172.21.15.150:0/2164196334 wait complete. 2024-05-10T04:37:38.272 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:38 smithi150 ceph-mon[42404]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:38.272 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:38 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2164196334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:37:38.274 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T04:37:38.274 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-10T04:37:38.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:38 smithi181 ceph-mon[47314]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:38.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:38 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2164196334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:37:38.494 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:37:38.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:38 smithi161 ceph-mon[46912]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:38.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:38 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2164196334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 -- 172.21.15.150:0/710636781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470100e00 msgr2=0x7f7470101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 --2- 172.21.15.150:0/710636781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470100e00 0x7f7470101210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7464009990 tx=0x7f7464009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 -- 172.21.15.150:0/710636781 shutdown_connections 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 --2- 172.21.15.150:0/710636781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470106080 0x7f747010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 --2- 172.21.15.150:0/710636781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470100e00 0x7f7470101210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 --2- 172.21.15.150:0/710636781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7470105730 0x7f7470105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 -- 172.21.15.150:0/710636781 >> 172.21.15.150:0/710636781 conn(0x7f74700fc7a0 msgr2=0x7f74700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:39.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 -- 172.21.15.150:0/710636781 shutdown_connections 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.416+0000 7f7476ce6700 1 -- 172.21.15.150:0/710636781 wait complete. 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 Processor -- start 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 -- start start 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470100e00 0x7f74701995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7470105730 0x7f7470199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470106080 0x7f74701a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747019a200 con 0x7f7470106080 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74701a49b0 con 0x7f7470105730 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7476ce6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74701a4af0 con 0x7f7470100e00 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7475283700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470106080 0x7f74701a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7475283700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470106080 0x7f74701a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59050/0 (socket says 172.21.15.150:59050) 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7475283700 1 -- 172.21.15.150:0/2251157897 learned_addr learned my addr 172.21.15.150:0/2251157897 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.417+0000 7f7474a82700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470100e00 0x7f74701995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:39.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f746ffff700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7470105730 0x7f7470199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7475283700 1 -- 172.21.15.150:0/2251157897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470100e00 msgr2=0x7f74701995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7475283700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470100e00 0x7f74701995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7475283700 1 -- 172.21.15.150:0/2251157897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7470105730 msgr2=0x7f7470199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7475283700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7470105730 0x7f7470199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7475283700 1 -- 172.21.15.150:0/2251157897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640095f0 con 0x7f7470106080 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7475283700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470106080 0x7f74701a4470 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f746800add0 tx=0x7f746800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f746dffb700 1 -- 172.21.15.150:0/2251157897 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746800f070 con 0x7f7470106080 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f746dffb700 1 -- 172.21.15.150:0/2251157897 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7468007b30 con 0x7f7470106080 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f746dffb700 1 -- 172.21.15.150:0/2251157897 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468005280 con 0x7f7470106080 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74701a4dd0 con 0x7f7470106080 2024-05-10T04:37:39.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.418+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74701a51c0 con 0x7f7470106080 2024-05-10T04:37:39.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.419+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74701090d0 con 0x7f7470106080 2024-05-10T04:37:39.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.420+0000 7f746dffb700 1 -- 172.21.15.150:0/2251157897 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f746800c890 con 0x7f7470106080 2024-05-10T04:37:39.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.421+0000 7f746dffb700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f74600a11a0 0x7f74600a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:39.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.421+0000 7f746dffb700 1 -- 172.21.15.150:0/2251157897 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f74680cb8c0 con 0x7f7470106080 2024-05-10T04:37:39.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.421+0000 7f7474a82700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f74600a11a0 0x7f74600a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:39.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.421+0000 7f7474a82700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f74600a11a0 0x7f74600a36b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7464006010 tx=0x7f7464003740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:39.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.426+0000 7f746dffb700 1 -- 172.21.15.150:0/2251157897 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7468095580 con 0x7f7470106080 2024-05-10T04:37:39.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.617+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f747004edc0 con 0x7f7470106080 2024-05-10T04:37:39.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.618+0000 7f746dffb700 1 -- 172.21.15.150:0/2251157897 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7468098ba0 con 0x7f7470106080 2024-05-10T04:37:39.620 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:37:39.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f74600a11a0 msgr2=0x7f74600a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:39.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f74600a11a0 0x7f74600a36b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7464006010 tx=0x7f7464003740 comp rx=0 tx=0).stop 2024-05-10T04:37:39.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470106080 msgr2=0x7f74701a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:39.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470106080 0x7f74701a4470 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f746800add0 tx=0x7f746800c330 comp rx=0 tx=0).stop 2024-05-10T04:37:39.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 shutdown_connections 2024-05-10T04:37:39.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470106080 0x7f74701a4470 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f74600a11a0 0x7f74600a36b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470100e00 0x7f74701995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 --2- 172.21.15.150:0/2251157897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7470105730 0x7f7470199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:39.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.620+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 >> 172.21.15.150:0/2251157897 conn(0x7f74700fc7a0 msgr2=0x7f7470106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:39.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.621+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 shutdown_connections 2024-05-10T04:37:39.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:39.621+0000 7f7476ce6700 1 -- 172.21.15.150:0/2251157897 wait complete. 2024-05-10T04:37:39.983 INFO:tasks.cephadm:Adding mds.a on smithi150 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.d on smithi150 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.g on smithi150 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.j on smithi150 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.b on smithi161 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.e on smithi161 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.h on smithi161 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.k on smithi161 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.c on smithi181 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.f on smithi181 2024-05-10T04:37:39.984 INFO:tasks.cephadm:Adding mds.i on smithi181 2024-05-10T04:37:39.985 INFO:tasks.cephadm:Adding mds.l on smithi181 2024-05-10T04:37:39.985 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l' 2024-05-10T04:37:40.222 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:37:40.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:40 smithi181 ceph-mon[47314]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:40.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:40 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2251157897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:37:40.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:40 smithi150 ceph-mon[42404]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:40.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:40 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2251157897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:37:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:40 smithi161 ceph-mon[46912]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:40 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2251157897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:37:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 -- 172.21.15.181:0/1359557672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102160 msgr2=0x7f84b8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 --2- 172.21.15.181:0/1359557672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102160 0x7f84b8102570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009990 tx=0x7f84ac009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 -- 172.21.15.181:0/1359557672 shutdown_connections 2024-05-10T04:37:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 --2- 172.21.15.181:0/1359557672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8109b70 0x7f84b810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 --2- 172.21.15.181:0/1359557672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102160 0x7f84b8102570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 --2- 172.21.15.181:0/1359557672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b8102b40 0x7f84b81094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 -- 172.21.15.181:0/1359557672 >> 172.21.15.181:0/1359557672 conn(0x7f84b80fb950 msgr2=0x7f84b80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:41.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 -- 172.21.15.181:0/1359557672 shutdown_connections 2024-05-10T04:37:41.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.288+0000 7f84bd126700 1 -- 172.21.15.181:0/1359557672 wait complete. 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 Processor -- start 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 -- start start 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8102160 0x7f84b819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102b40 0x7f84b819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b8109b70 0x7f84b819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b819bbf0 con 0x7f84b8109b70 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b819bd30 con 0x7f84b8102160 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84bd126700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b819bef0 con 0x7f84b8102b40 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84b659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102b40 0x7f84b819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84b659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102b40 0x7f84b819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54442/0 (socket says 172.21.15.181:54442) 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84b659c700 1 -- 172.21.15.181:0/2086108828 learned_addr learned my addr 172.21.15.181:0/2086108828 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-05-10T04:37:41.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.289+0000 7f84b759e700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b8109b70 0x7f84b819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84b6d9d700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8102160 0x7f84b819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84b659c700 1 -- 172.21.15.181:0/2086108828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8102160 msgr2=0x7f84b819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84b659c700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8102160 0x7f84b819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84b659c700 1 -- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b8109b70 msgr2=0x7f84b819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84b659c700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b8109b70 0x7f84b819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84b659c700 1 -- 172.21.15.181:0/2086108828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0095f0 con 0x7f84b8102b40 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84b659c700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102b40 0x7f84b819a870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c9e0 tx=0x7f84a800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f849ffff700 1 -- 172.21.15.181:0/2086108828 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8007aa0 con 0x7f84b8102b40 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b819c1d0 con 0x7f84b8102b40 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f849ffff700 1 -- 172.21.15.181:0/2086108828 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a8007c00 con 0x7f84b8102b40 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f849ffff700 1 -- 172.21.15.181:0/2086108828 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8018660 con 0x7f84b8102b40 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.290+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b81a2a40 con 0x7f84b8102b40 2024-05-10T04:37:41.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.291+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b80fd500 con 0x7f84b8102b40 2024-05-10T04:37:41.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.294+0000 7f849ffff700 1 -- 172.21.15.181:0/2086108828 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f84a800f450 con 0x7f84b8102b40 2024-05-10T04:37:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.294+0000 7f849ffff700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84a00a1120 0x7f84a00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.295+0000 7f849ffff700 1 -- 172.21.15.181:0/2086108828 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f84a80cc700 con 0x7f84b8102b40 2024-05-10T04:37:41.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.295+0000 7f84b6d9d700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84a00a1120 0x7f84a00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:41.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.295+0000 7f84b6d9d700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84a00a1120 0x7f84a00a3630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f84ac003730 tx=0x7f84ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:41.299 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.298+0000 7f849ffff700 1 -- 172.21.15.181:0/2086108828 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f84a8096320 con 0x7f84b8102b40 2024-05-10T04:37:41.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.500+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f84b810f820 con 0x7f84a00a1120 2024-05-10T04:37:41.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.509+0000 7f849ffff700 1 -- 172.21.15.181:0/2086108828 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f84b810f820 con 0x7f84a00a1120 2024-05-10T04:37:41.511 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mds.all update... 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84a00a1120 msgr2=0x7f84a00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84a00a1120 0x7f84a00a3630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f84ac003730 tx=0x7f84ac01a040 comp rx=0 tx=0).stop 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102b40 msgr2=0x7f84b819a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102b40 0x7f84b819a870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c9e0 tx=0x7f84a800ccf0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 shutdown_connections 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b8109b70 0x7f84b819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84b8102160 0x7f84b819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84a00a1120 0x7f84a00a3630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 --2- 172.21.15.181:0/2086108828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84b8102b40 0x7f84b819a870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 >> 172.21.15.181:0/2086108828 conn(0x7f84b80fb950 msgr2=0x7f84b8107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 shutdown_connections 2024-05-10T04:37:41.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-05-10T04:37:41.511+0000 7f84bd126700 1 -- 172.21.15.181:0/2086108828 wait complete. 2024-05-10T04:37:41.875 DEBUG:teuthology.orchestra.run.smithi150:mds.a> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.a.service 2024-05-10T04:37:41.878 DEBUG:teuthology.orchestra.run.smithi150:mds.d> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.d.service 2024-05-10T04:37:41.908 INFO:journalctl@ceph.mds.a.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:37:41.908 DEBUG:teuthology.orchestra.run.smithi150:mds.g> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.g.service 2024-05-10T04:37:41.936 INFO:journalctl@ceph.mds.d.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:37:41.936 DEBUG:teuthology.orchestra.run.smithi150:mds.j> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.j.service 2024-05-10T04:37:41.963 INFO:journalctl@ceph.mds.g.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:37:41.964 DEBUG:teuthology.orchestra.run.smithi161:mds.b> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.b.service 2024-05-10T04:37:41.967 DEBUG:teuthology.orchestra.run.smithi161:mds.e> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.e.service 2024-05-10T04:37:41.990 INFO:journalctl@ceph.mds.j.smithi150.stdout:-- Logs begin at Fri 2024-05-10 04:15:48 UTC. -- 2024-05-10T04:37:42.006 INFO:journalctl@ceph.mds.b.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:37:42.007 DEBUG:teuthology.orchestra.run.smithi161:mds.h> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.h.service 2024-05-10T04:37:42.037 INFO:journalctl@ceph.mds.e.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:37:42.038 DEBUG:teuthology.orchestra.run.smithi161:mds.k> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.k.service 2024-05-10T04:37:42.066 INFO:journalctl@ceph.mds.h.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:37:42.067 DEBUG:teuthology.orchestra.run.smithi181:mds.c> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.c.service 2024-05-10T04:37:42.069 DEBUG:teuthology.orchestra.run.smithi181:mds.f> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.f.service 2024-05-10T04:37:42.094 INFO:journalctl@ceph.mds.k.smithi161.stdout:-- Logs begin at Fri 2024-05-10 04:15:49 UTC. -- 2024-05-10T04:37:42.098 INFO:journalctl@ceph.mds.c.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:37:42.100 DEBUG:teuthology.orchestra.run.smithi181:mds.i> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.i.service 2024-05-10T04:37:42.127 INFO:journalctl@ceph.mds.f.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:37:42.128 DEBUG:teuthology.orchestra.run.smithi181:mds.l> sudo journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.l.service 2024-05-10T04:37:42.155 INFO:journalctl@ceph.mds.i.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:37:42.156 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-10T04:37:42.156 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:37:42.184 INFO:journalctl@ceph.mds.l.smithi181.stdout:-- Logs begin at Fri 2024-05-10 04:15:43 UTC. -- 2024-05-10T04:37:42.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 -- 172.21.15.150:0/3786493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c101be0 msgr2=0x7f043c101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 --2- 172.21.15.150:0/3786493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c101be0 0x7f043c101ff0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f042c009990 tx=0x7f042c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 -- 172.21.15.150:0/3786493786 shutdown_connections 2024-05-10T04:37:42.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 --2- 172.21.15.150:0/3786493786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c102580 0x7f043c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 --2- 172.21.15.150:0/3786493786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c100ac0 0x7f043c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 --2- 172.21.15.150:0/3786493786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c101be0 0x7f043c101ff0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 -- 172.21.15.150:0/3786493786 >> 172.21.15.150:0/3786493786 conn(0x7f043c0fc0d0 msgr2=0x7f043c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:42.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 -- 172.21.15.150:0/3786493786 shutdown_connections 2024-05-10T04:37:42.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 -- 172.21.15.150:0/3786493786 wait complete. 2024-05-10T04:37:42.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 Processor -- start 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.298+0000 7f04425a0700 1 -- start start 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04425a0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c100ac0 0x7f043c068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04425a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c101be0 0x7f043c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04425a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c102580 0x7f043c19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04425a0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c19fa20 con 0x7f043c102580 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04425a0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c19fb90 con 0x7f043c100ac0 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04425a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c19fd50 con 0x7f043c101be0 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c102580 0x7f043c19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c102580 0x7f043c19f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59066/0 (socket says 172.21.15.150:59066) 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 -- 172.21.15.150:0/151045509 learned_addr learned my addr 172.21.15.150:0/151045509 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f043b7fe700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c101be0 0x7f043c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f043bfff700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c100ac0 0x7f043c068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 -- 172.21.15.150:0/151045509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c101be0 msgr2=0x7f043c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c101be0 0x7f043c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 -- 172.21.15.150:0/151045509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c100ac0 msgr2=0x7f043c068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c100ac0 0x7f043c068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 -- 172.21.15.150:0/151045509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0095f0 con 0x7f043c102580 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f0440b3d700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c102580 0x7f043c19f320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0434007ee0 tx=0x7f043400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04397fa700 1 -- 172.21.15.150:0/151045509 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434017070 con 0x7f043c102580 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c19ffd0 con 0x7f043c102580 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04397fa700 1 -- 172.21.15.150:0/151045509 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f043400f040 con 0x7f043c102580 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.299+0000 7f04397fa700 1 -- 172.21.15.150:0/151045509 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434005580 con 0x7f043c102580 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.300+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c1a0480 con 0x7f043c102580 2024-05-10T04:37:42.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.300+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c107cd0 con 0x7f043c102580 2024-05-10T04:37:42.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.302+0000 7f04397fa700 1 -- 172.21.15.150:0/151045509 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f043400db20 con 0x7f043c102580 2024-05-10T04:37:42.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.303+0000 7f04397fa700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04240a1120 0x7f04240a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.303+0000 7f043bfff700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04240a1120 0x7f04240a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.303+0000 7f04397fa700 1 -- 172.21.15.150:0/151045509 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f04340cc120 con 0x7f043c102580 2024-05-10T04:37:42.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.303+0000 7f043bfff700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04240a1120 0x7f04240a3630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f043c101ae0 tx=0x7f0430007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:42.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.306+0000 7f04397fa700 1 -- 172.21.15.150:0/151045509 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0434095d50 con 0x7f043c102580 2024-05-10T04:37:42.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.506+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f043c0694d0 con 0x7f043c102580 2024-05-10T04:37:42.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.507+0000 7f04397fa700 1 -- 172.21.15.150:0/151045509 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18455 (secure 0 0 0) 0x7f0434099370 con 0x7f043c102580 2024-05-10T04:37:42.508 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:42.509 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":78,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:37:36.584832+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:37:42.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04240a1120 msgr2=0x7f04240a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04240a1120 0x7f04240a3630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f043c101ae0 tx=0x7f0430007400 comp rx=0 tx=0).stop 2024-05-10T04:37:42.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c102580 msgr2=0x7f043c19f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c102580 0x7f043c19f320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0434007ee0 tx=0x7f043400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 shutdown_connections 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043c102580 0x7f043c19f320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f043c100ac0 0x7f043c068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f04240a1120 0x7f04240a3630 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 --2- 172.21.15.150:0/151045509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c101be0 0x7f043c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 >> 172.21.15.150:0/151045509 conn(0x7f043c0fc0d0 msgr2=0x7f043c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 shutdown_connections 2024-05-10T04:37:42.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.509+0000 7f04425a0700 1 -- 172.21.15.150:0/151045509 wait complete. 2024-05-10T04:37:42.522 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:42.523 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:42 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:42.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T04:37:42.525 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:37:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:42.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:42.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:42.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:42.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:42.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:42 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:42.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 -- 172.21.15.150:0/2044510860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1013e0 msgr2=0x7fa06c1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 --2- 172.21.15.150:0/2044510860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1013e0 0x7fa06c1017f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009990 tx=0x7fa05c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 -- 172.21.15.150:0/2044510860 shutdown_connections 2024-05-10T04:37:42.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 --2- 172.21.15.150:0/2044510860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c101d80 0x7fa06c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 --2- 172.21.15.150:0/2044510860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa06c1002c0 0x7fa06c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 --2- 172.21.15.150:0/2044510860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1013e0 0x7fa06c1017f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 -- 172.21.15.150:0/2044510860 >> 172.21.15.150:0/2044510860 conn(0x7fa06c0fb8f0 msgr2=0x7fa06c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:42.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 -- 172.21.15.150:0/2044510860 shutdown_connections 2024-05-10T04:37:42.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.659+0000 7fa073de0700 1 -- 172.21.15.150:0/2044510860 wait complete. 2024-05-10T04:37:42.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 Processor -- start 2024-05-10T04:37:42.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 -- start start 2024-05-10T04:37:42.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1002c0 0x7fa06c196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa06c1013e0 0x7fa06c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c101d80 0x7fa06c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c19e900 con 0x7fa06c1002c0 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c19ea40 con 0x7fa06c1013e0 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa073de0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c19ebd0 con 0x7fa06c101d80 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1002c0 0x7fa06c196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1002c0 0x7fa06c196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59092/0 (socket says 172.21.15.150:59092) 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 -- 172.21.15.150:0/3318917187 learned_addr learned my addr 172.21.15.150:0/3318917187 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa07237d700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c101d80 0x7fa06c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa07137b700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa06c1013e0 0x7fa06c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 -- 172.21.15.150:0/3318917187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c101d80 msgr2=0x7fa06c19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c101d80 0x7fa06c19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 -- 172.21.15.150:0/3318917187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa06c1013e0 msgr2=0x7fa06c1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa06c1013e0 0x7fa06c1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 -- 172.21.15.150:0/3318917187 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c0095f0 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa07137b700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa06c1013e0 0x7fa06c1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa071b7c700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1002c0 0x7fa06c196e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa06800d980 tx=0x7fa06800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.660+0000 7fa062ffd700 1 -- 172.21.15.150:0/3318917187 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068004220 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.661+0000 7fa062ffd700 1 -- 172.21.15.150:0/3318917187 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa068004380 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.661+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c19eeb0 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.661+0000 7fa062ffd700 1 -- 172.21.15.150:0/3318917187 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068003bf0 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.661+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c19f2e0 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.661+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c107f00 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.663+0000 7fa062ffd700 1 -- 172.21.15.150:0/3318917187 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa068003d50 con 0x7fa06c1002c0 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.663+0000 7fa062ffd700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0580a11c0 0x7fa0580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:42.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.663+0000 7fa062ffd700 1 -- 172.21.15.150:0/3318917187 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0680cc780 con 0x7fa06c1002c0 2024-05-10T04:37:42.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.663+0000 7fa07137b700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0580a11c0 0x7fa0580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:42.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.664+0000 7fa07137b700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0580a11c0 0x7fa0580a36d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0038c0 tx=0x7fa05c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:42.670 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.667+0000 7fa062ffd700 1 -- 172.21.15.150:0/3318917187 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0680963b0 con 0x7fa06c1002c0 2024-05-10T04:37:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:37:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:42.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:37:42.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:42.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:42.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:42.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:42 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:42.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.863+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa06c10baa0 con 0x7fa06c1002c0 2024-05-10T04:37:42.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.864+0000 7fa062ffd700 1 -- 172.21.15.150:0/3318917187 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0680999d0 con 0x7fa06c1002c0 2024-05-10T04:37:42.865 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0580a11c0 msgr2=0x7fa0580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0580a11c0 0x7fa0580a36d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0038c0 tx=0x7fa05c01a040 comp rx=0 tx=0).stop 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1002c0 msgr2=0x7fa06c196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1002c0 0x7fa06c196e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa06800d980 tx=0x7fa06800dc90 comp rx=0 tx=0).stop 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 shutdown_connections 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0580a11c0 0x7fa0580a36d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa06c101d80 0x7fa06c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06c1002c0 0x7fa06c196e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 --2- 172.21.15.150:0/3318917187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa06c1013e0 0x7fa06c1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:42.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 >> 172.21.15.150:0/3318917187 conn(0x7fa06c0fb8f0 msgr2=0x7fa06c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:42.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 shutdown_connections 2024-05-10T04:37:42.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:42.866+0000 7fa073de0700 1 -- 172.21.15.150:0/3318917187 wait complete. 2024-05-10T04:37:42.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 -- 172.21.15.150:0/2639514352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 msgr2=0x7ff8d0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 --2- 172.21.15.150:0/2639514352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0100ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009990 tx=0x7ff8c0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 -- 172.21.15.150:0/2639514352 shutdown_connections 2024-05-10T04:37:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 --2- 172.21.15.150:0/2639514352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8d0102590 0x7ff8d0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 --2- 172.21.15.150:0/2639514352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0100ee0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 --2- 172.21.15.150:0/2639514352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8d0101bf0 0x7ff8d0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 -- 172.21.15.150:0/2639514352 >> 172.21.15.150:0/2639514352 conn(0x7ff8d00fc0e0 msgr2=0x7ff8d00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 -- 172.21.15.150:0/2639514352 shutdown_connections 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.012+0000 7ff8d5931700 1 -- 172.21.15.150:0/2639514352 wait complete. 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 Processor -- start 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 -- start start 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8d0101bf0 0x7ff8d019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8d0102590 0x7ff8d019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d019fbd0 con 0x7ff8d0100ad0 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d019fd40 con 0x7ff8d0101bf0 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8d5931700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d019ff00 con 0x7ff8d0102590 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8ceffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8ceffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0073ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59122/0 (socket says 172.21.15.150:59122) 2024-05-10T04:37:43.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8ceffd700 1 -- 172.21.15.150:0/1772363620 learned_addr learned my addr 172.21.15.150:0/1772363620 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.013+0000 7ff8cf7fe700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8d0102590 0x7ff8d019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8ce7fc700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8d0101bf0 0x7ff8d019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8ceffd700 1 -- 172.21.15.150:0/1772363620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8d0102590 msgr2=0x7ff8d019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8ceffd700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8d0102590 0x7ff8d019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8ceffd700 1 -- 172.21.15.150:0/1772363620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8d0101bf0 msgr2=0x7ff8d019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8ceffd700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8d0101bf0 0x7ff8d019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8ceffd700 1 -- 172.21.15.150:0/1772363620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c4009520 con 0x7ff8d0100ad0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8ceffd700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0073ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00098e0 tx=0x7ff8c0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8d492f700 1 -- 172.21.15.150:0/1772363620 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c001d070 con 0x7ff8d0100ad0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8d492f700 1 -- 172.21.15.150:0/1772363620 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8c0022470 con 0x7ff8d0100ad0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c8009c40 con 0x7ff8d0100ad0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8d492f700 1 -- 172.21.15.150:0/1772363620 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0017650 con 0x7ff8d0100ad0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.014+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c00095f0 con 0x7ff8d0100ad0 2024-05-10T04:37:43.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.015+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d0107db0 con 0x7ff8d0100ad0 2024-05-10T04:37:43.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.016+0000 7ff8d492f700 1 -- 172.21.15.150:0/1772363620 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff8c000fad0 con 0x7ff8d0100ad0 2024-05-10T04:37:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.017+0000 7ff8d492f700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff8b80a1120 0x7ff8b80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.017+0000 7ff8d492f700 1 -- 172.21.15.150:0/1772363620 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8c00cddf0 con 0x7ff8d0100ad0 2024-05-10T04:37:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.017+0000 7ff8ce7fc700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff8b80a1120 0x7ff8b80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:43.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.017+0000 7ff8ce7fc700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff8b80a1120 0x7ff8b80a3630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0073870 tx=0x7ff8c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:43.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.021+0000 7ff8d492f700 1 -- 172.21.15.150:0/1772363620 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff8c0097ab0 con 0x7ff8d0100ad0 2024-05-10T04:37:43.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.261+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8d0068730 con 0x7ff8d0100ad0 2024-05-10T04:37:43.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.261+0000 7ff8d492f700 1 -- 172.21.15.150:0/1772363620 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7ff8c009b0d0 con 0x7ff8d0100ad0 2024-05-10T04:37:43.263 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:43.263 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff8b80a1120 msgr2=0x7ff8b80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff8b80a1120 0x7ff8b80a3630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0073870 tx=0x7ff8c4009260 comp rx=0 tx=0).stop 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 msgr2=0x7ff8d0073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0073ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8c00098e0 tx=0x7ff8c0009580 comp rx=0 tx=0).stop 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 shutdown_connections 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff8b80a1120 0x7ff8b80a3630 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8d0102590 0x7ff8d019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8d0100ad0 0x7ff8d0073ed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 --2- 172.21.15.150:0/1772363620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8d0101bf0 0x7ff8d019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 >> 172.21.15.150:0/1772363620 conn(0x7ff8d00fc0e0 msgr2=0x7ff8d0104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 shutdown_connections 2024-05-10T04:37:43.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:43.263+0000 7ff8d5931700 1 -- 172.21.15.150:0/1772363620 wait complete. 2024-05-10T04:37:43.266 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 1 2024-05-10T04:37:43.364 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:37:43 smithi181 systemd[1]: Starting Ceph mds.l for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:43.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:43 smithi181 ceph-mon[47314]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:43 smithi181 ceph-mon[47314]: Saving service mds.all spec with placement smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:43 smithi181 ceph-mon[47314]: Deploying daemon mds.l on smithi181 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:43 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/151045509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:43 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3318917187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:43 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1772363620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:37:43 smithi181 bash[82318]: dfe76ae50d6bd6dd3802984eb7a5f05785c9b8615922a8685158f832f0b802c6 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:37:43 smithi181 systemd[1]: Started Ceph mds.l for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:37:43 smithi181 ceph-mds[82356]: -- 172.21.15.181:0/1829762353 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fbcc7aa1a0 con 0x55fbcc80ad00 2024-05-10T04:37:43.655 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:37:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-l[82333]: starting mds.l at 2024-05-10T04:37:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:43 smithi150 ceph-mon[42404]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:43 smithi150 ceph-mon[42404]: Saving service mds.all spec with placement smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:37:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:43 smithi150 ceph-mon[42404]: Deploying daemon mds.l on smithi181 2024-05-10T04:37:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:43 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/151045509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:37:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:43 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3318917187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:37:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:43 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1772363620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:43 smithi161 ceph-mon[46912]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:37:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:43 smithi161 ceph-mon[46912]: Saving service mds.all spec with placement smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:37:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:43 smithi161 ceph-mon[46912]: Deploying daemon mds.l on smithi181 2024-05-10T04:37:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:43 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/151045509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:37:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:43 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3318917187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:37:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:43 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1772363620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:44.631 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:44.631 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] up:boot 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: fsmap 1 up:standby 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:44.632 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:44 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:44.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] up:boot 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: fsmap 1 up:standby 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:44.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:44 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] up:boot 2024-05-10T04:37:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: fsmap 1 up:standby 2024-05-10T04:37:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:37:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:44.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:44.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:44.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:44 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:45.142 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:37:44 smithi161 systemd[1]: Starting Ceph mds.k for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:45.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:45.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 -- 172.21.15.150:0/1659466536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac101c00 msgr2=0x7f46ac102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:45.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/1659466536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac101c00 0x7f46ac102010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4698009990 tx=0x7f4698009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 -- 172.21.15.150:0/1659466536 shutdown_connections 2024-05-10T04:37:45.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/1659466536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46ac1025a0 0x7f46ac108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/1659466536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46ac100ae0 0x7f46ac100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/1659466536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac101c00 0x7f46ac102010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 -- 172.21.15.150:0/1659466536 >> 172.21.15.150:0/1659466536 conn(0x7f46ac0fc110 msgr2=0x7f46ac0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 -- 172.21.15.150:0/1659466536 shutdown_connections 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.432+0000 7f46b2b0a700 1 -- 172.21.15.150:0/1659466536 wait complete. 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 Processor -- start 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 -- start start 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46ac100ae0 0x7f46ac073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46ac101c00 0x7f46ac19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac1025a0 0x7f46ac19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac19fb20 con 0x7f46ac1025a0 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac19fc90 con 0x7f46ac101c00 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b2b0a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac19fe50 con 0x7f46ac100ae0 2024-05-10T04:37:45.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b10a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac1025a0 0x7f46ac19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.433+0000 7f46b10a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac1025a0 0x7f46ac19f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59140/0 (socket says 172.21.15.150:59140) 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b10a7700 1 -- 172.21.15.150:0/2828060839 learned_addr learned my addr 172.21.15.150:0/2828060839 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b08a6700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46ac100ae0 0x7f46ac073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46abfff700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46ac101c00 0x7f46ac19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b10a7700 1 -- 172.21.15.150:0/2828060839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46ac100ae0 msgr2=0x7f46ac073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b10a7700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46ac100ae0 0x7f46ac073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b10a7700 1 -- 172.21.15.150:0/2828060839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46ac101c00 msgr2=0x7f46ac19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b10a7700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46ac101c00 0x7f46ac19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b10a7700 1 -- 172.21.15.150:0/2828060839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980095f0 con 0x7f46ac1025a0 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46abfff700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46ac101c00 0x7f46ac19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b08a6700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46ac100ae0 0x7f46ac073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:37:45.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b10a7700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac1025a0 0x7f46ac19f3d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f46a4007ee0 tx=0x7f46a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:45.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46a9ffb700 1 -- 172.21.15.150:0/2828060839 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4017070 con 0x7f46ac1025a0 2024-05-10T04:37:45.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac1a00d0 con 0x7f46ac1025a0 2024-05-10T04:37:45.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46a9ffb700 1 -- 172.21.15.150:0/2828060839 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46a400f040 con 0x7f46ac1025a0 2024-05-10T04:37:45.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46a9ffb700 1 -- 172.21.15.150:0/2828060839 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4005740 con 0x7f46ac1025a0 2024-05-10T04:37:45.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.434+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac1a05e0 con 0x7f46ac1025a0 2024-05-10T04:37:45.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.436+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac107cf0 con 0x7f46ac1025a0 2024-05-10T04:37:45.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.437+0000 7f46a9ffb700 1 -- 172.21.15.150:0/2828060839 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f46a40058a0 con 0x7f46ac1025a0 2024-05-10T04:37:45.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.437+0000 7f46a9ffb700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f469c0a1120 0x7f469c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:45.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.438+0000 7f46b08a6700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f469c0a1120 0x7f469c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:45.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.438+0000 7f46a9ffb700 1 -- 172.21.15.150:0/2828060839 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f46a40cbc30 con 0x7f46ac1025a0 2024-05-10T04:37:45.441 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.438+0000 7f46b08a6700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f469c0a1120 0x7f469c0a3630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46ac101b00 tx=0x7f46a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:45.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.443+0000 7f46a9ffb700 1 -- 172.21.15.150:0/2828060839 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f46a4095860 con 0x7f46ac1025a0 2024-05-10T04:37:45.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:45 smithi161 ceph-mon[46912]: Deploying daemon mds.k on smithi161 2024-05-10T04:37:45.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:45.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:45 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:45.586 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:37:45 smithi161 ceph-mds[82710]: -- 172.21.15.161:0/4138913759 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ce24e01a0 con 0x563ce2540d00 2024-05-10T04:37:45.586 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:37:45 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-k[82687]: starting mds.k at 2024-05-10T04:37:45.586 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:37:45 smithi161 bash[82672]: 6d66cdf37c456879b809a6c3964fd1e851cf5b0fd353c365dd88a7fd946d9690 2024-05-10T04:37:45.586 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:37:45 smithi161 systemd[1]: Started Ceph mds.k for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:45.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:45 smithi181 ceph-mon[47314]: Deploying daemon mds.k on smithi161 2024-05-10T04:37:45.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:45.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:45 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:45.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:45 smithi150 ceph-mon[42404]: Deploying daemon mds.k on smithi161 2024-05-10T04:37:45.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:45.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:45 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:45.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.701+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46ac10c330 con 0x7f46ac1025a0 2024-05-10T04:37:45.703 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.702+0000 7f46a9ffb700 1 -- 172.21.15.150:0/2828060839 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f46a4098e80 con 0x7f46ac1025a0 2024-05-10T04:37:45.704 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:45.704 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-10T04:37:45.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f469c0a1120 msgr2=0x7f469c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:45.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f469c0a1120 0x7f469c0a3630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46ac101b00 tx=0x7f46a0009260 comp rx=0 tx=0).stop 2024-05-10T04:37:45.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac1025a0 msgr2=0x7f46ac19f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:45.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac1025a0 0x7f46ac19f3d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f46a4007ee0 tx=0x7f46a400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 shutdown_connections 2024-05-10T04:37:45.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac1025a0 0x7f46ac19f3d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f469c0a1120 0x7f469c0a3630 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46ac100ae0 0x7f46ac073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 --2- 172.21.15.150:0/2828060839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46ac101c00 0x7f46ac19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:45.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 >> 172.21.15.150:0/2828060839 conn(0x7f46ac0fc110 msgr2=0x7f46ac104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:45.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 shutdown_connections 2024-05-10T04:37:45.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:45.704+0000 7f46b2b0a700 1 -- 172.21.15.150:0/2828060839 wait complete. 2024-05-10T04:37:45.709 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 3 2024-05-10T04:37:46.645 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] up:boot 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: fsmap 2 up:standby 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:46.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: Deploying daemon mds.j on smithi150 2024-05-10T04:37:46.647 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:46 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2828060839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:46.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:46.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] up:boot 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: fsmap 2 up:standby 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: Deploying daemon mds.j on smithi150 2024-05-10T04:37:46.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:46 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2828060839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:46.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:46.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] up:boot 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: fsmap 2 up:standby 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: Deploying daemon mds.j on smithi150 2024-05-10T04:37:46.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:46 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2828060839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:47.411 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:37:47 smithi150 bash[91840]: 397e3b7dc3e7489626f38acaa6756a1bfb5968620bc9140ec4e4afde1a5a4b6a 2024-05-10T04:37:47.411 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:37:47 smithi150 systemd[1]: Started Ceph mds.j for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:47.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:47.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 -- 172.21.15.150:0/1650542573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c100ac0 msgr2=0x7f303c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:47.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 --2- 172.21.15.150:0/1650542573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c100ac0 0x7f303c100ed0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3034009990 tx=0x7f3034009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:47.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 -- 172.21.15.150:0/1650542573 shutdown_connections 2024-05-10T04:37:47.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 --2- 172.21.15.150:0/1650542573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f303c102580 0x7f303c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:47.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 --2- 172.21.15.150:0/1650542573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c100ac0 0x7f303c100ed0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:47.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 --2- 172.21.15.150:0/1650542573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f303c101be0 0x7f303c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:47.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 -- 172.21.15.150:0/1650542573 >> 172.21.15.150:0/1650542573 conn(0x7f303c0fc0d0 msgr2=0x7f303c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:47.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 -- 172.21.15.150:0/1650542573 shutdown_connections 2024-05-10T04:37:47.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.860+0000 7f3041ed5700 1 -- 172.21.15.150:0/1650542573 wait complete. 2024-05-10T04:37:47.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 Processor -- start 2024-05-10T04:37:47.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 -- start start 2024-05-10T04:37:47.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f303c100ac0 0x7f303c104880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:47.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c101be0 0x7f303c102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:47.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f303c102580 0x7f303c103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:47.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c103b40 con 0x7f303c101be0 2024-05-10T04:37:47.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c103cb0 con 0x7f303c102580 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f3041ed5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c1a20c0 con 0x7f303c100ac0 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f303affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c101be0 0x7f303c102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f303affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c101be0 0x7f303c102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60444/0 (socket says 172.21.15.150:60444) 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f303affd700 1 -- 172.21.15.150:0/289163449 learned_addr learned my addr 172.21.15.150:0/289163449 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.861+0000 7f303bfff700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f303c102580 0x7f303c103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303b7fe700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f303c100ac0 0x7f303c104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303affd700 1 -- 172.21.15.150:0/289163449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f303c100ac0 msgr2=0x7f303c104880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303affd700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f303c100ac0 0x7f303c104880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303affd700 1 -- 172.21.15.150:0/289163449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f303c102580 msgr2=0x7f303c103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303affd700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f303c102580 0x7f303c103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303affd700 1 -- 172.21.15.150:0/289163449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340095f0 con 0x7f303c101be0 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303b7fe700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f303c100ac0 0x7f303c104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:37:47.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f303affd700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c101be0 0x7f303c102f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f302400e9a0 tx=0x7f302400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:47.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f3038ff9700 1 -- 172.21.15.150:0/289163449 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024009790 con 0x7f303c101be0 2024-05-10T04:37:47.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c1a2320 con 0x7f303c101be0 2024-05-10T04:37:47.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f3038ff9700 1 -- 172.21.15.150:0/289163449 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f302400cbd0 con 0x7f303c101be0 2024-05-10T04:37:47.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f3038ff9700 1 -- 172.21.15.150:0/289163449 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024018680 con 0x7f303c101be0 2024-05-10T04:37:47.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.862+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c1a2850 con 0x7f303c101be0 2024-05-10T04:37:47.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.863+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303c0694d0 con 0x7f303c101be0 2024-05-10T04:37:47.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.865+0000 7f3038ff9700 1 -- 172.21.15.150:0/289163449 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f302400cd40 con 0x7f303c101be0 2024-05-10T04:37:47.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.866+0000 7f3038ff9700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f30280a1120 0x7f30280a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:47.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.866+0000 7f303b7fe700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f30280a1120 0x7f30280a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:47.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.866+0000 7f3038ff9700 1 -- 172.21.15.150:0/289163449 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3024014070 con 0x7f303c101be0 2024-05-10T04:37:47.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.866+0000 7f303b7fe700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f30280a1120 0x7f30280a3630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f303400b7b0 tx=0x7f303401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:47.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:47.871+0000 7f3038ff9700 1 -- 172.21.15.150:0/289163449 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3024052d60 con 0x7f303c101be0 2024-05-10T04:37:48.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.135+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f303c10c2f0 con 0x7f303c101be0 2024-05-10T04:37:48.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.135+0000 7f3038ff9700 1 -- 172.21.15.150:0/289163449 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f303c10c2f0 con 0x7f303c101be0 2024-05-10T04:37:48.137 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:48.137 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-10T04:37:48.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f30280a1120 msgr2=0x7f30280a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:48.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f30280a1120 0x7f30280a3630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f303400b7b0 tx=0x7f303401a040 comp rx=0 tx=0).stop 2024-05-10T04:37:48.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c101be0 msgr2=0x7f303c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:48.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c101be0 0x7f303c102f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f302400e9a0 tx=0x7f302400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:37:48.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 shutdown_connections 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f303c102580 0x7f303c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f30280a1120 0x7f30280a3630 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f303c100ac0 0x7f303c104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 --2- 172.21.15.150:0/289163449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f303c101be0 0x7f303c102f20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 >> 172.21.15.150:0/289163449 conn(0x7f303c0fc0d0 msgr2=0x7f303c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 shutdown_connections 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:48.138+0000 7f3041ed5700 1 -- 172.21.15.150:0/289163449 wait complete. 2024-05-10T04:37:48.140 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 4 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: Deploying daemon mds.d on smithi150 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:48.313 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.150:6834/2690665605,v1:172.21.15.150:6835/2690665605] up:boot 2024-05-10T04:37:48.314 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: fsmap 3 up:standby 2024-05-10T04:37:48.314 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:48 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T04:37:48.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: Deploying daemon mds.d on smithi150 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.150:6834/2690665605,v1:172.21.15.150:6835/2690665605] up:boot 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: fsmap 3 up:standby 2024-05-10T04:37:48.336 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:48 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T04:37:48.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: Deploying daemon mds.d on smithi150 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.150:6834/2690665605,v1:172.21.15.150:6835/2690665605] up:boot 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: fsmap 3 up:standby 2024-05-10T04:37:48.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:48 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-10T04:37:49.116 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:37:48 smithi150 bash[92230]: d71287ddeb469308ff713c0e1b2d412f394708d6068d40865cd2d1b0484acbac 2024-05-10T04:37:49.116 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:37:48 smithi150 systemd[1]: Started Ceph mds.d for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:49 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/289163449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:49 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:49 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:49 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:49 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:49.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:49 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:49.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:49 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:49 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/289163449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:49 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:49 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:49 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:49 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:49 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:49.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:49 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:49.498 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:49 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/289163449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:49.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:49 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:49 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:49 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:49.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:49 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:49.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:49 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:49.499 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:49 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:50.028 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:37:50 smithi161 systemd[1]: Starting Ceph mds.e for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:50.154 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:50.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.310+0000 7f2369bfa700 1 -- 172.21.15.150:0/4000910889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23641025a0 msgr2=0x7f2364108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.310+0000 7f2369bfa700 1 --2- 172.21.15.150:0/4000910889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23641025a0 0x7f2364108e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f235c00b0a0 tx=0x7f235c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 -- 172.21.15.150:0/4000910889 shutdown_connections 2024-05-10T04:37:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 --2- 172.21.15.150:0/4000910889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23641025a0 0x7f2364108e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 --2- 172.21.15.150:0/4000910889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2364100ae0 0x7f2364100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 --2- 172.21.15.150:0/4000910889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2364101c00 0x7f2364102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 -- 172.21.15.150:0/4000910889 >> 172.21.15.150:0/4000910889 conn(0x7f23640fc0f0 msgr2=0x7f23640fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:50.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 -- 172.21.15.150:0/4000910889 shutdown_connections 2024-05-10T04:37:50.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 -- 172.21.15.150:0/4000910889 wait complete. 2024-05-10T04:37:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 Processor -- start 2024-05-10T04:37:50.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.311+0000 7f2369bfa700 1 -- start start 2024-05-10T04:37:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2369bfa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2364100ae0 0x7f236410b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2369bfa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2364101c00 0x7f2364109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2369bfa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23641025a0 0x7f2364109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2369bfa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236410a620 con 0x7f2364101c00 2024-05-10T04:37:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2369bfa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236410a790 con 0x7f2364100ae0 2024-05-10T04:37:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2369bfa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23641a21d0 con 0x7f23641025a0 2024-05-10T04:37:50.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2364101c00 0x7f2364109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:50.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2364101c00 0x7f2364109a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60494/0 (socket says 172.21.15.150:60494) 2024-05-10T04:37:50.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 -- 172.21.15.150:0/669197229 learned_addr learned my addr 172.21.15.150:0/669197229 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:50.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f23637fe700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2364100ae0 0x7f236410b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:50.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2363fff700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23641025a0 0x7f2364109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:50.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 -- 172.21.15.150:0/669197229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23641025a0 msgr2=0x7f2364109f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:50.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23641025a0 0x7f2364109f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 -- 172.21.15.150:0/669197229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2364100ae0 msgr2=0x7f236410b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:50.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2364100ae0 0x7f236410b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 -- 172.21.15.150:0/669197229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009d00 con 0x7f2364101c00 2024-05-10T04:37:50.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2362ffd700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2364101c00 0x7f2364109a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f235400d950 tx=0x7f235400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:50.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2360ff9700 1 -- 172.21.15.150:0/669197229 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354009b40 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23641a2430 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.312+0000 7f2360ff9700 1 -- 172.21.15.150:0/669197229 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2354009ca0 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.313+0000 7f2360ff9700 1 -- 172.21.15.150:0/669197229 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400f620 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.313+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23641a2960 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.313+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2364107cf0 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.315+0000 7f2360ff9700 1 -- 172.21.15.150:0/669197229 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2354010460 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.315+0000 7f2360ff9700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23500a1120 0x7f23500a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.316+0000 7f23637fe700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23500a1120 0x7f23500a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.316+0000 7f2360ff9700 1 -- 172.21.15.150:0/669197229 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f23540cc780 con 0x7f2364101c00 2024-05-10T04:37:50.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.316+0000 7f23637fe700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23500a1120 0x7f23500a3630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2364101b00 tx=0x7f234c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:50.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.320+0000 7f2360ff9700 1 -- 172.21.15.150:0/669197229 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f23540963b0 con 0x7f2364101c00 2024-05-10T04:37:50.390 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:50 smithi161 ceph-mon[46912]: Deploying daemon mds.e on smithi161 2024-05-10T04:37:50.390 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:50 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.150:6836/2935316346,v1:172.21.15.150:6837/2935316346] up:boot 2024-05-10T04:37:50.391 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:50 smithi161 ceph-mon[46912]: fsmap 4 up:standby 2024-05-10T04:37:50.391 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:50 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T04:37:50.391 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:50 smithi161 ceph-mon[46912]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:50.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:50 smithi181 ceph-mon[47314]: Deploying daemon mds.e on smithi161 2024-05-10T04:37:50.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:50 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.150:6836/2935316346,v1:172.21.15.150:6837/2935316346] up:boot 2024-05-10T04:37:50.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:50 smithi181 ceph-mon[47314]: fsmap 4 up:standby 2024-05-10T04:37:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:50 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T04:37:50.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:50 smithi181 ceph-mon[47314]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:50.416 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:50 smithi150 ceph-mon[42404]: Deploying daemon mds.e on smithi161 2024-05-10T04:37:50.416 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:50 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.150:6836/2935316346,v1:172.21.15.150:6837/2935316346] up:boot 2024-05-10T04:37:50.416 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:50 smithi150 ceph-mon[42404]: fsmap 4 up:standby 2024-05-10T04:37:50.416 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:50 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-10T04:37:50.416 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:50 smithi150 ceph-mon[42404]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:50.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.567+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2364068730 con 0x7f2364101c00 2024-05-10T04:37:50.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.568+0000 7f2360ff9700 1 -- 172.21.15.150:0/669197229 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3537 (secure 0 0 0) 0x7f2354016df0 con 0x7f2364101c00 2024-05-10T04:37:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:50.569 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-10T04:37:50.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23500a1120 msgr2=0x7f23500a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:50.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23500a1120 0x7f23500a3630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2364101b00 tx=0x7f234c00b5b0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2364101c00 msgr2=0x7f2364109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:50.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2364101c00 0x7f2364109a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f235400d950 tx=0x7f235400dd10 comp rx=0 tx=0).stop 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 shutdown_connections 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23500a1120 0x7f23500a3630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23641025a0 0x7f2364109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2364100ae0 0x7f236410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 --2- 172.21.15.150:0/669197229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2364101c00 0x7f2364109a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 >> 172.21.15.150:0/669197229 conn(0x7f23640fc0f0 msgr2=0x7f2364104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 shutdown_connections 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:50.570+0000 7f2369bfa700 1 -- 172.21.15.150:0/669197229 wait complete. 2024-05-10T04:37:50.572 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 5 2024-05-10T04:37:50.835 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:37:50 smithi161 bash[83013]: 89037170548ba98236f55278db515926d9d70ff9b18ea6a5e861e8db1003f8da 2024-05-10T04:37:50.835 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:37:50 smithi161 systemd[1]: Started Ceph mds.e for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:50.835 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:37:50 smithi161 ceph-mds[83051]: -- 172.21.15.161:0/2858441553 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fbfd4f41a0 con 0x55fbfd554400 2024-05-10T04:37:50.835 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:37:50 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-e[83027]: starting mds.e at 2024-05-10T04:37:51.467 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: Deploying daemon mds.f on smithi181 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/669197229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6834/2306790268,v1:172.21.15.161:6835/2306790268] up:boot 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: fsmap 5 up:standby 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:37:51.468 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:51 smithi181 ceph-mon[47314]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:51.719 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:37:51 smithi181 systemd[1]: Starting Ceph mds.f for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:51.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: Deploying daemon mds.f on smithi181 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/669197229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6834/2306790268,v1:172.21.15.161:6835/2306790268] up:boot 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: fsmap 5 up:standby 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:37:51.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:51 smithi161 ceph-mon[46912]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:51.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: Deploying daemon mds.f on smithi181 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/669197229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6834/2306790268,v1:172.21.15.161:6835/2306790268] up:boot 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: fsmap 5 up:standby 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:37:51.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:51 smithi150 ceph-mon[42404]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:52.585 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:52.654 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:37:52 smithi181 ceph-mds[82706]: -- 172.21.15.181:0/1238212230 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5598d29361a0 con 0x5598d2996d00 2024-05-10T04:37:52.655 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:37:52 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-f[82683]: starting mds.f at 2024-05-10T04:37:52.655 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:37:52 smithi181 bash[82668]: 59e9a070ffb5c203fbf90e6eb0b5d40c0a6d7ce256599e4682b0f3fc3038ff2e 2024-05-10T04:37:52.655 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:37:52 smithi181 systemd[1]: Started Ceph mds.f for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 -- 172.21.15.150:0/1452102509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 msgr2=0x7ff6d4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 --2- 172.21.15.150:0/1452102509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d4108e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00b0a0 tx=0x7ff6cc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 -- 172.21.15.150:0/1452102509 shutdown_connections 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 --2- 172.21.15.150:0/1452102509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d4108e30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 --2- 172.21.15.150:0/1452102509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d4100ac0 0x7ff6d4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 --2- 172.21.15.150:0/1452102509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101be0 0x7ff6d4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 -- 172.21.15.150:0/1452102509 >> 172.21.15.150:0/1452102509 conn(0x7ff6d40fc0d0 msgr2=0x7ff6d40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 -- 172.21.15.150:0/1452102509 shutdown_connections 2024-05-10T04:37:52.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.731+0000 7ff6db814700 1 -- 172.21.15.150:0/1452102509 wait complete. 2024-05-10T04:37:52.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.732+0000 7ff6db814700 1 Processor -- start 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.732+0000 7ff6db814700 1 -- start start 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6db814700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d4100ac0 0x7ff6d4196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6db814700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101be0 0x7ff6d41974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6db814700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6db814700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d419ea50 con 0x7ff6d4102580 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6db814700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d419ebc0 con 0x7ff6d4100ac0 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6db814700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d419ed80 con 0x7ff6d4101be0 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60510/0 (socket says 172.21.15.150:60510) 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 -- 172.21.15.150:0/3472712804 learned_addr learned my addr 172.21.15.150:0/3472712804 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d8daf700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101be0 0x7ff6d41974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d95b0700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d4100ac0 0x7ff6d4196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:52.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 -- 172.21.15.150:0/3472712804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101be0 msgr2=0x7ff6d41974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101be0 0x7ff6d41974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 -- 172.21.15.150:0/3472712804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d4100ac0 msgr2=0x7ff6d4196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d4100ac0 0x7ff6d4196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 -- 172.21.15.150:0/3472712804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc009d00 con 0x7ff6d4102580 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6d9db1700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d419e440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006070 tx=0x7ff6cc009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6ca7fc700 1 -- 172.21.15.150:0/3472712804 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00e040 con 0x7ff6d4102580 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d419f000 con 0x7ff6d4102580 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6ca7fc700 1 -- 172.21.15.150:0/3472712804 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6cc004ce0 con 0x7ff6d4102580 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.733+0000 7ff6ca7fc700 1 -- 172.21.15.150:0/3472712804 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc01d480 con 0x7ff6d4102580 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.734+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d419f480 con 0x7ff6d4102580 2024-05-10T04:37:52.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.734+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d4107cd0 con 0x7ff6d4102580 2024-05-10T04:37:52.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.736+0000 7ff6ca7fc700 1 -- 172.21.15.150:0/3472712804 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff6cc003a00 con 0x7ff6d4102580 2024-05-10T04:37:52.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.736+0000 7ff6ca7fc700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff6c00a1120 0x7ff6c00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:52.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.736+0000 7ff6d95b0700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff6c00a1120 0x7ff6c00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:52.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.737+0000 7ff6ca7fc700 1 -- 172.21.15.150:0/3472712804 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6cc0cddf0 con 0x7ff6d4102580 2024-05-10T04:37:52.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.737+0000 7ff6d95b0700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff6c00a1120 0x7ff6c00a3630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4101ae0 tx=0x7ff6c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:52.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.740+0000 7ff6ca7fc700 1 -- 172.21.15.150:0/3472712804 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff6cc097ab0 con 0x7ff6d4102580 2024-05-10T04:37:52.987 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.986+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6d40694d0 con 0x7ff6d4102580 2024-05-10T04:37:52.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.986+0000 7ff6ca7fc700 1 -- 172.21.15.150:0/3472712804 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4297 (secure 0 0 0) 0x7ff6cc09b0d0 con 0x7ff6d4102580 2024-05-10T04:37:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:52.988 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-10T04:37:52.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff6c00a1120 msgr2=0x7ff6c00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:52.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff6c00a1120 0x7ff6c00a3630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4101ae0 tx=0x7ff6c400b410 comp rx=0 tx=0).stop 2024-05-10T04:37:52.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 msgr2=0x7ff6d419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:52.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d419e440 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006070 tx=0x7ff6cc009440 comp rx=0 tx=0).stop 2024-05-10T04:37:52.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 shutdown_connections 2024-05-10T04:37:52.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d4102580 0x7ff6d419e440 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6d4100ac0 0x7ff6d4196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff6c00a1120 0x7ff6c00a3630 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.988+0000 7ff6db814700 1 --2- 172.21.15.150:0/3472712804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4101be0 0x7ff6d41974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:52.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.989+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 >> 172.21.15.150:0/3472712804 conn(0x7ff6d40fc0d0 msgr2=0x7ff6d4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:52.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.989+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 shutdown_connections 2024-05-10T04:37:52.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:52.989+0000 7ff6db814700 1 -- 172.21.15.150:0/3472712804 wait complete. 2024-05-10T04:37:52.991 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 6 2024-05-10T04:37:53.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:53.288 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:53.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:53.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: Deploying daemon mds.c on smithi181 2024-05-10T04:37:53.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3472712804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:53.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] up:boot 2024-05-10T04:37:53.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: fsmap 6 up:standby 2024-05-10T04:37:53.289 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:37:53.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: Deploying daemon mds.c on smithi181 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3472712804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] up:boot 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: fsmap 6 up:standby 2024-05-10T04:37:53.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:53 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:37:53.539 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:37:53 smithi181 systemd[1]: Starting Ceph mds.c for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:53.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: Deploying daemon mds.c on smithi181 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3472712804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] up:boot 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: fsmap 6 up:standby 2024-05-10T04:37:53.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:53 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:37:54.240 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-mds[83045]: -- 172.21.15.181:0/2445363543 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c97fa5e1a0 con 0x55c97fabe880 2024-05-10T04:37:54.241 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:37:53 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-c[83022]: starting mds.c at 2024-05-10T04:37:54.241 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:37:54 smithi181 bash[83007]: 998d55b8922f90de1a285e0d8777ab293e62464ae75bca6f35f5b727eb74aff9 2024-05-10T04:37:54.241 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:37:54 smithi181 systemd[1]: Started Ceph mds.c for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:54 smithi161 ceph-mon[46912]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.492 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:54.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:54.493 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:54 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:54.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:54 smithi181 ceph-mon[47314]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:54.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:54 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:54.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:54 smithi150 ceph-mon[42404]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:54.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:54.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:54.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:54.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:54 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:55.004 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:55.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 -- 172.21.15.150:0/2906505203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350101be0 msgr2=0x7fe350101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:55.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 --2- 172.21.15.150:0/2906505203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350101be0 0x7fe350101ff0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe338009990 tx=0x7fe338009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 -- 172.21.15.150:0/2906505203 shutdown_connections 2024-05-10T04:37:55.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 --2- 172.21.15.150:0/2906505203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe350102580 0x7fe350108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 --2- 172.21.15.150:0/2906505203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe350100ac0 0x7fe350100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 --2- 172.21.15.150:0/2906505203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350101be0 0x7fe350101ff0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 -- 172.21.15.150:0/2906505203 >> 172.21.15.150:0/2906505203 conn(0x7fe3500fc0d0 msgr2=0x7fe3500fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 -- 172.21.15.150:0/2906505203 shutdown_connections 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.149+0000 7fe3555eb700 1 -- 172.21.15.150:0/2906505203 wait complete. 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 Processor -- start 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 -- start start 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350100ac0 0x7fe350197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe350101be0 0x7fe3501984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe350102580 0x7fe35019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35019fa40 con 0x7fe350100ac0 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35019fb80 con 0x7fe350101be0 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe3555eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35019fd10 con 0x7fe350102580 2024-05-10T04:37:55.154 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe34effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350100ac0 0x7fe350197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe34effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350100ac0 0x7fe350197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60536/0 (socket says 172.21.15.150:60536) 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.150+0000 7fe34effd700 1 -- 172.21.15.150:0/4001073025 learned_addr learned my addr 172.21.15.150:0/4001073025 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34e7fc700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe350101be0 0x7fe3501984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34f7fe700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe350102580 0x7fe35019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34effd700 1 -- 172.21.15.150:0/4001073025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe350102580 msgr2=0x7fe35019f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34effd700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe350102580 0x7fe35019f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34effd700 1 -- 172.21.15.150:0/4001073025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe350101be0 msgr2=0x7fe3501984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34effd700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe350101be0 0x7fe3501984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34effd700 1 -- 172.21.15.150:0/4001073025 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380095f0 con 0x7fe350100ac0 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe34effd700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350100ac0 0x7fe350197fa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe340007e00 tx=0x7fe34000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe347fff700 1 -- 172.21.15.150:0/4001073025 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34000ad00 con 0x7fe350100ac0 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35019fff0 con 0x7fe350100ac0 2024-05-10T04:37:55.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe347fff700 1 -- 172.21.15.150:0/4001073025 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe34000ae60 con 0x7fe350100ac0 2024-05-10T04:37:55.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe347fff700 1 -- 172.21.15.150:0/4001073025 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340011700 con 0x7fe350100ac0 2024-05-10T04:37:55.156 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.151+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3501a0420 con 0x7fe350100ac0 2024-05-10T04:37:55.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.152+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3500694d0 con 0x7fe350100ac0 2024-05-10T04:37:55.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.153+0000 7fe347fff700 1 -- 172.21.15.150:0/4001073025 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe3400129e0 con 0x7fe350100ac0 2024-05-10T04:37:55.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.154+0000 7fe347fff700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe33c0a1120 0x7fe33c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:55.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.154+0000 7fe347fff700 1 -- 172.21.15.150:0/4001073025 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3400cc880 con 0x7fe350100ac0 2024-05-10T04:37:55.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.154+0000 7fe34e7fc700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe33c0a1120 0x7fe33c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:55.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.154+0000 7fe34e7fc700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe33c0a1120 0x7fe33c0a3630 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe350199040 tx=0x7fe33801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:55.159 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.158+0000 7fe347fff700 1 -- 172.21.15.150:0/4001073025 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3400964b0 con 0x7fe350100ac0 2024-05-10T04:37:55.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.402+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe35010c310 con 0x7fe350100ac0 2024-05-10T04:37:55.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.403+0000 7fe347fff700 1 -- 172.21.15.150:0/4001073025 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5817 (secure 0 0 0) 0x7fe340099ad0 con 0x7fe350100ac0 2024-05-10T04:37:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:55.404 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-10T04:37:55.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe33c0a1120 msgr2=0x7fe33c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:55.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe33c0a1120 0x7fe33c0a3630 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe350199040 tx=0x7fe33801a040 comp rx=0 tx=0).stop 2024-05-10T04:37:55.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350100ac0 msgr2=0x7fe350197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:55.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350100ac0 0x7fe350197fa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe340007e00 tx=0x7fe34000a340 comp rx=0 tx=0).stop 2024-05-10T04:37:55.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 shutdown_connections 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe33c0a1120 0x7fe33c0a3630 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe350102580 0x7fe35019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe350100ac0 0x7fe350197fa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 --2- 172.21.15.150:0/4001073025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe350101be0 0x7fe3501984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 >> 172.21.15.150:0/4001073025 conn(0x7fe3500fc0d0 msgr2=0x7fe3500fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 shutdown_connections 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:55.405+0000 7fe3555eb700 1 -- 172.21.15.150:0/4001073025 wait complete. 2024-05-10T04:37:55.407 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 8 2024-05-10T04:37:55.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:55 smithi150 ceph-mon[42404]: Deploying daemon mds.h on smithi161 2024-05-10T04:37:55.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:55 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6836/1032605592,v1:172.21.15.181:6837/1032605592] up:boot 2024-05-10T04:37:55.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:55 smithi150 ceph-mon[42404]: fsmap 7 up:standby 2024-05-10T04:37:55.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:55 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:37:55.571 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:55 smithi161 ceph-mon[46912]: Deploying daemon mds.h on smithi161 2024-05-10T04:37:55.571 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:55 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6836/1032605592,v1:172.21.15.181:6837/1032605592] up:boot 2024-05-10T04:37:55.571 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:55 smithi161 ceph-mon[46912]: fsmap 7 up:standby 2024-05-10T04:37:55.571 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:55 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:37:55.571 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:37:55 smithi161 systemd[1]: Starting Ceph mds.h for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:55.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:55 smithi181 ceph-mon[47314]: Deploying daemon mds.h on smithi161 2024-05-10T04:37:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:55 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6836/1032605592,v1:172.21.15.181:6837/1032605592] up:boot 2024-05-10T04:37:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:55 smithi181 ceph-mon[47314]: fsmap 7 up:standby 2024-05-10T04:37:55.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:55 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:37:56.085 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:37:55 smithi161 ceph-mds[83391]: -- 172.21.15.161:0/1740030501 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561650c541a0 con 0x561650cb4400 2024-05-10T04:37:56.085 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:37:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-h[83368]: starting mds.h at 2024-05-10T04:37:56.085 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:37:55 smithi161 bash[83353]: 9da4c01be77dbb2c06067a860aa3235325f3a0e51053da4354c6ab7ae732aed8 2024-05-10T04:37:56.086 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:37:55 smithi161 systemd[1]: Started Ceph mds.h for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:56.505 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:56.506 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4001073025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:56.506 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.506 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.506 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.506 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:56.506 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:56.506 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:56 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4001073025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:56.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:56 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:56.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:56.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4001073025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:56.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:56.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:56.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:56.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:56 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:57.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:57 smithi181 ceph-mon[47314]: Deploying daemon mds.i on smithi181 2024-05-10T04:37:57.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:57 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6836/55043330,v1:172.21.15.161:6837/55043330] up:boot 2024-05-10T04:37:57.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:57 smithi181 ceph-mon[47314]: fsmap 8 up:standby 2024-05-10T04:37:57.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:57 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:37:57.405 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:37:57 smithi181 systemd[1]: Starting Ceph mds.i for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:57.420 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:57.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 -- 172.21.15.150:0/593348513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c1002c0 msgr2=0x7f437c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 --2- 172.21.15.150:0/593348513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c1002c0 0x7f437c1006d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f436c00aaa0 tx=0x7f436c00adb0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 -- 172.21.15.150:0/593348513 shutdown_connections 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 --2- 172.21.15.150:0/593348513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437c101d80 0x7f437c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 --2- 172.21.15.150:0/593348513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c1002c0 0x7f437c1006d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 --2- 172.21.15.150:0/593348513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f437c1013e0 0x7f437c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 -- 172.21.15.150:0/593348513 >> 172.21.15.150:0/593348513 conn(0x7f437c0fb8f0 msgr2=0x7f437c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 -- 172.21.15.150:0/593348513 shutdown_connections 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.567+0000 7f4380d14700 1 -- 172.21.15.150:0/593348513 wait complete. 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 Processor -- start 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 -- start start 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437c1002c0 0x7f437c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:57.570 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f437c1013e0 0x7f437c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c101d80 0x7f437c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c19fae0 con 0x7f437c101d80 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c19fc50 con 0x7f437c1013e0 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4380d14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c19fe10 con 0x7f437c1002c0 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c101d80 0x7f437c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437c1002c0 0x7f437c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437c1002c0 0x7f437c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:48012/0 (socket says 172.21.15.150:48012) 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437a59c700 1 -- 172.21.15.150:0/1109045759 learned_addr learned my addr 172.21.15.150:0/1109045759 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f4379d9b700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f437c1013e0 0x7f437c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437ad9d700 1 -- 172.21.15.150:0/1109045759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437c1002c0 msgr2=0x7f437c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437ad9d700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437c1002c0 0x7f437c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437ad9d700 1 -- 172.21.15.150:0/1109045759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f437c1013e0 msgr2=0x7f437c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437ad9d700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f437c1013e0 0x7f437c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.571 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.568+0000 7f437ad9d700 1 -- 172.21.15.150:0/1109045759 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c00a700 con 0x7f437c101d80 2024-05-10T04:37:57.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.569+0000 7f437ad9d700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c101d80 0x7f437c19f4d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4374007f10 tx=0x7f437400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:57.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.569+0000 7f436b7fe700 1 -- 172.21.15.150:0/1109045759 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374004db0 con 0x7f437c101d80 2024-05-10T04:37:57.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.569+0000 7f436b7fe700 1 -- 172.21.15.150:0/1109045759 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f437400f040 con 0x7f437c101d80 2024-05-10T04:37:57.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.569+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437c1a00f0 con 0x7f437c101d80 2024-05-10T04:37:57.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.569+0000 7f436b7fe700 1 -- 172.21.15.150:0/1109045759 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374005990 con 0x7f437c101d80 2024-05-10T04:37:57.572 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.569+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437c1a05b0 con 0x7f437c101d80 2024-05-10T04:37:57.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.570+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437c1074d0 con 0x7f437c101d80 2024-05-10T04:37:57.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.571+0000 7f436b7fe700 1 -- 172.21.15.150:0/1109045759 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f437400db20 con 0x7f437c101d80 2024-05-10T04:37:57.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.571+0000 7f436b7fe700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43640a1120 0x7f43640a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:57.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.572+0000 7f437a59c700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43640a1120 0x7f43640a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:57.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.572+0000 7f436b7fe700 1 -- 172.21.15.150:0/1109045759 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43740cc550 con 0x7f437c101d80 2024-05-10T04:37:57.573 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.572+0000 7f437a59c700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43640a1120 0x7f43640a3630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c6d0 tx=0x7f436c003cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:57.575 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.575+0000 7f436b7fe700 1 -- 172.21.15.150:0/1109045759 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f43740960e0 con 0x7f437c101d80 2024-05-10T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:57 smithi161 ceph-mon[46912]: Deploying daemon mds.i on smithi181 2024-05-10T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:57 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6836/55043330,v1:172.21.15.161:6837/55043330] up:boot 2024-05-10T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:57 smithi161 ceph-mon[46912]: fsmap 8 up:standby 2024-05-10T04:37:57.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:57 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:37:57.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:57 smithi150 ceph-mon[42404]: Deploying daemon mds.i on smithi181 2024-05-10T04:37:57.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:57 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6836/55043330,v1:172.21.15.161:6837/55043330] up:boot 2024-05-10T04:37:57.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:57 smithi150 ceph-mon[42404]: fsmap 8 up:standby 2024-05-10T04:37:57.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:57 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:37:57.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.829+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f437c0678b0 con 0x7f437c101d80 2024-05-10T04:37:57.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.830+0000 7f436b7fe700 1 -- 172.21.15.150:0/1109045759 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6571 (secure 0 0 0) 0x7f4374099700 con 0x7f437c101d80 2024-05-10T04:37:57.831 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:37:57.831 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43640a1120 msgr2=0x7f43640a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43640a1120 0x7f43640a3630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c6d0 tx=0x7f436c003cf0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c101d80 msgr2=0x7f437c19f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c101d80 0x7f437c19f4d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4374007f10 tx=0x7f437400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 shutdown_connections 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f437c101d80 0x7f437c19f4d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43640a1120 0x7f43640a3630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437c1002c0 0x7f437c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 --2- 172.21.15.150:0/1109045759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f437c1013e0 0x7f437c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.832+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 >> 172.21.15.150:0/1109045759 conn(0x7f437c0fb8f0 msgr2=0x7f437c104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.833+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 shutdown_connections 2024-05-10T04:37:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:57.833+0000 7f4380d14700 1 -- 172.21.15.150:0/1109045759 wait complete. 2024-05-10T04:37:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 9 2024-05-10T04:37:57.904 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:37:57 smithi181 ceph-mds[83386]: -- 172.21.15.181:0/4084424587 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56443e2981a0 con 0x56443e2f8d00 2024-05-10T04:37:57.904 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:37:57 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-i[83363]: starting mds.i at 2024-05-10T04:37:57.905 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:37:57 smithi181 bash[83348]: eb02523e4b74fb061b15d2c17fb4c1485fabbdd67671c845f2ee8679b570b6e3 2024-05-10T04:37:57.905 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:37:57 smithi181 systemd[1]: Started Ceph mds.i for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:58.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:58.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:58.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:58.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:58.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:58 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1109045759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:58.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:58.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:58.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:58.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:58.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:58 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1109045759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:58.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:37:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:37:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:37:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:37:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:37:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:58 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1109045759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:37:59.162 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:37:58 smithi150 systemd[1]: Starting Ceph mds.g for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:37:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:59 smithi161 ceph-mon[46912]: Deploying daemon mds.g on smithi150 2024-05-10T04:37:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:59 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] up:boot 2024-05-10T04:37:59.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:59 smithi161 ceph-mon[46912]: fsmap 9 up:standby 2024-05-10T04:37:59.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:37:59 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:37:59.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:59 smithi181 ceph-mon[47314]: Deploying daemon mds.g on smithi150 2024-05-10T04:37:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:59 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] up:boot 2024-05-10T04:37:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:59 smithi181 ceph-mon[47314]: fsmap 9 up:standby 2024-05-10T04:37:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:37:59 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:37:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:59 smithi150 ceph-mon[42404]: Deploying daemon mds.g on smithi150 2024-05-10T04:37:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:59 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] up:boot 2024-05-10T04:37:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:59 smithi150 ceph-mon[42404]: fsmap 9 up:standby 2024-05-10T04:37:59.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:37:59 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:37:59.661 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:37:59 smithi150 bash[92750]: 1210f90d139698bd0aeab99347ccff17ec563a2fd051d96b85d9d97c9774fa0f 2024-05-10T04:37:59.662 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:37:59 smithi150 systemd[1]: Started Ceph mds.g for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:37:59.662 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:37:59 smithi150 ceph-mds[92788]: -- 172.21.15.150:0/2312798983 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555c86621a0 con 0x5555c86c2400 2024-05-10T04:37:59.662 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:37:59 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-g[92765]: starting mds.g at 2024-05-10T04:37:59.847 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:37:59.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 -- 172.21.15.150:0/3064795985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88001050e0 msgr2=0x7f8800107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:59.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 --2- 172.21.15.150:0/3064795985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88001050e0 0x7f8800107500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009990 tx=0x7f87fc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 -- 172.21.15.150:0/3064795985 shutdown_connections 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 --2- 172.21.15.150:0/3064795985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8800107a40 0x7f8800109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 --2- 172.21.15.150:0/3064795985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8800100050 0x7f88001004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 --2- 172.21.15.150:0/3064795985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88001050e0 0x7f8800107500 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 -- 172.21.15.150:0/3064795985 >> 172.21.15.150:0/3064795985 conn(0x7f88000fb8f0 msgr2=0x7f88000fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 -- 172.21.15.150:0/3064795985 shutdown_connections 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.987+0000 7f8807979700 1 -- 172.21.15.150:0/3064795985 wait complete. 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 Processor -- start 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 -- start start 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8800100050 0x7f8800198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88001050e0 0x7f8800198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8800107a40 0x7f880019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019faf0 con 0x7f8800100050 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019fc60 con 0x7f8800107a40 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8807979700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880019fe20 con 0x7f88001050e0 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8805715700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8800100050 0x7f8800198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8805715700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8800100050 0x7f8800198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44516/0 (socket says 172.21.15.150:44516) 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.988+0000 7f8805715700 1 -- 172.21.15.150:0/422535840 learned_addr learned my addr 172.21.15.150:0/422535840 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805f16700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8800107a40 0x7f880019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8804f14700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88001050e0 0x7f8800198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805715700 1 -- 172.21.15.150:0/422535840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88001050e0 msgr2=0x7f8800198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805715700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88001050e0 0x7f8800198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805715700 1 -- 172.21.15.150:0/422535840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8800107a40 msgr2=0x7f880019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805715700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8800107a40 0x7f880019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805715700 1 -- 172.21.15.150:0/422535840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0095f0 con 0x7f8800100050 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805f16700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8800107a40 0x7f880019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:37:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8805715700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8800100050 0x7f8800198010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e970 tx=0x7f87f000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:59.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f87f67fc700 1 -- 172.21.15.150:0/422535840 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0009790 con 0x7f8800100050 2024-05-10T04:37:59.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88001a0100 con 0x7f8800100050 2024-05-10T04:37:59.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f87f67fc700 1 -- 172.21.15.150:0/422535840 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f0004cf0 con 0x7f8800100050 2024-05-10T04:37:59.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f87f67fc700 1 -- 172.21.15.150:0/422535840 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0005680 con 0x7f8800100050 2024-05-10T04:37:59.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.989+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88001a0610 con 0x7f8800100050 2024-05-10T04:37:59.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.990+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88000fd4a0 con 0x7f8800100050 2024-05-10T04:37:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.991+0000 7f87f67fc700 1 -- 172.21.15.150:0/422535840 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f87f0004750 con 0x7f8800100050 2024-05-10T04:37:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.992+0000 7f87f67fc700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87ec0a11a0 0x7f87ec0a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:37:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.992+0000 7f8804f14700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87ec0a11a0 0x7f87ec0a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:37:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.992+0000 7f87f67fc700 1 -- 172.21.15.150:0/422535840 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87f0014070 con 0x7f8800100050 2024-05-10T04:37:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.992+0000 7f8804f14700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87ec0a11a0 0x7f87ec0a36b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00b2e0 tx=0x7f87fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:37:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:37:59.995+0000 7f87f67fc700 1 -- 172.21.15.150:0/422535840 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f87f00965b0 con 0x7f8800100050 2024-05-10T04:38:00.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.259+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88000678b0 con 0x7f8800100050 2024-05-10T04:38:00.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.262+0000 7f87f67fc700 1 -- 172.21.15.150:0/422535840 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7333 (secure 0 0 0) 0x7f87f0099bd0 con 0x7f8800100050 2024-05-10T04:38:00.262 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:00.262 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87ec0a11a0 msgr2=0x7f87ec0a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87ec0a11a0 0x7f87ec0a36b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00b2e0 tx=0x7f87fc01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8800100050 msgr2=0x7f8800198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8800100050 0x7f8800198010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e970 tx=0x7f87f000ed30 comp rx=0 tx=0).stop 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 shutdown_connections 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8800107a40 0x7f880019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8800100050 0x7f8800198010 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87ec0a11a0 0x7f87ec0a36b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 --2- 172.21.15.150:0/422535840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88001050e0 0x7f8800198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:00.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 >> 172.21.15.150:0/422535840 conn(0x7f88000fb8f0 msgr2=0x7f88001060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:00.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 shutdown_connections 2024-05-10T04:38:00.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:00.265+0000 7f8807979700 1 -- 172.21.15.150:0/422535840 wait complete. 2024-05-10T04:38:00.267 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 10 2024-05-10T04:38:00.492 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:00 smithi150 ceph-mon[42404]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:00.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:38:00.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:38:00.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:00 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:00 smithi181 ceph-mon[47314]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:00 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:00 smithi161 ceph-mon[46912]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.664 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:00.665 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:38:00.665 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:38:00.665 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:00 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:00.748 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:38:00 smithi150 systemd[1]: Starting Ceph mds.a for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:38:01.411 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:38:01 smithi150 bash[93137]: 00c25cdc06386d4cf4229046a5841f819762881065830f428109a030eae0e917 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:38:01 smithi150 systemd[1]: Started Ceph mds.a for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mds[93176]: -- 172.21.15.150:0/14152790 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5654df90a1a0 con 0x5654df96ad00 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-a[93152]: starting mds.a at 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: Deploying daemon mds.a on smithi150 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/422535840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.150:6838/308347873,v1:172.21.15.150:6839/308347873] up:boot 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: fsmap 10 up:standby 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:38:01.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:38:01.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:01 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:01.578 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: Deploying daemon mds.a on smithi150 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/422535840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.150:6838/308347873,v1:172.21.15.150:6839/308347873] up:boot 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: fsmap 10 up:standby 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:38:01.579 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:01 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:01.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: Deploying daemon mds.a on smithi150 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/422535840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.150:6838/308347873,v1:172.21.15.150:6839/308347873] up:boot 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: fsmap 10 up:standby 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-10T04:38:01.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:01 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:02.282 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:38:02.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:02 smithi161 ceph-mon[46912]: Deploying daemon mds.b on smithi161 2024-05-10T04:38:02.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:02 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.150:6840/3789106275,v1:172.21.15.150:6841/3789106275] up:boot 2024-05-10T04:38:02.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:02 smithi161 ceph-mon[46912]: fsmap 11 up:standby 2024-05-10T04:38:02.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:02 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:38:02.402 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:02 smithi161 ceph-mon[46912]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:02.402 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:02 smithi161 systemd[1]: Starting Ceph mds.b for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:38:02.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 -- 172.21.15.150:0/4001102264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c1026b0 msgr2=0x7fd14c104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:02.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 --2- 172.21.15.150:0/4001102264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c1026b0 0x7fd14c104ad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009990 tx=0x7fd13c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 -- 172.21.15.150:0/4001102264 shutdown_connections 2024-05-10T04:38:02.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 --2- 172.21.15.150:0/4001102264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c107970 0x7fd14c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 --2- 172.21.15.150:0/4001102264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c1026b0 0x7fd14c104ad0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 --2- 172.21.15.150:0/4001102264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd14c105010 0x7fd14c107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 -- 172.21.15.150:0/4001102264 >> 172.21.15.150:0/4001102264 conn(0x7fd14c0fc0d0 msgr2=0x7fd14c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 -- 172.21.15.150:0/4001102264 shutdown_connections 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 -- 172.21.15.150:0/4001102264 wait complete. 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.426+0000 7fd154502700 1 Processor -- start 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd154502700 1 -- start start 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd154502700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c1026b0 0x7fd14c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd154502700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c105010 0x7fd14c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd154502700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd14c107970 0x7fd14c19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd154502700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c19fb20 con 0x7fd14c105010 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd154502700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c19fc90 con 0x7fd14c107970 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd154502700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c19fe50 con 0x7fd14c1026b0 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c105010 0x7fd14c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:02.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c105010 0x7fd14c198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44552/0 (socket says 172.21.15.150:44552) 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 -- 172.21.15.150:0/3872658644 learned_addr learned my addr 172.21.15.150:0/3872658644 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd152a9f700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd14c107970 0x7fd14c19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd15229e700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c1026b0 0x7fd14c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 -- 172.21.15.150:0/3872658644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c1026b0 msgr2=0x7fd14c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c1026b0 0x7fd14c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 -- 172.21.15.150:0/3872658644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd14c107970 msgr2=0x7fd14c19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd14c107970 0x7fd14c19f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 -- 172.21.15.150:0/3872658644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0095f0 con 0x7fd14c105010 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd151a9d700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c105010 0x7fd14c198580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c920 tx=0x7fd14800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.427+0000 7fd1437fe700 1 -- 172.21.15.150:0/3872658644 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148007b00 con 0x7fd14c105010 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.428+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c1a0130 con 0x7fd14c105010 2024-05-10T04:38:02.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.428+0000 7fd1437fe700 1 -- 172.21.15.150:0/3872658644 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14800ce70 con 0x7fd14c105010 2024-05-10T04:38:02.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.428+0000 7fd1437fe700 1 -- 172.21.15.150:0/3872658644 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148005460 con 0x7fd14c105010 2024-05-10T04:38:02.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.428+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c1a0690 con 0x7fd14c105010 2024-05-10T04:38:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.428+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c0fdc80 con 0x7fd14c105010 2024-05-10T04:38:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.430+0000 7fd1437fe700 1 -- 172.21.15.150:0/3872658644 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd1480055c0 con 0x7fd14c105010 2024-05-10T04:38:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.431+0000 7fd1437fe700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1380a1120 0x7fd1380a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.431+0000 7fd15229e700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1380a1120 0x7fd1380a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.431+0000 7fd1437fe700 1 -- 172.21.15.150:0/3872658644 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1480cc440 con 0x7fd14c105010 2024-05-10T04:38:02.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.432+0000 7fd15229e700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1380a1120 0x7fd1380a3630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0037f0 tx=0x7fd13c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:02.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.435+0000 7fd1437fe700 1 -- 172.21.15.150:0/3872658644 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd148096070 con 0x7fd14c105010 2024-05-10T04:38:02.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:02 smithi181 ceph-mon[47314]: Deploying daemon mds.b on smithi161 2024-05-10T04:38:02.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:02 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.150:6840/3789106275,v1:172.21.15.150:6841/3789106275] up:boot 2024-05-10T04:38:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:02 smithi181 ceph-mon[47314]: fsmap 11 up:standby 2024-05-10T04:38:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:02 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:38:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:02 smithi181 ceph-mon[47314]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:02 smithi150 ceph-mon[42404]: Deploying daemon mds.b on smithi161 2024-05-10T04:38:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:02 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.150:6840/3789106275,v1:172.21.15.150:6841/3789106275] up:boot 2024-05-10T04:38:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:02 smithi150 ceph-mon[42404]: fsmap 11 up:standby 2024-05-10T04:38:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:02 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:38:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:02 smithi150 ceph-mon[42404]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:02.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.677+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd14c0694d0 con 0x7fd14c105010 2024-05-10T04:38:02.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.677+0000 7fd1437fe700 1 -- 172.21.15.150:0/3872658644 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8852 (secure 0 0 0) 0x7fd148099690 con 0x7fd14c105010 2024-05-10T04:38:02.678 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:02.678 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-10T04:38:02.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1380a1120 msgr2=0x7fd1380a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:02.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1380a1120 0x7fd1380a3630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0037f0 tx=0x7fd13c01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:02.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c105010 msgr2=0x7fd14c198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:02.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c105010 0x7fd14c198580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c920 tx=0x7fd14800cc30 comp rx=0 tx=0).stop 2024-05-10T04:38:02.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 shutdown_connections 2024-05-10T04:38:02.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd14c107970 0x7fd14c19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1380a1120 0x7fd1380a3630 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c1026b0 0x7fd14c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 --2- 172.21.15.150:0/3872658644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd14c105010 0x7fd14c198580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:02.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 >> 172.21.15.150:0/3872658644 conn(0x7fd14c0fc0d0 msgr2=0x7fd14c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:02.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 shutdown_connections 2024-05-10T04:38:02.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:02.680+0000 7fd154502700 1 -- 172.21.15.150:0/3872658644 wait complete. 2024-05-10T04:38:02.681 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 12 2024-05-10T04:38:03.085 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:02 smithi161 ceph-mds[83741]: -- 172.21.15.161:0/914746033 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e6063161a0 con 0x55e606376880 2024-05-10T04:38:03.085 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:02 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: starting mds.b at 2024-05-10T04:38:03.085 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:02 smithi161 bash[83703]: acf4ac310ae442d5ac435b1f22932b10d28da16dbc91a57cc71b1a68494a4fde 2024-05-10T04:38:03.085 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:02 smithi161 systemd[1]: Started Ceph mds.b for 28928ad0-0e86-11ef-bc98-c7b262605968. 2024-05-10T04:38:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:03 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3872658644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:03 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:38:03.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:03 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3872658644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:03 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:38:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:03 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3872658644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:03.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:03 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:38:04.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] up:boot 2024-05-10T04:38:04.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: fsmap 12 up:standby 2024-05-10T04:38:04.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:38:04.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:04.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:04 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] up:boot 2024-05-10T04:38:04.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: fsmap 12 up:standby 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:04 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] up:boot 2024-05-10T04:38:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: fsmap 12 up:standby 2024-05-10T04:38:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:38:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:04 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:04.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:38:04.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 -- 172.21.15.150:0/1331465994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 msgr2=0x7fe35c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:04.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 --2- 172.21.15.150:0/1331465994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1020a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe350009990 tx=0x7fe350009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 -- 172.21.15.150:0/1331465994 shutdown_connections 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 --2- 172.21.15.150:0/1331465994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c102630 0x7fe35c108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 --2- 172.21.15.150:0/1331465994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c100b70 0x7fe35c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 --2- 172.21.15.150:0/1331465994 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1020a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 -- 172.21.15.150:0/1331465994 >> 172.21.15.150:0/1331465994 conn(0x7fe35c0fc180 msgr2=0x7fe35c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 -- 172.21.15.150:0/1331465994 shutdown_connections 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.831+0000 7fe36281f700 1 -- 172.21.15.150:0/1331465994 wait complete. 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.832+0000 7fe36281f700 1 Processor -- start 2024-05-10T04:38:04.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.832+0000 7fe36281f700 1 -- start start 2024-05-10T04:38:04.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe36281f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c100b70 0x7fe35c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:04.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe36281f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:04.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe36281f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c102630 0x7fe35c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:04.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe36281f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19fb50 con 0x7fe35c101c90 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe36281f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19fcc0 con 0x7fe35c100b70 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe36281f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19fe80 con 0x7fe35c102630 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c100b70 0x7fe35c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44574/0 (socket says 172.21.15.150:44574) 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35b7fe700 1 -- 172.21.15.150:0/3886141539 learned_addr learned my addr 172.21.15.150:0/3886141539 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe360dbc700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c102630 0x7fe35c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35bfff700 1 -- 172.21.15.150:0/3886141539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c102630 msgr2=0x7fe35c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35bfff700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c102630 0x7fe35c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35bfff700 1 -- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 msgr2=0x7fe35c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35bfff700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35bfff700 1 -- 172.21.15.150:0/3886141539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500095f0 con 0x7fe35c100b70 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.833+0000 7fe35b7fe700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.834+0000 7fe35bfff700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c100b70 0x7fe35c198070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe35c100860 tx=0x7fe34c00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:04.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.834+0000 7fe3597fa700 1 -- 172.21.15.150:0/3886141539 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c004220 con 0x7fe35c100b70 2024-05-10T04:38:04.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.834+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c1a0160 con 0x7fe35c100b70 2024-05-10T04:38:04.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.834+0000 7fe3597fa700 1 -- 172.21.15.150:0/3886141539 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe34c004380 con 0x7fe35c100b70 2024-05-10T04:38:04.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.834+0000 7fe3597fa700 1 -- 172.21.15.150:0/3886141539 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003be0 con 0x7fe35c100b70 2024-05-10T04:38:04.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.834+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c1a06c0 con 0x7fe35c100b70 2024-05-10T04:38:04.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.835+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c107d80 con 0x7fe35c100b70 2024-05-10T04:38:04.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.837+0000 7fe3597fa700 1 -- 172.21.15.150:0/3886141539 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe34c003d40 con 0x7fe35c100b70 2024-05-10T04:38:04.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.838+0000 7fe3597fa700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3440a1120 0x7fe3440a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:04.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.838+0000 7fe35b7fe700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3440a1120 0x7fe3440a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:04.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.838+0000 7fe3597fa700 1 -- 172.21.15.150:0/3886141539 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe34c014070 con 0x7fe35c100b70 2024-05-10T04:38:04.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.838+0000 7fe35b7fe700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3440a1120 0x7fe3440a3630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe350006010 tx=0x7fe35001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:04.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:04.841+0000 7fe3597fa700 1 -- 172.21.15.150:0/3886141539 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe34c0969e0 con 0x7fe35c100b70 2024-05-10T04:38:05.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.113+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe35c10d4d0 con 0x7fe35c100b70 2024-05-10T04:38:05.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.113+0000 7fe3597fa700 1 -- 172.21.15.150:0/3886141539 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9613 (secure 0 0 0) 0x7fe34c09a000 con 0x7fe35c100b70 2024-05-10T04:38:05.114 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:05.114 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-10T04:38:05.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3440a1120 msgr2=0x7fe3440a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3440a1120 0x7fe3440a3630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe350006010 tx=0x7fe35001a040 comp rx=0 tx=0).stop 2024-05-10T04:38:05.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c100b70 msgr2=0x7fe35c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c100b70 0x7fe35c198070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe35c100860 tx=0x7fe34c00eca0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 shutdown_connections 2024-05-10T04:38:05.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3440a1120 0x7fe3440a3630 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe35c102630 0x7fe35c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c100b70 0x7fe35c198070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 --2- 172.21.15.150:0/3886141539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe35c101c90 0x7fe35c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 >> 172.21.15.150:0/3886141539 conn(0x7fe35c0fc180 msgr2=0x7fe35c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 shutdown_connections 2024-05-10T04:38:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.116+0000 7fe36281f700 1 -- 172.21.15.150:0/3886141539 wait complete. 2024-05-10T04:38:05.117 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 13 2024-05-10T04:38:05.133 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:05.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 -- 172.21.15.150:0/3827634115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c8101c10 msgr2=0x7fd7c8102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 --2- 172.21.15.150:0/3827634115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c8101c10 0x7fd7c8102020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009990 tx=0x7fd7b8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 -- 172.21.15.150:0/3827634115 shutdown_connections 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 --2- 172.21.15.150:0/3827634115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c81025b0 0x7fd7c8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 --2- 172.21.15.150:0/3827634115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c8100af0 0x7fd7c8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 --2- 172.21.15.150:0/3827634115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c8101c10 0x7fd7c8102020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 -- 172.21.15.150:0/3827634115 >> 172.21.15.150:0/3827634115 conn(0x7fd7c80fc0e0 msgr2=0x7fd7c80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 -- 172.21.15.150:0/3827634115 shutdown_connections 2024-05-10T04:38:05.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.272+0000 7fd7cd549700 1 -- 172.21.15.150:0/3827634115 wait complete. 2024-05-10T04:38:05.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 Processor -- start 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 -- start start 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c8100af0 0x7fd7c819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c8101c10 0x7fd7c819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c81025b0 0x7fd7c81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81a1ca0 con 0x7fd7c81025b0 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81a1e10 con 0x7fd7c8100af0 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7cd549700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81a1fd0 con 0x7fd7c8101c10 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7c77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c81025b0 0x7fd7c81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7c6ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c8100af0 0x7fd7c819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7c77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c81025b0 0x7fd7c81a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44584/0 (socket says 172.21.15.150:44584) 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7c77fe700 1 -- 172.21.15.150:0/1449168178 learned_addr learned my addr 172.21.15.150:0/1449168178 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.273+0000 7fd7c67fc700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c8101c10 0x7fd7c819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7c6ffd700 1 -- 172.21.15.150:0/1449168178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c8101c10 msgr2=0x7fd7c819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7c6ffd700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c8101c10 0x7fd7c819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7c6ffd700 1 -- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c81025b0 msgr2=0x7fd7c81a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7c6ffd700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c81025b0 0x7fd7c81a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7c6ffd700 1 -- 172.21.15.150:0/1449168178 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80095f0 con 0x7fd7c8100af0 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7c77fe700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c81025b0 0x7fd7c81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7c6ffd700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c8100af0 0x7fd7c819a1c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7c81007e0 tx=0x7fd7bc00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7affff700 1 -- 172.21.15.150:0/1449168178 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc004220 con 0x7fd7c8100af0 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c81a22b0 con 0x7fd7c8100af0 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7affff700 1 -- 172.21.15.150:0/1449168178 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7bc004380 con 0x7fd7c8100af0 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7affff700 1 -- 172.21.15.150:0/1449168178 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc003be0 con 0x7fd7c8100af0 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.274+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c81a27c0 con 0x7fd7c8100af0 2024-05-10T04:38:05.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.275+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c8068730 con 0x7fd7c8100af0 2024-05-10T04:38:05.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.276+0000 7fd7affff700 1 -- 172.21.15.150:0/1449168178 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd7bc003d40 con 0x7fd7c8100af0 2024-05-10T04:38:05.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.277+0000 7fd7affff700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd7b00a1120 0x7fd7b00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.277+0000 7fd7c67fc700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd7b00a1120 0x7fd7b00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.277+0000 7fd7affff700 1 -- 172.21.15.150:0/1449168178 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7bc014070 con 0x7fd7c8100af0 2024-05-10T04:38:05.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.277+0000 7fd7c67fc700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd7b00a1120 0x7fd7b00a3630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800b2e0 tx=0x7fd7b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:05.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.280+0000 7fd7affff700 1 -- 172.21.15.150:0/1449168178 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd7bc0965f0 con 0x7fd7c8100af0 2024-05-10T04:38:05.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:05.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:38:05.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:05.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:05 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:38:05.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:05 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3886141539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:05.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.503+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7c81a2ae0 con 0x7fd7c8100af0 2024-05-10T04:38:05.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.504+0000 7fd7affff700 1 -- 172.21.15.150:0/1449168178 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18455 (secure 0 0 0) 0x7fd7bc099c10 con 0x7fd7c8100af0 2024-05-10T04:38:05.505 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:05.505 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":78,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:37:36.584832+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:05.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd7b00a1120 msgr2=0x7fd7b00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd7b00a1120 0x7fd7b00a3630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800b2e0 tx=0x7fd7b801a040 comp rx=0 tx=0).stop 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c8100af0 msgr2=0x7fd7c819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c8100af0 0x7fd7c819a1c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7c81007e0 tx=0x7fd7bc00eca0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 shutdown_connections 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7c81025b0 0x7fd7c81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7c8100af0 0x7fd7c819a1c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd7b00a1120 0x7fd7b00a3630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 --2- 172.21.15.150:0/1449168178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c8101c10 0x7fd7c819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 >> 172.21.15.150:0/1449168178 conn(0x7fd7c80fc0e0 msgr2=0x7fd7c80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:05.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.507+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 shutdown_connections 2024-05-10T04:38:05.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.508+0000 7fd7cd549700 1 -- 172.21.15.150:0/1449168178 wait complete. 2024-05-10T04:38:05.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-10T04:38:05.523 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:38:05.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:05.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:38:05.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:05.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:05 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:38:05.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:05 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3886141539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:05.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:05.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:38:05.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:05.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:05 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:38:05.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:05 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3886141539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:05.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 -- 172.21.15.150:0/2430221209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81002c0 msgr2=0x7fa5f81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 --2- 172.21.15.150:0/2430221209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81002c0 0x7fa5f81006d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400aaa0 tx=0x7fa5f400adb0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 -- 172.21.15.150:0/2430221209 shutdown_connections 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 --2- 172.21.15.150:0/2430221209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5f8101d80 0x7fa5f8108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 --2- 172.21.15.150:0/2430221209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81002c0 0x7fa5f81006d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 --2- 172.21.15.150:0/2430221209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f81013e0 0x7fa5f81017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 -- 172.21.15.150:0/2430221209 >> 172.21.15.150:0/2430221209 conn(0x7fa5f80fb8f0 msgr2=0x7fa5f80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 -- 172.21.15.150:0/2430221209 shutdown_connections 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 -- 172.21.15.150:0/2430221209 wait complete. 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 Processor -- start 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.691+0000 7fa6001c4700 1 -- start start 2024-05-10T04:38:05.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa6001c4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5f81002c0 0x7fa5f8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa6001c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81013e0 0x7fa5f8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa6001c4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8101d80 0x7fa5f819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa6001c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f819f9e0 con 0x7fa5f81013e0 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa6001c4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f819fb50 con 0x7fa5f8101d80 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa6001c4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f819fd10 con 0x7fa5f81002c0 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81013e0 0x7fa5f8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fdf60700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5f81002c0 0x7fa5f8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81013e0 0x7fa5f8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44600/0 (socket says 172.21.15.150:44600) 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 -- 172.21.15.150:0/210488295 learned_addr learned my addr 172.21.15.150:0/210488295 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fe761700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8101d80 0x7fa5f819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 -- 172.21.15.150:0/210488295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5f81002c0 msgr2=0x7fa5f8197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5f81002c0 0x7fa5f8197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 -- 172.21.15.150:0/210488295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8101d80 msgr2=0x7fa5f819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8101d80 0x7fa5f819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 -- 172.21.15.150:0/210488295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f400a700 con 0x7fa5f81013e0 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fdf60700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5f81002c0 0x7fa5f8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5fd75f700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81013e0 0x7fa5f8198440 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800e9a0 tx=0x7fa5e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.692+0000 7fa5eeffd700 1 -- 172.21.15.150:0/210488295 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8009790 con 0x7fa5f81013e0 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.693+0000 7fa5eeffd700 1 -- 172.21.15.150:0/210488295 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5e8010430 con 0x7fa5f81013e0 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.693+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f819fff0 con 0x7fa5f81013e0 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.693+0000 7fa5eeffd700 1 -- 172.21.15.150:0/210488295 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8018680 con 0x7fa5f81013e0 2024-05-10T04:38:05.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.693+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f81a0470 con 0x7fa5f81013e0 2024-05-10T04:38:05.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.696+0000 7fa5eeffd700 1 -- 172.21.15.150:0/210488295 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa5e80105a0 con 0x7fa5f81013e0 2024-05-10T04:38:05.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.696+0000 7fa5eeffd700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa5e40a1240 0x7fa5e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:05.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.697+0000 7fa5eeffd700 1 -- 172.21.15.150:0/210488295 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5e8014070 con 0x7fa5f81013e0 2024-05-10T04:38:05.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.697+0000 7fa5fdf60700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa5e40a1240 0x7fa5e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:05.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.697+0000 7fa5fdf60700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa5e40a1240 0x7fa5e40a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400a9f0 tx=0x7fa5f4003c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:05.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.699+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f81074d0 con 0x7fa5f81013e0 2024-05-10T04:38:05.705 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.705+0000 7fa5eeffd700 1 -- 172.21.15.150:0/210488295 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa5e8052d60 con 0x7fa5f81013e0 2024-05-10T04:38:05.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.915+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5f80678b0 con 0x7fa5f81013e0 2024-05-10T04:38:05.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.915+0000 7fa5eeffd700 1 -- 172.21.15.150:0/210488295 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5e8099c40 con 0x7fa5f81013e0 2024-05-10T04:38:05.915 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa5e40a1240 msgr2=0x7fa5e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa5e40a1240 0x7fa5e40a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400a9f0 tx=0x7fa5f4003c90 comp rx=0 tx=0).stop 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81013e0 msgr2=0x7fa5f8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81013e0 0x7fa5f8198440 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800e9a0 tx=0x7fa5e800ecb0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 shutdown_connections 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f8101d80 0x7fa5f819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa5e40a1240 0x7fa5e40a3750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5f81002c0 0x7fa5f8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 --2- 172.21.15.150:0/210488295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5f81013e0 0x7fa5f8198440 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 >> 172.21.15.150:0/210488295 conn(0x7fa5f80fb8f0 msgr2=0x7fa5f8104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 shutdown_connections 2024-05-10T04:38:05.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:05.917+0000 7fa6001c4700 1 -- 172.21.15.150:0/210488295 wait complete. 2024-05-10T04:38:05.930 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-10T04:38:06.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 -- 172.21.15.150:0/50245467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 msgr2=0x7f87e01017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 --2- 172.21.15.150:0/50245467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e01017f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009990 tx=0x7f87d0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 -- 172.21.15.150:0/50245467 shutdown_connections 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 --2- 172.21.15.150:0/50245467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87e0101d80 0x7f87e0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 --2- 172.21.15.150:0/50245467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e01002c0 0x7f87e01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 --2- 172.21.15.150:0/50245467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e01017f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 -- 172.21.15.150:0/50245467 >> 172.21.15.150:0/50245467 conn(0x7f87e00fb8f0 msgr2=0x7f87e00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 -- 172.21.15.150:0/50245467 shutdown_connections 2024-05-10T04:38:06.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.064+0000 7f87e4c70700 1 -- 172.21.15.150:0/50245467 wait complete. 2024-05-10T04:38:06.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 Processor -- start 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 -- start start 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e01002c0 0x7f87e0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87e0101d80 0x7f87e019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e019fab0 con 0x7f87e01013e0 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e019fc20 con 0x7f87e0101d80 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87e4c70700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e019fde0 con 0x7f87e01002c0 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e0198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44612/0 (socket says 172.21.15.150:44612) 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 -- 172.21.15.150:0/1080932251 learned_addr learned my addr 172.21.15.150:0/1080932251 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ded9d700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87e0101d80 0x7f87e019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87de59c700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e01002c0 0x7f87e0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 -- 172.21.15.150:0/1080932251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e01002c0 msgr2=0x7f87e0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e01002c0 0x7f87e0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 -- 172.21.15.150:0/1080932251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87e0101d80 msgr2=0x7f87e019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87e0101d80 0x7f87e019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.065+0000 7f87ddd9b700 1 -- 172.21.15.150:0/1080932251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00095f0 con 0x7f87e01013e0 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.066+0000 7f87ddd9b700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e0198510 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87d00098e0 tx=0x7f87d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.066+0000 7f87cf7fe700 1 -- 172.21.15.150:0/1080932251 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d001d070 con 0x7f87e01013e0 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.066+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e01a0060 con 0x7f87e01013e0 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.066+0000 7f87cf7fe700 1 -- 172.21.15.150:0/1080932251 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87d0021410 con 0x7f87e01013e0 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.066+0000 7f87cf7fe700 1 -- 172.21.15.150:0/1080932251 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0017650 con 0x7f87e01013e0 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.066+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01a0490 con 0x7f87e01013e0 2024-05-10T04:38:06.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.067+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e00678b0 con 0x7f87e01013e0 2024-05-10T04:38:06.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.069+0000 7f87cf7fe700 1 -- 172.21.15.150:0/1080932251 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f87d0021580 con 0x7f87e01013e0 2024-05-10T04:38:06.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.069+0000 7f87cf7fe700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87c80a1120 0x7f87c80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.069+0000 7f87de59c700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87c80a1120 0x7f87c80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.069+0000 7f87cf7fe700 1 -- 172.21.15.150:0/1080932251 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87d00cdf00 con 0x7f87e01013e0 2024-05-10T04:38:06.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.070+0000 7f87de59c700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87c80a1120 0x7f87c80a3630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005950 tx=0x7f87d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:06.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.073+0000 7f87cf7fe700 1 -- 172.21.15.150:0/1080932251 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f87d0097bc0 con 0x7f87e01013e0 2024-05-10T04:38:06.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.270+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f87e010ba60 con 0x7f87e01013e0 2024-05-10T04:38:06.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.270+0000 7f87cf7fe700 1 -- 172.21.15.150:0/1080932251 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f87d009b1e0 con 0x7f87e01013e0 2024-05-10T04:38:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:06.271 INFO:teuthology.orchestra.run.smithi150.stdout:[ 2024-05-10T04:38:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:38:06.271 INFO:teuthology.orchestra.run.smithi150.stdout: "poolnum": 1, 2024-05-10T04:38:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: "poolname": ".mgr" 2024-05-10T04:38:06.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:38:06.272 INFO:teuthology.orchestra.run.smithi150.stdout:] 2024-05-10T04:38:06.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.272+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87c80a1120 msgr2=0x7f87c80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.272+0000 7f87e4c70700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87c80a1120 0x7f87c80a3630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005950 tx=0x7f87d40058e0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.272+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 msgr2=0x7f87e0198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e0198510 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87d00098e0 tx=0x7f87d0009580 comp rx=0 tx=0).stop 2024-05-10T04:38:06.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 shutdown_connections 2024-05-10T04:38:06.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87e0101d80 0x7f87e019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87c80a1120 0x7f87c80a3630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e01002c0 0x7f87e0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 --2- 172.21.15.150:0/1080932251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87e01013e0 0x7f87e0198510 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 >> 172.21.15.150:0/1080932251 conn(0x7f87e00fb8f0 msgr2=0x7f87e00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:06.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 shutdown_connections 2024-05-10T04:38:06.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.273+0000 7f87e4c70700 1 -- 172.21.15.150:0/1080932251 wait complete. 2024-05-10T04:38:06.286 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-10T04:38:06.286 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-10T04:38:06.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 -- 172.21.15.150:0/1414945102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1025d0 msgr2=0x7f118c109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 --2- 172.21.15.150:0/1414945102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1025d0 0x7f118c109090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f118400b0a0 tx=0x7f118400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 -- 172.21.15.150:0/1414945102 shutdown_connections 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 --2- 172.21.15.150:0/1414945102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1025d0 0x7f118c109090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 --2- 172.21.15.150:0/1414945102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c1010d0 0x7f118c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 --2- 172.21.15.150:0/1414945102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f118c101ab0 0x7f118c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 -- 172.21.15.150:0/1414945102 >> 172.21.15.150:0/1414945102 conn(0x7f118c0fb8f0 msgr2=0x7f118c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 -- 172.21.15.150:0/1414945102 shutdown_connections 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 -- 172.21.15.150:0/1414945102 wait complete. 2024-05-10T04:38:06.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 Processor -- start 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.424+0000 7f11928fe700 1 -- start start 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f11928fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f118c1010d0 0x7f118c100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f11928fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c101ab0 0x7f118c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f11928fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c1025d0 0x7f118c19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f11928fe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19fac0 con 0x7f118c101ab0 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f11928fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19fc30 con 0x7f118c1010d0 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f11928fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19fdf0 con 0x7f118c1025d0 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c101ab0 0x7f118c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c101ab0 0x7f118c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44632/0 (socket says 172.21.15.150:44632) 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 -- 172.21.15.150:0/3724958418 learned_addr learned my addr 172.21.15.150:0/3724958418 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118bfff700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f118c1010d0 0x7f118c100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f1190e9b700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c1025d0 0x7f118c19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 -- 172.21.15.150:0/3724958418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c1025d0 msgr2=0x7f118c19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c1025d0 0x7f118c19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 -- 172.21.15.150:0/3724958418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f118c1010d0 msgr2=0x7f118c100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f118c1010d0 0x7f118c100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.425+0000 7f118b7fe700 1 -- 172.21.15.150:0/3724958418 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184009d00 con 0x7f118c101ab0 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.426+0000 7f118b7fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c101ab0 0x7f118c19cde0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f117c00d950 tx=0x7f117c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.426+0000 7f11897fa700 1 -- 172.21.15.150:0/3724958418 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00f890 con 0x7f118c101ab0 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.426+0000 7f11897fa700 1 -- 172.21.15.150:0/3724958418 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f117c010460 con 0x7f118c101ab0 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.426+0000 7f11897fa700 1 -- 172.21.15.150:0/3724958418 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00e610 con 0x7f118c101ab0 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.426+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c1a00d0 con 0x7f118c101ab0 2024-05-10T04:38:06.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.426+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c1a05e0 con 0x7f118c101ab0 2024-05-10T04:38:06.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.427+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c107f30 con 0x7f118c101ab0 2024-05-10T04:38:06.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.428+0000 7f11897fa700 1 -- 172.21.15.150:0/3724958418 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f117c0108d0 con 0x7f118c101ab0 2024-05-10T04:38:06.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.428+0000 7f11897fa700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11740a1240 0x7f11740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:06.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.429+0000 7f118bfff700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11740a1240 0x7f11740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:06.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.429+0000 7f11897fa700 1 -- 172.21.15.150:0/3724958418 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f117c0cc1c0 con 0x7f118c101ab0 2024-05-10T04:38:06.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.429+0000 7f118bfff700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11740a1240 0x7f11740a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f118c0680e0 tx=0x7f118000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:06.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.432+0000 7f11897fa700 1 -- 172.21.15.150:0/3724958418 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f117c095df0 con 0x7f118c101ab0 2024-05-10T04:38:06.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1449168178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/210488295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:06.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:06 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1080932251' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T04:38:06.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:06.634+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f118c0678b0 con 0x7f118c101ab0 2024-05-10T04:38:06.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1449168178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/210488295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:06.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:06 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1080932251' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T04:38:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1449168178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/210488295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:06.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:06 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1080932251' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-10T04:38:07.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.325+0000 7f11897fa700 1 -- 172.21.15.150:0/3724958418 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f117c099410 con 0x7f118c101ab0 2024-05-10T04:38:07.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11740a1240 msgr2=0x7f11740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11740a1240 0x7f11740a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f118c0680e0 tx=0x7f118000a380 comp rx=0 tx=0).stop 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c101ab0 msgr2=0x7f118c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c101ab0 0x7f118c19cde0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f117c00d950 tx=0x7f117c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 shutdown_connections 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11740a1240 0x7f11740a3750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c1025d0 0x7f118c19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f118c1010d0 0x7f118c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 --2- 172.21.15.150:0/3724958418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c101ab0 0x7f118c19cde0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.329+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 >> 172.21.15.150:0/3724958418 conn(0x7f118c0fb8f0 msgr2=0x7f118c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:07.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.330+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 shutdown_connections 2024-05-10T04:38:07.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.330+0000 7f11928fe700 1 -- 172.21.15.150:0/3724958418 wait complete. 2024-05-10T04:38:07.331 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' created 2024-05-10T04:38:07.345 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 -- 172.21.15.150:0/14460988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c100ac0 msgr2=0x7f922c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 --2- 172.21.15.150:0/14460988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c100ac0 0x7f922c100ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f921c009990 tx=0x7f921c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 -- 172.21.15.150:0/14460988 shutdown_connections 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 --2- 172.21.15.150:0/14460988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c102580 0x7f922c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 --2- 172.21.15.150:0/14460988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c100ac0 0x7f922c100ed0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 --2- 172.21.15.150:0/14460988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f922c101be0 0x7f922c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 -- 172.21.15.150:0/14460988 >> 172.21.15.150:0/14460988 conn(0x7f922c0fc0d0 msgr2=0x7f922c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 -- 172.21.15.150:0/14460988 shutdown_connections 2024-05-10T04:38:07.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.485+0000 7f92340e9700 1 -- 172.21.15.150:0/14460988 wait complete. 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 Processor -- start 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 -- start start 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f922c100ac0 0x7f922c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c101be0 0x7f922c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c102580 0x7f922c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c19e910 con 0x7f922c101be0 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c19ea80 con 0x7f922c102580 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f92340e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c19ec40 con 0x7f922c100ac0 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.486+0000 7f9231684700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c101be0 0x7f922c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c101be0 0x7f922c197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54030/0 (socket says 172.21.15.150:54030) 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 -- 172.21.15.150:0/1592129755 learned_addr learned my addr 172.21.15.150:0/1592129755 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231e85700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f922c100ac0 0x7f922c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:07.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9232686700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c102580 0x7f922c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 -- 172.21.15.150:0/1592129755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f922c100ac0 msgr2=0x7f922c196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f922c100ac0 0x7f922c196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 -- 172.21.15.150:0/1592129755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c102580 msgr2=0x7f922c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c102580 0x7f922c19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 -- 172.21.15.150:0/1592129755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0095f0 con 0x7f922c101be0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9231684700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c101be0 0x7f922c197370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f922800d950 tx=0x7f922800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9222ffd700 1 -- 172.21.15.150:0/1592129755 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922800f890 con 0x7f922c101be0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9222ffd700 1 -- 172.21.15.150:0/1592129755 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9228010460 con 0x7f922c101be0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c19ef20 con 0x7f922c101be0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f9222ffd700 1 -- 172.21.15.150:0/1592129755 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922800e610 con 0x7f922c101be0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.487+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c19f3a0 con 0x7f922c101be0 2024-05-10T04:38:07.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.488+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c107cd0 con 0x7f922c101be0 2024-05-10T04:38:07.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.489+0000 7f9222ffd700 1 -- 172.21.15.150:0/1592129755 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f92280108d0 con 0x7f922c101be0 2024-05-10T04:38:07.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.490+0000 7f9222ffd700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92180a1240 0x7f92180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:07.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.490+0000 7f9222ffd700 1 -- 172.21.15.150:0/1592129755 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f92280cc360 con 0x7f922c101be0 2024-05-10T04:38:07.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.490+0000 7f9231e85700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92180a1240 0x7f92180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:07.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.490+0000 7f9231e85700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92180a1240 0x7f92180a3750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f921c0038c0 tx=0x7f921c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:07.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.494+0000 7f9222ffd700 1 -- 172.21.15.150:0/1592129755 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9228095d80 con 0x7f922c101be0 2024-05-10T04:38:07.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:07.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:07 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3724958418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T04:38:07.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:07.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:07 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3724958418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T04:38:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:38:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:07 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3724958418' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-10T04:38:07.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:07.693+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f922c02ce20 con 0x7f922c101be0 2024-05-10T04:38:08.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.327+0000 7f9222ffd700 1 -- 172.21.15.150:0/1592129755 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) v1 ==== 171+0+0 (secure 0 0 0) 0x7f92280993a0 con 0x7f922c101be0 2024-05-10T04:38:08.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.339+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92180a1240 msgr2=0x7f92180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.339+0000 7f92340e9700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92180a1240 0x7f92180a3750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f921c0038c0 tx=0x7f921c01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:08.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.339+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c101be0 msgr2=0x7f922c197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.339+0000 7f92340e9700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c101be0 0x7f922c197370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f922800d950 tx=0x7f922800dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:08.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.342+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 shutdown_connections 2024-05-10T04:38:08.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.342+0000 7f92340e9700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f922c102580 0x7f922c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.342+0000 7f92340e9700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92180a1240 0x7f92180a3750 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.342+0000 7f92340e9700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f922c100ac0 0x7f922c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.342+0000 7f92340e9700 1 --2- 172.21.15.150:0/1592129755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f922c101be0 0x7f922c197370 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f922800d950 tx=0x7f922800dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:08.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.342+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 >> 172.21.15.150:0/1592129755 conn(0x7f922c0fc0d0 msgr2=0x7f922c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:08.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.347+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 shutdown_connections 2024-05-10T04:38:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.348+0000 7f92340e9700 1 -- 172.21.15.150:0/1592129755 wait complete. 2024-05-10T04:38:08.350 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' created 2024-05-10T04:38:08.363 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 -- 172.21.15.150:0/1330557122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 msgr2=0x7f5994108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 --2- 172.21.15.150:0/1330557122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f5994108e30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f598c00b0a0 tx=0x7f598c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 -- 172.21.15.150:0/1330557122 shutdown_connections 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 --2- 172.21.15.150:0/1330557122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f5994108e30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 --2- 172.21.15.150:0/1330557122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5994100ac0 0x7f5994100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 --2- 172.21.15.150:0/1330557122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101be0 0x7f5994101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 -- 172.21.15.150:0/1330557122 >> 172.21.15.150:0/1330557122 conn(0x7f59940fc0d0 msgr2=0x7f59940fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.498+0000 7f599c659700 1 -- 172.21.15.150:0/1330557122 shutdown_connections 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 -- 172.21.15.150:0/1330557122 wait complete. 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 Processor -- start 2024-05-10T04:38:08.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 -- start start 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5994100ac0 0x7f5994068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101be0 0x7f599419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f599419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599419fa90 con 0x7f5994102580 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599419fc00 con 0x7f5994100ac0 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599c659700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599419fdc0 con 0x7f5994101be0 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599abf6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f599419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599abf6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f599419f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54048/0 (socket says 172.21.15.150:54048) 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.499+0000 7f599abf6700 1 -- 172.21.15.150:0/3389736106 learned_addr learned my addr 172.21.15.150:0/3389736106 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599a3f5700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5994100ac0 0x7f5994068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f5999bf4700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101be0 0x7f599419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599abf6700 1 -- 172.21.15.150:0/3389736106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101be0 msgr2=0x7f599419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599abf6700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101be0 0x7f599419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599abf6700 1 -- 172.21.15.150:0/3389736106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5994100ac0 msgr2=0x7f5994068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599abf6700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5994100ac0 0x7f5994068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599abf6700 1 -- 172.21.15.150:0/3389736106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c009d00 con 0x7f5994102580 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599a3f5700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5994100ac0 0x7f5994068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599abf6700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f599419f390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f598c00b070 tx=0x7f598c0079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f598b7fe700 1 -- 172.21.15.150:0/3389736106 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c00e040 con 0x7f5994102580 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f598b7fe700 1 -- 172.21.15.150:0/3389736106 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598c003c00 con 0x7f5994102580 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f598b7fe700 1 -- 172.21.15.150:0/3389736106 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c01b6c0 con 0x7f5994102580 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59941a0040 con 0x7f5994102580 2024-05-10T04:38:08.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.500+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59941a0410 con 0x7f5994102580 2024-05-10T04:38:08.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.501+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5994107cd0 con 0x7f5994102580 2024-05-10T04:38:08.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.502+0000 7f598b7fe700 1 -- 172.21.15.150:0/3389736106 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f598c01b8e0 con 0x7f5994102580 2024-05-10T04:38:08.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.503+0000 7f598b7fe700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f59800a1240 0x7f59800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.503+0000 7f598b7fe700 1 -- 172.21.15.150:0/3389736106 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f598c0cdb30 con 0x7f5994102580 2024-05-10T04:38:08.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.503+0000 7f599a3f5700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f59800a1240 0x7f59800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.503+0000 7f599a3f5700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f59800a1240 0x7f59800a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5994101ae0 tx=0x7f5984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:08.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.507+0000 7f598b7fe700 1 -- 172.21.15.150:0/3389736106 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f598c097460 con 0x7f5994102580 2024-05-10T04:38:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:08 smithi181 ceph-mon[47314]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:08 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3724958418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T04:38:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:08 smithi181 ceph-mon[47314]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T04:38:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:08 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1592129755' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T04:38:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:08 smithi150 ceph-mon[42404]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:08 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3724958418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T04:38:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:08 smithi150 ceph-mon[42404]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T04:38:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:08 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1592129755' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T04:38:08.749 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.749+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f599410c450 con 0x7f5994102580 2024-05-10T04:38:08.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.758+0000 7f598b7fe700 1 -- 172.21.15.150:0/3389736106 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f598c09aa80 con 0x7f5994102580 2024-05-10T04:38:08.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.769+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f59800a1240 msgr2=0x7f59800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.769+0000 7f599c659700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f59800a1240 0x7f59800a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5994101ae0 tx=0x7f5984009260 comp rx=0 tx=0).stop 2024-05-10T04:38:08.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.769+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 msgr2=0x7f599419f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.769+0000 7f599c659700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f599419f390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f598c00b070 tx=0x7f598c0079b0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.771+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 shutdown_connections 2024-05-10T04:38:08.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.771+0000 7f599c659700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5994102580 0x7f599419f390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.771+0000 7f599c659700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5994100ac0 0x7f5994068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.771+0000 7f599c659700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f59800a1240 0x7f59800a3750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.771+0000 7f599c659700 1 --2- 172.21.15.150:0/3389736106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5994101be0 0x7f599419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.771+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 >> 172.21.15.150:0/3389736106 conn(0x7f59940fc0d0 msgr2=0x7f5994104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:08.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.775+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 shutdown_connections 2024-05-10T04:38:08.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.775+0000 7f599c659700 1 -- 172.21.15.150:0/3389736106 wait complete. 2024-05-10T04:38:08.783 INFO:teuthology.orchestra.run.smithi150.stderr:new fs with metadata pool 2 and data pool 3 2024-05-10T04:38:08.796 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-10T04:38:08.796 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-10T04:38:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:08 smithi161 ceph-mon[46912]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:08.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:08 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3724958418' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-10T04:38:08.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:08 smithi161 ceph-mon[46912]: osdmap e79: 12 total, 12 up, 12 in 2024-05-10T04:38:08.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:08 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1592129755' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-10T04:38:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 -- 172.21.15.150:0/4068792835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 msgr2=0x7f6888103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 --2- 172.21.15.150:0/4068792835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888103110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f687c009990 tx=0x7f687c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 -- 172.21.15.150:0/4068792835 shutdown_connections 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 --2- 172.21.15.150:0/4068792835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68881036a0 0x7f6888109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 --2- 172.21.15.150:0/4068792835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6888101be0 0x7f6888101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 --2- 172.21.15.150:0/4068792835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888103110 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 -- 172.21.15.150:0/4068792835 >> 172.21.15.150:0/4068792835 conn(0x7f68880fd210 msgr2=0x7f68880ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 -- 172.21.15.150:0/4068792835 shutdown_connections 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.934+0000 7f688e59e700 1 -- 172.21.15.150:0/4068792835 wait complete. 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 Processor -- start 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 -- start start 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6888101be0 0x7f6888074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68881036a0 0x7f6888073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888074180 con 0x7f6888102d00 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68880742f0 con 0x7f68881036a0 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688e59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881a1fd0 con 0x7f6888101be0 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f68877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f68877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888073560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54070/0 (socket says 172.21.15.150:54070) 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f68877fe700 1 -- 172.21.15.150:0/2240341210 learned_addr learned my addr 172.21.15.150:0/2240341210 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.935+0000 7f688cb3b700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68881036a0 0x7f6888073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f6887fff700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6888101be0 0x7f6888074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68877fe700 1 -- 172.21.15.150:0/2240341210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6888101be0 msgr2=0x7f6888074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68877fe700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6888101be0 0x7f6888074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68877fe700 1 -- 172.21.15.150:0/2240341210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68881036a0 msgr2=0x7f6888073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68877fe700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68881036a0 0x7f6888073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68877fe700 1 -- 172.21.15.150:0/2240341210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0095f0 con 0x7f6888102d00 2024-05-10T04:38:08.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f688cb3b700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68881036a0 0x7f6888073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68877fe700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888073560 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68857fa700 1 -- 172.21.15.150:0/2240341210 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c01d070 con 0x7f6888102d00 2024-05-10T04:38:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68857fa700 1 -- 172.21.15.150:0/2240341210 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f687c00fbd0 con 0x7f6888102d00 2024-05-10T04:38:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68881a21d0 con 0x7f6888102d00 2024-05-10T04:38:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f68857fa700 1 -- 172.21.15.150:0/2240341210 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c017650 con 0x7f6888102d00 2024-05-10T04:38:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.936+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68881a25f0 con 0x7f6888102d00 2024-05-10T04:38:08.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.937+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6888108df0 con 0x7f6888102d00 2024-05-10T04:38:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.938+0000 7f68857fa700 1 -- 172.21.15.150:0/2240341210 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f687c00fd40 con 0x7f6888102d00 2024-05-10T04:38:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.939+0000 7f68857fa700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f68700a11c0 0x7f68700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.939+0000 7f68857fa700 1 -- 172.21.15.150:0/2240341210 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f687c0ce2f0 con 0x7f6888102d00 2024-05-10T04:38:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.939+0000 7f6887fff700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f68700a11c0 0x7f68700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:08.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.939+0000 7f6887fff700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f68700a11c0 0x7f68700a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6888102c00 tx=0x7f6878009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:08.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:08.942+0000 7f68857fa700 1 -- 172.21.15.150:0/2240341210 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f687c097c60 con 0x7f6888102d00 2024-05-10T04:38:09.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.150+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f6888068730 con 0x7f6888102d00 2024-05-10T04:38:09.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1592129755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3389736106' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3389736106' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T04:38:09.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:09 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2240341210' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T04:38:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1592129755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T04:38:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T04:38:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3389736106' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T04:38:09.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T04:38:09.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T04:38:09.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3389736106' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T04:38:09.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T04:38:09.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T04:38:09.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:09 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2240341210' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T04:38:09.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.758+0000 7f68857fa700 1 -- 172.21.15.150:0/2240341210 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v82) v1 ==== 157+0+0 (secure 0 0 0) 0x7f687c09b280 con 0x7f6888102d00 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f68700a11c0 msgr2=0x7f68700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f68700a11c0 0x7f68700a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6888102c00 tx=0x7f6878009260 comp rx=0 tx=0).stop 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 msgr2=0x7f6888073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888073560 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c009580 comp rx=0 tx=0).stop 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 shutdown_connections 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68881036a0 0x7f6888073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f68700a11c0 0x7f68700a36d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6888101be0 0x7f6888074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 --2- 172.21.15.150:0/2240341210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6888102d00 0x7f6888073560 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 >> 172.21.15.150:0/2240341210 conn(0x7f68880fd210 msgr2=0x7f6888105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 shutdown_connections 2024-05-10T04:38:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.761+0000 7f688e59e700 1 -- 172.21.15.150:0/2240341210 wait complete. 2024-05-10T04:38:09.776 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-10T04:38:09.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1592129755' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-10T04:38:09.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: osdmap e80: 12 total, 12 up, 12 in 2024-05-10T04:38:09.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3389736106' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-10T04:38:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-10T04:38:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: osdmap e81: 12 total, 12 up, 12 in 2024-05-10T04:38:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3389736106' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-10T04:38:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-10T04:38:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-10T04:38:09.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:09 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2240341210' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-10T04:38:09.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2862914762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 msgr2=0x7f0e44102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:09.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2862914762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44102010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009990 tx=0x7f0e34009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2862914762 shutdown_connections 2024-05-10T04:38:09.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2862914762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e441036c0 0x7f0e44109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2862914762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44102010 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2862914762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e44102d20 0x7f0e44103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2862914762 >> 172.21.15.150:0/2862914762 conn(0x7f0e440fd210 msgr2=0x7f0e440ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2862914762 shutdown_connections 2024-05-10T04:38:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.919+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2862914762 wait complete. 2024-05-10T04:38:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.920+0000 7f0e4b81b700 1 Processor -- start 2024-05-10T04:38:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.920+0000 7f0e4b81b700 1 -- start start 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.920+0000 7f0e4b81b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.920+0000 7f0e4b81b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e44102d20 0x7f0e44198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.920+0000 7f0e4b81b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e441036c0 0x7f0e4419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e4b81b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4419fa10 con 0x7f0e44101c00 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e4b81b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4419fb50 con 0x7f0e441036c0 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e4b81b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4419fce0 con 0x7f0e44102d20 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54096/0 (socket says 172.21.15.150:54096) 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 -- 172.21.15.150:0/2438479676 learned_addr learned my addr 172.21.15.150:0/2438479676 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e48db6700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e44102d20 0x7f0e44198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e49db8700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e441036c0 0x7f0e4419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 -- 172.21.15.150:0/2438479676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e44102d20 msgr2=0x7f0e44198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e44102d20 0x7f0e44198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 -- 172.21.15.150:0/2438479676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e441036c0 msgr2=0x7f0e4419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e441036c0 0x7f0e4419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 -- 172.21.15.150:0/2438479676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340095f0 con 0x7f0e44101c00 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e49db8700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e441036c0 0x7f0e4419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e495b7700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44198000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400bc60 tx=0x7f0e34003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e3a7fc700 1 -- 172.21.15.150:0/2438479676 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3401d070 con 0x7f0e44101c00 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e3a7fc700 1 -- 172.21.15.150:0/2438479676 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e3400f460 con 0x7f0e44101c00 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4419ff60 con 0x7f0e44101c00 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e3a7fc700 1 -- 172.21.15.150:0/2438479676 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34017650 con 0x7f0e44101c00 2024-05-10T04:38:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.921+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e441a0320 con 0x7f0e44101c00 2024-05-10T04:38:09.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.922+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44108e10 con 0x7f0e44101c00 2024-05-10T04:38:09.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.924+0000 7f0e3a7fc700 1 -- 172.21.15.150:0/2438479676 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0e3400f5d0 con 0x7f0e44101c00 2024-05-10T04:38:09.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.924+0000 7f0e3a7fc700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0e300a11a0 0x7f0e300a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:09.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.924+0000 7f0e48db6700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0e300a11a0 0x7f0e300a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:09.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.924+0000 7f0e3a7fc700 1 -- 172.21.15.150:0/2438479676 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f0e340cde30 con 0x7f0e44101c00 2024-05-10T04:38:09.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.925+0000 7f0e48db6700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0e300a11a0 0x7f0e300a36b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005ea0 tx=0x7f0e40005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:09.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:09.928+0000 7f0e3a7fc700 1 -- 172.21.15.150:0/2438479676 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0e34097670 con 0x7f0e44101c00 2024-05-10T04:38:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:10.128+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0e44068730 con 0x7f0e44101c00 2024-05-10T04:38:10.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:10 smithi181 ceph-mon[47314]: pgmap v220: 97 pgs: 23 creating+peering, 7 active+clean, 67 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:10 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2240341210' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T04:38:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:10 smithi181 ceph-mon[47314]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T04:38:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:10 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] up:active 2024-05-10T04:38:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:10 smithi181 ceph-mon[47314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:10 smithi181 ceph-mon[47314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:10.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:10 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2438479676' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T04:38:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:10 smithi150 ceph-mon[42404]: pgmap v220: 97 pgs: 23 creating+peering, 7 active+clean, 67 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:10 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2240341210' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T04:38:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:10 smithi150 ceph-mon[42404]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T04:38:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:10 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] up:active 2024-05-10T04:38:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:10 smithi150 ceph-mon[42404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:10 smithi150 ceph-mon[42404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:10 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2438479676' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T04:38:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:10 smithi161 ceph-mon[46912]: pgmap v220: 97 pgs: 23 creating+peering, 7 active+clean, 67 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:38:10.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:10 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2240341210' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-10T04:38:10.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:10 smithi161 ceph-mon[46912]: osdmap e82: 12 total, 12 up, 12 in 2024-05-10T04:38:10.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:10 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] up:active 2024-05-10T04:38:10.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:10 smithi161 ceph-mon[46912]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:10.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:10 smithi161 ceph-mon[46912]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:10.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:10 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2438479676' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-10T04:38:11.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.783+0000 7f0e3a7fc700 1 -- 172.21.15.150:0/2438479676 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v84) v1 ==== 228+0+0 (secure 0 0 0) 0x7f0e3409ac90 con 0x7f0e44101c00 2024-05-10T04:38:11.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0e300a11a0 msgr2=0x7f0e300a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:11.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0e300a11a0 0x7f0e300a36b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005ea0 tx=0x7f0e40005e30 comp rx=0 tx=0).stop 2024-05-10T04:38:11.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 msgr2=0x7f0e44198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44198000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400bc60 tx=0x7f0e34003c40 comp rx=0 tx=0).stop 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 shutdown_connections 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e441036c0 0x7f0e4419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e44101c00 0x7f0e44198000 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0e300a11a0 0x7f0e300a36b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 --2- 172.21.15.150:0/2438479676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e44102d20 0x7f0e44198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.791+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 >> 172.21.15.150:0/2438479676 conn(0x7f0e440fd210 msgr2=0x7f0e44105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.792+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 shutdown_connections 2024-05-10T04:38:11.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.792+0000 7f0e4b81b700 1 -- 172.21.15.150:0/2438479676 wait complete. 2024-05-10T04:38:11.795 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data_ec' created 2024-05-10T04:38:11.808 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-10T04:38:11.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 -- 172.21.15.150:0/1352984034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 msgr2=0x7ff7581085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:11.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 --2- 172.21.15.150:0/1352984034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff7581085e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff75000b0a0 tx=0x7ff75000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 -- 172.21.15.150:0/1352984034 shutdown_connections 2024-05-10T04:38:11.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 --2- 172.21.15.150:0/1352984034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff7581085e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 --2- 172.21.15.150:0/1352984034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff758100270 0x7ff758100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 --2- 172.21.15.150:0/1352984034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff758101390 0x7ff7581017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 -- 172.21.15.150:0/1352984034 >> 172.21.15.150:0/1352984034 conn(0x7ff7580fb880 msgr2=0x7ff7580fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:11.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 -- 172.21.15.150:0/1352984034 shutdown_connections 2024-05-10T04:38:11.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.950+0000 7ff75d389700 1 -- 172.21.15.150:0/1352984034 wait complete. 2024-05-10T04:38:11.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.951+0000 7ff75d389700 1 Processor -- start 2024-05-10T04:38:11.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.951+0000 7ff75d389700 1 -- start start 2024-05-10T04:38:11.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.951+0000 7ff75d389700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff758100270 0x7ff758104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:11.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.951+0000 7ff75d389700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff758101390 0x7ff7581026d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:11.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.951+0000 7ff75d389700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff758102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:11.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff75d389700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581032f0 con 0x7ff758101d30 2024-05-10T04:38:11.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff75d389700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758103460 con 0x7ff758100270 2024-05-10T04:38:11.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff75d389700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581a20f0 con 0x7ff758101390 2024-05-10T04:38:11.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff758102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:11.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff758102c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54114/0 (socket says 172.21.15.150:54114) 2024-05-10T04:38:11.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 -- 172.21.15.150:0/4166915015 learned_addr learned my addr 172.21.15.150:0/4166915015 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:11.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff756ffd700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff758100270 0x7ff758104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:11.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7567fc700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff758101390 0x7ff7581026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:11.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 -- 172.21.15.150:0/4166915015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff758101390 msgr2=0x7ff7581026d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:11.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff758101390 0x7ff7581026d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 -- 172.21.15.150:0/4166915015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff758100270 msgr2=0x7ff758104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff758100270 0x7ff758104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 -- 172.21.15.150:0/4166915015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff750009d00 con 0x7ff758101d30 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7567fc700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff758101390 0x7ff7581026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff7577fe700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff758102c10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff750006070 tx=0x7ff7500078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff74ffff700 1 -- 172.21.15.150:0/4166915015 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75000e040 con 0x7ff758101d30 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff74ffff700 1 -- 172.21.15.150:0/4166915015 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff75000be40 con 0x7ff758101d30 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff74ffff700 1 -- 172.21.15.150:0/4166915015 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500052a0 con 0x7ff758101d30 2024-05-10T04:38:11.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7581a22f0 con 0x7ff758101d30 2024-05-10T04:38:11.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.952+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7581a2770 con 0x7ff758101d30 2024-05-10T04:38:11.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.953+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff758068730 con 0x7ff758101d30 2024-05-10T04:38:11.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.954+0000 7ff74ffff700 1 -- 172.21.15.150:0/4166915015 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff75001d470 con 0x7ff758101d30 2024-05-10T04:38:11.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.955+0000 7ff74ffff700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7440a1240 0x7ff7440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:11.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.955+0000 7ff756ffd700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7440a1240 0x7ff7440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:11.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.955+0000 7ff74ffff700 1 -- 172.21.15.150:0/4166915015 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7ff7500cdbc0 con 0x7ff758101d30 2024-05-10T04:38:11.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.955+0000 7ff756ffd700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7440a1240 0x7ff7440a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff758101290 tx=0x7ff740009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:11.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:11.959+0000 7ff74ffff700 1 -- 172.21.15.150:0/4166915015 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff750097290 con 0x7ff758101d30 2024-05-10T04:38:12.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:11 smithi161 ceph-mon[46912]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T04:38:12.087 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:11 smithi161 ceph-mon[46912]: pgmap v223: 97 pgs: 23 creating+peering, 57 active+clean, 17 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 771 B/s wr, 2 op/s 2024-05-10T04:38:12.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:11 smithi181 ceph-mon[47314]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T04:38:12.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:11 smithi181 ceph-mon[47314]: pgmap v223: 97 pgs: 23 creating+peering, 57 active+clean, 17 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 771 B/s wr, 2 op/s 2024-05-10T04:38:12.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.156+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7ff75810b360 con 0x7ff758101d30 2024-05-10T04:38:12.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:11 smithi150 ceph-mon[42404]: osdmap e83: 12 total, 12 up, 12 in 2024-05-10T04:38:12.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:11 smithi150 ceph-mon[42404]: pgmap v223: 97 pgs: 23 creating+peering, 57 active+clean, 17 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 771 B/s wr, 2 op/s 2024-05-10T04:38:12.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.782+0000 7ff74ffff700 1 -- 172.21.15.150:0/4166915015 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff75009a8b0 con 0x7ff758101d30 2024-05-10T04:38:12.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.790+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7440a1240 msgr2=0x7ff7440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:12.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.790+0000 7ff75d389700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7440a1240 0x7ff7440a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff758101290 tx=0x7ff740009260 comp rx=0 tx=0).stop 2024-05-10T04:38:12.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.790+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 msgr2=0x7ff758102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:12.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.790+0000 7ff75d389700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff758102c10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff750006070 tx=0x7ff7500078e0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.791+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 shutdown_connections 2024-05-10T04:38:12.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.791+0000 7ff75d389700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff758101d30 0x7ff758102c10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.791+0000 7ff75d389700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff758100270 0x7ff758104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.791+0000 7ff75d389700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7440a1240 0x7ff7440a3750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.791+0000 7ff75d389700 1 --2- 172.21.15.150:0/4166915015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff758101390 0x7ff7581026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.791+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 >> 172.21.15.150:0/4166915015 conn(0x7ff7580fb880 msgr2=0x7ff7581067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:12.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.798+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 shutdown_connections 2024-05-10T04:38:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.801+0000 7ff75d389700 1 -- 172.21.15.150:0/4166915015 wait complete. 2024-05-10T04:38:12.805 INFO:teuthology.orchestra.run.smithi150.stderr:set pool 4 allow_ec_overwrites to true 2024-05-10T04:38:12.820 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-10T04:38:12.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 -- 172.21.15.150:0/1079937399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830101be0 msgr2=0x7f2830101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:12.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 --2- 172.21.15.150:0/1079937399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830101be0 0x7f2830101ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f281c009990 tx=0x7f281c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 -- 172.21.15.150:0/1079937399 shutdown_connections 2024-05-10T04:38:12.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 --2- 172.21.15.150:0/1079937399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2830102580 0x7f2830108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 --2- 172.21.15.150:0/1079937399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830100ac0 0x7f2830100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 --2- 172.21.15.150:0/1079937399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830101be0 0x7f2830101ff0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 -- 172.21.15.150:0/1079937399 >> 172.21.15.150:0/1079937399 conn(0x7f28300fc0d0 msgr2=0x7f28300fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 -- 172.21.15.150:0/1079937399 shutdown_connections 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 -- 172.21.15.150:0/1079937399 wait complete. 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 Processor -- start 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.956+0000 7f2836e68700 1 -- start start 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2836e68700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830100ac0 0x7f2830068e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2836e68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2830101be0 0x7f283019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2836e68700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830102580 0x7f283019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2836e68700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283019f900 con 0x7f2830100ac0 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2836e68700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283019fa70 con 0x7f2830102580 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2836e68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283019fc30 con 0x7f2830101be0 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830100ac0 0x7f2830068e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:12.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830100ac0 0x7f2830068e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54136/0 (socket says 172.21.15.150:54136) 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 -- 172.21.15.150:0/1740858726 learned_addr learned my addr 172.21.15.150:0/1740858726 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2835405700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830102580 0x7f283019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f282ffff700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2830101be0 0x7f283019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 -- 172.21.15.150:0/1740858726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2830101be0 msgr2=0x7f283019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2830101be0 0x7f283019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 -- 172.21.15.150:0/1740858726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830102580 msgr2=0x7f283019f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830102580 0x7f283019f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 -- 172.21.15.150:0/1740858726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0095f0 con 0x7f2830100ac0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2834c04700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830100ac0 0x7f2830068e90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f282400d950 tx=0x7f282400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f282dffb700 1 -- 172.21.15.150:0/1740858726 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824004220 con 0x7f2830100ac0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f282dffb700 1 -- 172.21.15.150:0/1740858726 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2824004380 con 0x7f2830100ac0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283019ff10 con 0x7f2830100ac0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.957+0000 7f282dffb700 1 -- 172.21.15.150:0/1740858726 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824003be0 con 0x7f2830100ac0 2024-05-10T04:38:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.958+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28301a0390 con 0x7f2830100ac0 2024-05-10T04:38:12.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.959+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2830107cd0 con 0x7f2830100ac0 2024-05-10T04:38:12.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.960+0000 7f282dffb700 1 -- 172.21.15.150:0/1740858726 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2824003d40 con 0x7f2830100ac0 2024-05-10T04:38:12.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.960+0000 7f282dffb700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f28200a1240 0x7f28200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:12.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.960+0000 7f282ffff700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f28200a1240 0x7f28200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:12.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.960+0000 7f282dffb700 1 -- 172.21.15.150:0/1740858726 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f28240cc4d0 con 0x7f2830100ac0 2024-05-10T04:38:12.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.961+0000 7f282ffff700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f28200a1240 0x7f28200a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b5c0 tx=0x7f281c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:12.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:12.965+0000 7f282dffb700 1 -- 172.21.15.150:0/1740858726 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2824095a90 con 0x7f2830100ac0 2024-05-10T04:38:13.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:12 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2438479676' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T04:38:13.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:12 smithi161 ceph-mon[46912]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T04:38:13.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:12 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4166915015' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T04:38:13.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:12 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2438479676' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T04:38:13.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:12 smithi181 ceph-mon[47314]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T04:38:13.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:12 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4166915015' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T04:38:13.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:12 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2438479676' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-10T04:38:13.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:12 smithi150 ceph-mon[42404]: osdmap e84: 12 total, 12 up, 12 in 2024-05-10T04:38:13.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:12 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4166915015' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-10T04:38:13.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.211+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f28300694b0 con 0x7f2830100ac0 2024-05-10T04:38:13.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.217+0000 7f282dffb700 1 -- 172.21.15.150:0/1740858726 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f28240990b0 con 0x7f2830100ac0 2024-05-10T04:38:13.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f28200a1240 msgr2=0x7f28200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f28200a1240 0x7f28200a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f281c00b5c0 tx=0x7f281c01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830100ac0 msgr2=0x7f2830068e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830100ac0 0x7f2830068e90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f282400d950 tx=0x7f282400dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 shutdown_connections 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2830102580 0x7f283019f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2830100ac0 0x7f2830068e90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f28200a1240 0x7f28200a3750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 --2- 172.21.15.150:0/1740858726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2830101be0 0x7f283019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 >> 172.21.15.150:0/1740858726 conn(0x7f28300fc0d0 msgr2=0x7f2830104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 shutdown_connections 2024-05-10T04:38:13.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.220+0000 7f2836e68700 1 -- 172.21.15.150:0/1740858726 wait complete. 2024-05-10T04:38:13.223 INFO:teuthology.orchestra.run.smithi150.stderr:added data pool 4 to fsmap 2024-05-10T04:38:13.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:38:13.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 -- 172.21.15.150:0/2385050956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c41025a0 msgr2=0x7fd1c4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 --2- 172.21.15.150:0/2385050956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c41025a0 0x7fd1c4108e50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00b0a0 tx=0x7fd1bc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 -- 172.21.15.150:0/2385050956 shutdown_connections 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 --2- 172.21.15.150:0/2385050956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c41025a0 0x7fd1c4108e50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 --2- 172.21.15.150:0/2385050956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4100ae0 0x7fd1c4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 --2- 172.21.15.150:0/2385050956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c4101c00 0x7fd1c4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 -- 172.21.15.150:0/2385050956 >> 172.21.15.150:0/2385050956 conn(0x7fd1c40fc0f0 msgr2=0x7fd1c40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 -- 172.21.15.150:0/2385050956 shutdown_connections 2024-05-10T04:38:13.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.366+0000 7fd1cba17700 1 -- 172.21.15.150:0/2385050956 wait complete. 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 Processor -- start 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 -- start start 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c4100ae0 0x7fd1c4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4101c00 0x7fd1c41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c41025a0 0x7fd1c419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c419ea70 con 0x7fd1c4100ae0 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c419ebe0 con 0x7fd1c4101c00 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1cba17700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c419eda0 con 0x7fd1c41025a0 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1c97b3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c4100ae0 0x7fd1c4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1c97b3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c4100ae0 0x7fd1c4196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54152/0 (socket says 172.21.15.150:54152) 2024-05-10T04:38:13.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.367+0000 7fd1c97b3700 1 -- 172.21.15.150:0/3229598084 learned_addr learned my addr 172.21.15.150:0/3229598084 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c9fb4700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c41025a0 0x7fd1c419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c8fb2700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4101c00 0x7fd1c41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c97b3700 1 -- 172.21.15.150:0/3229598084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c41025a0 msgr2=0x7fd1c419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c97b3700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c41025a0 0x7fd1c419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c97b3700 1 -- 172.21.15.150:0/3229598084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4101c00 msgr2=0x7fd1c41974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c97b3700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4101c00 0x7fd1c41974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c97b3700 1 -- 172.21.15.150:0/3229598084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc009d00 con 0x7fd1c4100ae0 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c9fb4700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c41025a0 0x7fd1c419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c8fb2700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4101c00 0x7fd1c41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1c97b3700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c4100ae0 0x7fd1c4196f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400d950 tx=0x7fd1b400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1ba7fc700 1 -- 172.21.15.150:0/3229598084 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4004220 con 0x7fd1c4100ae0 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1ba7fc700 1 -- 172.21.15.150:0/3229598084 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1b400de30 con 0x7fd1c4100ae0 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c419f080 con 0x7fd1c4100ae0 2024-05-10T04:38:13.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1ba7fc700 1 -- 172.21.15.150:0/3229598084 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4003cb0 con 0x7fd1c4100ae0 2024-05-10T04:38:13.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.368+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c419f5e0 con 0x7fd1c4100ae0 2024-05-10T04:38:13.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.369+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c4107cf0 con 0x7fd1c4100ae0 2024-05-10T04:38:13.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.370+0000 7fd1ba7fc700 1 -- 172.21.15.150:0/3229598084 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd1b4004380 con 0x7fd1c4100ae0 2024-05-10T04:38:13.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.371+0000 7fd1ba7fc700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1b00a11a0 0x7fd1b00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.371+0000 7fd1ba7fc700 1 -- 172.21.15.150:0/3229598084 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd1b40cce50 con 0x7fd1c4100ae0 2024-05-10T04:38:13.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.371+0000 7fd1c8fb2700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1b00a11a0 0x7fd1b00a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.371+0000 7fd1c8fb2700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1b00a11a0 0x7fd1b00a36b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0005fd0 tx=0x7fd1c0005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:13.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.375+0000 7fd1ba7fc700 1 -- 172.21.15.150:0/3229598084 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1b4096410 con 0x7fd1c4100ae0 2024-05-10T04:38:13.585 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: 2024-05-10T04:38:13.286+0000 7fa2dad01700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-10T04:38:13.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.616+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1c40678b0 con 0x7fd1c4100ae0 2024-05-10T04:38:13.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.617+0000 7fd1ba7fc700 1 -- 172.21.15.150:0/3229598084 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10822 (secure 0 0 0) 0x7fd1b4017070 con 0x7fd1c4100ae0 2024-05-10T04:38:13.618 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:13.618 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:13.213661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-10T04:38:13.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1b00a11a0 msgr2=0x7fd1b00a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1b00a11a0 0x7fd1b00a36b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0005fd0 tx=0x7fd1c0005f00 comp rx=0 tx=0).stop 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c4100ae0 msgr2=0x7fd1c4196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c4100ae0 0x7fd1c4196f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400d950 tx=0x7fd1b400dd10 comp rx=0 tx=0).stop 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 shutdown_connections 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1b00a11a0 0x7fd1b00a36b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1c41025a0 0x7fd1c419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1c4100ae0 0x7fd1c4196f90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 --2- 172.21.15.150:0/3229598084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1c4101c00 0x7fd1c41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 >> 172.21.15.150:0/3229598084 conn(0x7fd1c40fc0f0 msgr2=0x7fd1c4104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 shutdown_connections 2024-05-10T04:38:13.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.619+0000 7fd1cba17700 1 -- 172.21.15.150:0/3229598084 wait complete. 2024-05-10T04:38:13.622 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 17 2024-05-10T04:38:13.633 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 -- 172.21.15.150:0/565192421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76200ffc20 msgr2=0x7f7620100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 --2- 172.21.15.150:0/565192421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76200ffc20 0x7f7620100030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f761c009990 tx=0x7f761c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 -- 172.21.15.150:0/565192421 shutdown_connections 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 --2- 172.21.15.150:0/565192421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7620100570 0x7f762010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 --2- 172.21.15.150:0/565192421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76201031e0 0x7f76201035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 --2- 172.21.15.150:0/565192421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76200ffc20 0x7f7620100030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 -- 172.21.15.150:0/565192421 >> 172.21.15.150:0/565192421 conn(0x7f76200fb8f0 msgr2=0x7f76200fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 -- 172.21.15.150:0/565192421 shutdown_connections 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.765+0000 7f76284f9700 1 -- 172.21.15.150:0/565192421 wait complete. 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 Processor -- start 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 -- start start 2024-05-10T04:38:13.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76200ffc20 0x7f7620197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7620100570 0x7f76201984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76201031e0 0x7f762019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762019fa90 con 0x7f76201031e0 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762019fc00 con 0x7f7620100570 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f76284f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762019fdc0 con 0x7f76200ffc20 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76201031e0 0x7f762019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626295700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76200ffc20 0x7f7620197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76201031e0 0x7f762019f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54158/0 (socket says 172.21.15.150:54158) 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 -- 172.21.15.150:0/609293457 learned_addr learned my addr 172.21.15.150:0/609293457 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7625a94700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7620100570 0x7f76201984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 -- 172.21.15.150:0/609293457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76200ffc20 msgr2=0x7f7620197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76200ffc20 0x7f7620197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 -- 172.21.15.150:0/609293457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7620100570 msgr2=0x7f76201984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7620100570 0x7f76201984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.766+0000 7f7626a96700 1 -- 172.21.15.150:0/609293457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c0095f0 con 0x7f76201031e0 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.767+0000 7f7626a96700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76201031e0 0x7f762019f480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7618007ee0 tx=0x7f761800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.767+0000 7f76177fe700 1 -- 172.21.15.150:0/609293457 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800dd20 con 0x7f76201031e0 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.767+0000 7f76177fe700 1 -- 172.21.15.150:0/609293457 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761800f040 con 0x7f76201031e0 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.767+0000 7f76177fe700 1 -- 172.21.15.150:0/609293457 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180048a0 con 0x7f76201031e0 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.767+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76201a00a0 con 0x7f76201031e0 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.767+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76201a0560 con 0x7f76201031e0 2024-05-10T04:38:13.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.768+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7620109030 con 0x7f76201031e0 2024-05-10T04:38:13.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.769+0000 7f76177fe700 1 -- 172.21.15.150:0/609293457 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7618003a00 con 0x7f76201031e0 2024-05-10T04:38:13.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.770+0000 7f76177fe700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f760c0a1120 0x7f760c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:13.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.770+0000 7f7626295700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f760c0a1120 0x7f760c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:13.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.770+0000 7f76177fe700 1 -- 172.21.15.150:0/609293457 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f76180ccc00 con 0x7f76201031e0 2024-05-10T04:38:13.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.770+0000 7f7626295700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f760c0a1120 0x7f760c0a3630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7620068120 tx=0x7f7610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:13.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.774+0000 7f76177fe700 1 -- 172.21.15.150:0/609293457 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7618096140 con 0x7f76201031e0 2024-05-10T04:38:13.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4166915015' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T04:38:13.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T04:38:13.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1740858726' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T04:38:13.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T04:38:13.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1740858726' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T04:38:13.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:13.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: pgmap v227: 129 pgs: 17 creating+peering, 84 active+clean, 28 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 10 op/s 2024-05-10T04:38:13.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:13 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3229598084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:13.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.977+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76200678b0 con 0x7f76201031e0 2024-05-10T04:38:13.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.978+0000 7f76177fe700 1 -- 172.21.15.150:0/609293457 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f7618099760 con 0x7f76201031e0 2024-05-10T04:38:13.980 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:13.980 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f760c0a1120 msgr2=0x7f760c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f760c0a1120 0x7f760c0a3630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7620068120 tx=0x7f7610009260 comp rx=0 tx=0).stop 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76201031e0 msgr2=0x7f762019f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76201031e0 0x7f762019f480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7618007ee0 tx=0x7f761800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 shutdown_connections 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76201031e0 0x7f762019f480 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f760c0a1120 0x7f760c0a3630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76200ffc20 0x7f7620197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 --2- 172.21.15.150:0/609293457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7620100570 0x7f76201984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:13.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 >> 172.21.15.150:0/609293457 conn(0x7f76200fb8f0 msgr2=0x7f7620106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:13.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 shutdown_connections 2024-05-10T04:38:13.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:13.981+0000 7f76284f9700 1 -- 172.21.15.150:0/609293457 wait complete. 2024-05-10T04:38:13.996 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:14.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4166915015' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T04:38:14.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T04:38:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1740858726' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T04:38:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T04:38:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1740858726' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T04:38:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: pgmap v227: 129 pgs: 17 creating+peering, 84 active+clean, 28 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 10 op/s 2024-05-10T04:38:14.086 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:13 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3229598084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:14.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 -- 172.21.15.150:0/446443152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81013a0 msgr2=0x7f45f81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/446443152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81013a0 0x7f45f81017b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f45e000aaa0 tx=0x7f45e000adb0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 -- 172.21.15.150:0/446443152 shutdown_connections 2024-05-10T04:38:14.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/446443152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45f8101d40 0x7f45f81085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/446443152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f8100280 0x7f45f8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/446443152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f81013a0 0x7f45f81017b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 -- 172.21.15.150:0/446443152 >> 172.21.15.150:0/446443152 conn(0x7f45f80fb8b0 msgr2=0x7f45f80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:14.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 -- 172.21.15.150:0/446443152 shutdown_connections 2024-05-10T04:38:14.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.127+0000 7f45fd0f1700 1 -- 172.21.15.150:0/446443152 wait complete. 2024-05-10T04:38:14.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.128+0000 7f45fd0f1700 1 Processor -- start 2024-05-10T04:38:14.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.128+0000 7f45fd0f1700 1 -- start start 2024-05-10T04:38:14.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.128+0000 7f45fd0f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f8100280 0x7f45f8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.128+0000 7f45fd0f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45f81013a0 0x7f45f8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.128+0000 7f45fd0f1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f8101d40 0x7f45f819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45fd0f1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f819faa0 con 0x7f45f8101d40 2024-05-10T04:38:14.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45fd0f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f819fc10 con 0x7f45f81013a0 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45fd0f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f819fdd0 con 0x7f45f8100280 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f8101d40 0x7f45f819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f8101d40 0x7f45f819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54188/0 (socket says 172.21.15.150:54188) 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 -- 172.21.15.150:0/2046811489 learned_addr learned my addr 172.21.15.150:0/2046811489 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f6d9d700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f8100280 0x7f45f8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45ee59c700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45f81013a0 0x7f45f8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 -- 172.21.15.150:0/2046811489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f8100280 msgr2=0x7f45f8197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f8100280 0x7f45f8197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 -- 172.21.15.150:0/2046811489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45f81013a0 msgr2=0x7f45f8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45f81013a0 0x7f45f8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 -- 172.21.15.150:0/2046811489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e000a700 con 0x7f45f8101d40 2024-05-10T04:38:14.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f759e700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f8101d40 0x7f45f819f490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f45f0007ee0 tx=0x7f45f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:14.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f4d99700 1 -- 172.21.15.150:0/2046811489 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0017070 con 0x7f45f8101d40 2024-05-10T04:38:14.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f4d99700 1 -- 172.21.15.150:0/2046811489 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45f000f040 con 0x7f45f8101d40 2024-05-10T04:38:14.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f81a00b0 con 0x7f45f8101d40 2024-05-10T04:38:14.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45f4d99700 1 -- 172.21.15.150:0/2046811489 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0005580 con 0x7f45f8101d40 2024-05-10T04:38:14.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.129+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f81a05c0 con 0x7f45f8101d40 2024-05-10T04:38:14.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.130+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f8107490 con 0x7f45f8101d40 2024-05-10T04:38:14.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.132+0000 7f45f4d99700 1 -- 172.21.15.150:0/2046811489 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f45f000db20 con 0x7f45f8101d40 2024-05-10T04:38:14.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.132+0000 7f45f4d99700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f45e40a11a0 0x7f45e40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.132+0000 7f45f6d9d700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f45e40a11a0 0x7f45e40a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.132+0000 7f45f4d99700 1 -- 172.21.15.150:0/2046811489 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45f00cc7b0 con 0x7f45f8101d40 2024-05-10T04:38:14.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.133+0000 7f45f6d9d700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f45e40a11a0 0x7f45e40a36b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45f81012a0 tx=0x7f45e800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:14.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.136+0000 7f45f4d99700 1 -- 172.21.15.150:0/2046811489 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f45f0095d50 con 0x7f45f8101d40 2024-05-10T04:38:14.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4166915015' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-10T04:38:14.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: osdmap e85: 12 total, 12 up, 12 in 2024-05-10T04:38:14.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1740858726' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-10T04:38:14.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: osdmap e86: 12 total, 12 up, 12 in 2024-05-10T04:38:14.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1740858726' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-10T04:38:14.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:14.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: pgmap v227: 129 pgs: 17 creating+peering, 84 active+clean, 28 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 10 op/s 2024-05-10T04:38:14.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:13 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3229598084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:14.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.336+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45f8068730 con 0x7f45f8101d40 2024-05-10T04:38:14.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.337+0000 7f45f4d99700 1 -- 172.21.15.150:0/2046811489 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f45f0099370 con 0x7f45f8101d40 2024-05-10T04:38:14.339 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:14.339 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:14.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f45e40a11a0 msgr2=0x7f45e40a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f45e40a11a0 0x7f45e40a36b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45f81012a0 tx=0x7f45e800a380 comp rx=0 tx=0).stop 2024-05-10T04:38:14.341 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f8101d40 msgr2=0x7f45f819f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f8101d40 0x7f45f819f490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f45f0007ee0 tx=0x7f45f000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 shutdown_connections 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f8101d40 0x7f45f819f490 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f45e40a11a0 0x7f45e40a36b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45f8100280 0x7f45f8197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 --2- 172.21.15.150:0/2046811489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45f81013a0 0x7f45f8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 >> 172.21.15.150:0/2046811489 conn(0x7f45f80fb8b0 msgr2=0x7f45f81045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 shutdown_connections 2024-05-10T04:38:14.342 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.340+0000 7f45fd0f1700 1 -- 172.21.15.150:0/2046811489 wait complete. 2024-05-10T04:38:14.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 -- 172.21.15.150:0/902912617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 msgr2=0x7f7470101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 --2- 172.21.15.150:0/902912617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470101480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7458009990 tx=0x7f7458009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 -- 172.21.15.150:0/902912617 shutdown_connections 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 --2- 172.21.15.150:0/902912617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74701025b0 0x7f7470109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 --2- 172.21.15.150:0/902912617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470101480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 --2- 172.21.15.150:0/902912617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7470101a50 0x7f7470101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 -- 172.21.15.150:0/902912617 >> 172.21.15.150:0/902912617 conn(0x7f74700fb8b0 msgr2=0x7f74700fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:14.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 -- 172.21.15.150:0/902912617 shutdown_connections 2024-05-10T04:38:14.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.501+0000 7f747513c700 1 -- 172.21.15.150:0/902912617 wait complete. 2024-05-10T04:38:14.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 Processor -- start 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 -- start start 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470101a50 0x7f7470103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74701025b0 0x7f7470103660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7470103d40 con 0x7f7470101070 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7470103eb0 con 0x7f74701025b0 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f747513c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74701a2070 con 0x7f7470101a50 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f746ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f746ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54204/0 (socket says 172.21.15.150:54204) 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.502+0000 7f746ed9d700 1 -- 172.21.15.150:0/3324660755 learned_addr learned my addr 172.21.15.150:0/3324660755 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:14.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746e59c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470101a50 0x7f7470103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746f59e700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74701025b0 0x7f7470103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746ed9d700 1 -- 172.21.15.150:0/3324660755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470101a50 msgr2=0x7f7470103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746ed9d700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470101a50 0x7f7470103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746ed9d700 1 -- 172.21.15.150:0/3324660755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74701025b0 msgr2=0x7f7470103660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746ed9d700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74701025b0 0x7f7470103660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746ed9d700 1 -- 172.21.15.150:0/3324660755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580095f0 con 0x7f7470101070 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746f59e700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74701025b0 0x7f7470103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f746ed9d700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470104a80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f74580098e0 tx=0x7f7458009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f7467fff700 1 -- 172.21.15.150:0/3324660755 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745801d070 con 0x7f7470101070 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74701a2270 con 0x7f7470101070 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f7467fff700 1 -- 172.21.15.150:0/3324660755 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7458021410 con 0x7f7470101070 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f7467fff700 1 -- 172.21.15.150:0/3324660755 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458017650 con 0x7f7470101070 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.503+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74701a26f0 con 0x7f7470101070 2024-05-10T04:38:14.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.504+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7470068730 con 0x7f7470101070 2024-05-10T04:38:14.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.505+0000 7f7467fff700 1 -- 172.21.15.150:0/3324660755 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7458021580 con 0x7f7470101070 2024-05-10T04:38:14.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.506+0000 7f7467fff700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f745c0a1120 0x7f745c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.506+0000 7f746e59c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f745c0a1120 0x7f745c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.506+0000 7f7467fff700 1 -- 172.21.15.150:0/3324660755 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f74580ce570 con 0x7f7470101070 2024-05-10T04:38:14.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.506+0000 7f746e59c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f745c0a1120 0x7f745c0a3630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7470104420 tx=0x7f7460009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:14.511 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.510+0000 7f7467fff700 1 -- 172.21.15.150:0/3324660755 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7458097ba0 con 0x7f7470101070 2024-05-10T04:38:14.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.750+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f747010c4f0 con 0x7f7470101070 2024-05-10T04:38:14.752 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.751+0000 7f7467fff700 1 -- 172.21.15.150:0/3324660755 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10822 (secure 0 0 0) 0x7f745809b1c0 con 0x7f7470101070 2024-05-10T04:38:14.752 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:14.752 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":14664,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:13.213661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-10T04:38:14.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f745c0a1120 msgr2=0x7f745c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f745c0a1120 0x7f745c0a3630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7470104420 tx=0x7f7460009260 comp rx=0 tx=0).stop 2024-05-10T04:38:14.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 msgr2=0x7f7470104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470104a80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f74580098e0 tx=0x7f7458009580 comp rx=0 tx=0).stop 2024-05-10T04:38:14.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 shutdown_connections 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74701025b0 0x7f7470103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7470101070 0x7f7470104a80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f745c0a1120 0x7f745c0a3630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 --2- 172.21.15.150:0/3324660755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7470101a50 0x7f7470103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 >> 172.21.15.150:0/3324660755 conn(0x7f74700fb8b0 msgr2=0x7f74700fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 shutdown_connections 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.753+0000 7f747513c700 1 -- 172.21.15.150:0/3324660755 wait complete. 2024-05-10T04:38:14.755 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 17 2024-05-10T04:38:14.767 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:14.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3778390375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd274100ae0 msgr2=0x7fd274100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3778390375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd274100ae0 0x7fd274100ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd264009990 tx=0x7fd264009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3778390375 shutdown_connections 2024-05-10T04:38:14.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3778390375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2741025a0 0x7fd274108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3778390375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd274100ae0 0x7fd274100ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3778390375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd274101c00 0x7fd274102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3778390375 >> 172.21.15.150:0/3778390375 conn(0x7fd2740fc0f0 msgr2=0x7fd2740fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3778390375 shutdown_connections 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.898+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3778390375 wait complete. 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 Processor -- start 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 -- start start 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd274100ae0 0x7fd274196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd274101c00 0x7fd2741974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2741025a0 0x7fd27419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27419ea00 con 0x7fd2741025a0 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27419eb40 con 0x7fd274100ae0 2024-05-10T04:38:14.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27c3d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27419ecd0 con 0x7fd274101c00 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2741025a0 0x7fd27419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2741025a0 0x7fd27419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54230/0 (socket says 172.21.15.150:54230) 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 -- 172.21.15.150:0/3069192860 learned_addr learned my addr 172.21.15.150:0/3069192860 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd279974700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd274101c00 0x7fd2741974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a175700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd274100ae0 0x7fd274196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 -- 172.21.15.150:0/3069192860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd274101c00 msgr2=0x7fd2741974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd274101c00 0x7fd2741974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 -- 172.21.15.150:0/3069192860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd274100ae0 msgr2=0x7fd274196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd274100ae0 0x7fd274196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 -- 172.21.15.150:0/3069192860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640095f0 con 0x7fd2741025a0 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.899+0000 7fd27a976700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2741025a0 0x7fd27419e430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007da0 tx=0x7fd26c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:14.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.900+0000 7fd26b7fe700 1 -- 172.21.15.150:0/3069192860 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00ac50 con 0x7fd2741025a0 2024-05-10T04:38:14.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.900+0000 7fd26b7fe700 1 -- 172.21.15.150:0/3069192860 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd26c00adb0 con 0x7fd2741025a0 2024-05-10T04:38:14.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.900+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27419efb0 con 0x7fd2741025a0 2024-05-10T04:38:14.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.900+0000 7fd26b7fe700 1 -- 172.21.15.150:0/3069192860 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0115c0 con 0x7fd2741025a0 2024-05-10T04:38:14.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.900+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27419f450 con 0x7fd2741025a0 2024-05-10T04:38:14.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.901+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd274107cf0 con 0x7fd2741025a0 2024-05-10T04:38:14.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.902+0000 7fd26b7fe700 1 -- 172.21.15.150:0/3069192860 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd26c004ad0 con 0x7fd2741025a0 2024-05-10T04:38:14.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.902+0000 7fd26b7fe700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd2600a11a0 0x7fd2600a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:14.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.903+0000 7fd27a175700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd2600a11a0 0x7fd2600a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:14.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.903+0000 7fd26b7fe700 1 -- 172.21.15.150:0/3069192860 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd26c0ccfe0 con 0x7fd2741025a0 2024-05-10T04:38:14.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.903+0000 7fd27a175700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd2600a11a0 0x7fd2600a36b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2640038c0 tx=0x7fd26401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:14.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:14.906+0000 7fd26b7fe700 1 -- 172.21.15.150:0/3069192860 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd26c096580 con 0x7fd2741025a0 2024-05-10T04:38:15.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:14 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/609293457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:14 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2046811489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:14 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3324660755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.104+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd27410c390 con 0x7fd2741025a0 2024-05-10T04:38:15.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.105+0000 7fd26b7fe700 1 -- 172.21.15.150:0/3069192860 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7fd26c016070 con 0x7fd2741025a0 2024-05-10T04:38:15.107 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:15.107 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:15.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.107+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd2600a11a0 msgr2=0x7fd2600a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.107+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd2600a11a0 0x7fd2600a36b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2640038c0 tx=0x7fd26401a040 comp rx=0 tx=0).stop 2024-05-10T04:38:15.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.107+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2741025a0 msgr2=0x7fd27419e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.107+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2741025a0 0x7fd27419e430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007da0 tx=0x7fd26c00a340 comp rx=0 tx=0).stop 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.107+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 shutdown_connections 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.107+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2741025a0 0x7fd27419e430 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.108+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd274100ae0 0x7fd274196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.108+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd2600a11a0 0x7fd2600a36b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.108+0000 7fd27c3d9700 1 --2- 172.21.15.150:0/3069192860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd274101c00 0x7fd2741974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.108+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 >> 172.21.15.150:0/3069192860 conn(0x7fd2740fc0f0 msgr2=0x7fd274104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.108+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 shutdown_connections 2024-05-10T04:38:15.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.108+0000 7fd27c3d9700 1 -- 172.21.15.150:0/3069192860 wait complete. 2024-05-10T04:38:15.121 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:14 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/609293457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.121 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:14 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2046811489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.121 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:14 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3324660755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-10T04:38:08.750329+0000', 'modified': '2024-05-10T04:38:13.213661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/3001288873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 3001288873}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 3001288873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-10T04:38:15.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-10T04:38:15.123 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T04:38:15.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:14 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/609293457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:14 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2046811489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:14 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3324660755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 -- 172.21.15.150:0/1088754650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460102580 msgr2=0x7f8460108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 --2- 172.21.15.150:0/1088754650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460102580 0x7f8460108e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f845800b0a0 tx=0x7f845800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 -- 172.21.15.150:0/1088754650 shutdown_connections 2024-05-10T04:38:15.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 --2- 172.21.15.150:0/1088754650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460102580 0x7f8460108e30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 --2- 172.21.15.150:0/1088754650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8460100ac0 0x7f8460100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 --2- 172.21.15.150:0/1088754650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460101be0 0x7f8460101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 -- 172.21.15.150:0/1088754650 >> 172.21.15.150:0/1088754650 conn(0x7f84600fc0d0 msgr2=0x7f84600fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:15.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 -- 172.21.15.150:0/1088754650 shutdown_connections 2024-05-10T04:38:15.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.256+0000 7f8466af6700 1 -- 172.21.15.150:0/1088754650 wait complete. 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 Processor -- start 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 -- start start 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460100ac0 0x7f8460196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460101be0 0x7f8460197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8460102580 0x7f846019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846019e9d0 con 0x7f8460100ac0 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846019eb40 con 0x7f8460101be0 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8466af6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846019ed00 con 0x7f8460102580 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460100ac0 0x7f8460196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460100ac0 0x7f8460196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54260/0 (socket says 172.21.15.150:54260) 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 -- 172.21.15.150:0/1177975127 learned_addr learned my addr 172.21.15.150:0/1177975127 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f845ffff700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460101be0 0x7f8460197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8465093700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8460102580 0x7f846019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 -- 172.21.15.150:0/1177975127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8460102580 msgr2=0x7f846019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8460102580 0x7f846019e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 -- 172.21.15.150:0/1177975127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460101be0 msgr2=0x7f8460197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460101be0 0x7f8460197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 -- 172.21.15.150:0/1177975127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009d00 con 0x7f8460100ac0 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f845ffff700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460101be0 0x7f8460197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.257+0000 7f8464892700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460100ac0 0x7f8460196ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f845400e940 tx=0x7f845400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.258+0000 7f845dffb700 1 -- 172.21.15.150:0/1177975127 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400cbd0 con 0x7f8460100ac0 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.258+0000 7f845dffb700 1 -- 172.21.15.150:0/1177975127 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84540044e0 con 0x7f8460100ac0 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.258+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846019efe0 con 0x7f8460100ac0 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.258+0000 7f845dffb700 1 -- 172.21.15.150:0/1177975127 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454003dc0 con 0x7f8460100ac0 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.258+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846019f410 con 0x7f8460100ac0 2024-05-10T04:38:15.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.260+0000 7f845dffb700 1 -- 172.21.15.150:0/1177975127 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8454003f20 con 0x7f8460100ac0 2024-05-10T04:38:15.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.261+0000 7f845dffb700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84500a11c0 0x7f84500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.261+0000 7f845ffff700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84500a11c0 0x7f84500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.261+0000 7f845dffb700 1 -- 172.21.15.150:0/1177975127 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8454014070 con 0x7f8460100ac0 2024-05-10T04:38:15.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.261+0000 7f845ffff700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84500a11c0 0x7f84500a36d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8460197f90 tx=0x7f844c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:15.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 Processor -- start 2024-05-10T04:38:15.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 -- start start 2024-05-10T04:38:15.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fdacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fdbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37eff100 con 0x563e37fda860 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37efcac0 con 0x563e37fdc080 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37ef83b0 con 0x563e37fdb730 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fdacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fdacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54274/0 (socket says 172.21.15.150:54274) 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.262+0000 7f8464892700 1 -- 172.21.15.150:0/6520557 learned_addr learned my addr 172.21.15.150:0/6520557 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f845ffff700 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fdbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f8465093700 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f8464892700 1 -- 172.21.15.150:0/6520557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 msgr2=0x563e37fdbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f8464892700 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fdbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f8464892700 1 -- 172.21.15.150:0/6520557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 msgr2=0x563e37fe2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f8464892700 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f8464892700 1 -- 172.21.15.150:0/6520557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e37d79360 con 0x563e37fda860 2024-05-10T04:38:15.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f8464892700 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fdacb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f845409a030 tx=0x7f8454003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dc873e80f6c9dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:15.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f844affd700 1 -- 172.21.15.150:0/6520557 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845409a7a0 con 0x563e37fda860 2024-05-10T04:38:15.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f844affd700 1 -- 172.21.15.150:0/6520557 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f845409a0a0 con 0x563e37fda860 2024-05-10T04:38:15.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.263+0000 7f844affd700 1 -- 172.21.15.150:0/6520557 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84540967b0 con 0x563e37fda860 2024-05-10T04:38:15.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 -- 172.21.15.150:0/6520557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 msgr2=0x563e37fdacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fdacb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f845409a030 tx=0x7f8454003680 comp rx=0 tx=0).stop 2024-05-10T04:38:15.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 -- 172.21.15.150:0/6520557 shutdown_connections 2024-05-10T04:38:15.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fdacb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/6520557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fdbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 -- 172.21.15.150:0/6520557 >> 172.21.15.150:0/6520557 conn(0x563e37f3aa10 msgr2=0x563e37fd2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 -- 172.21.15.150:0/6520557 shutdown_connections 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 -- 172.21.15.150:0/6520557 wait complete. 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 -- start start 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.264+0000 7f8476e3ab80 1 Processor -- start 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdc080 0x563e37febd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37e94510 con 0x563e37fda860 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37e941b0 con 0x563e37fdb730 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37d7b650 con 0x563e37fdc080 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8476e3ab80 10 client.0 _ll_register_callbacks cb 0x563e37e9dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:38:15.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54278/0 (socket says 172.21.15.150:54278) 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 -- 172.21.15.150:0/132425522 learned_addr learned my addr 172.21.15.150:0/132425522 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8465093700 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdc080 0x563e37febd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f845ffff700 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 -- 172.21.15.150:0/132425522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdc080 msgr2=0x563e37febd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdc080 0x563e37febd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 -- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 msgr2=0x563e37fe4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 -- 172.21.15.150:0/132425522 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846019f410 con 0x563e37fda860 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.265+0000 7f8464892700 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe48b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84540ca7a0 tx=0x7f845400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f734be3776299c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f843ffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f843ffff700 1 -- 172.21.15.150:0/132425522 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84540a2520 con 0x563e37fda860 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f843ffff700 1 -- 172.21.15.150:0/132425522 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f84540b81f0 con 0x563e37fda860 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e37e51220 con 0x563e37fda860 2024-05-10T04:38:15.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f843ffff700 1 -- 172.21.15.150:0/132425522 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84540b83b0 con 0x563e37fda860 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 10 client.14793 fetch_fsmap learned FSMap version 17 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x563e37d25b10 con 0x563e37fda860 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f843ffff700 1 -- 172.21.15.150:0/132425522 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e37e51220 con 0x563e37fda860 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f843ffff700 1 -- 172.21.15.150:0/132425522 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f845407c070 con 0x563e37fda860 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 10 client.14793 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 10 client.14793 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 20 client.14793 populate_metadata read hostname 'smithi150' 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e37ff0090 0x563e37ff47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 4 client.14793 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e37fa7730 con 0x563e37ff0090 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.266+0000 7f845ffff700 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e37ff0090 0x563e37ff47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.267+0000 7f843ffff700 10 client.14793 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.267+0000 7f845ffff700 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e37ff0090 0x563e37ff47b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c5918d7097e1bf server_cookie=2adf89cdc822ad2d in_seq=0 out_seq=0 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f843ffff700 1 -- 172.21.15.150:0/132425522 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e37fa7730 con 0x563e37ff0090 2024-05-10T04:38:15.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f843ffff700 10 client.14793 handle_command_reply: tid=0 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 client.14793 shutdown 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 msgr2=0x563e37fe48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe48b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84540ca7a0 tx=0x7f845400cef0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 shutdown_connections 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e37ff0090 0x563e37ff47b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdc080 0x563e37febd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe48b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f843ffff700 0 client.14793 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/132425522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 >> 172.21.15.150:0/132425522 conn(0x563e37f3aa10 msgr2=0x563e37fd2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 shutdown_connections 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 1 -- 172.21.15.150:0/132425522 wait complete. 2024-05-10T04:38:15.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.268+0000 7f8476e3ab80 20 client.14793 trim_cache size 0 max 16384 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 Processor -- start 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 -- start start 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fe3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37e51220 con 0x563e37fda860 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37eb62b0 con 0x563e37fdc080 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37ef5010 con 0x563e37fdb730 2024-05-10T04:38:15.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54288/0 (socket says 172.21.15.150:54288) 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 -- 172.21.15.150:0/1554561926 learned_addr learned my addr 172.21.15.150:0/1554561926 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8465093700 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f845ffff700 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fe3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 -- 172.21.15.150:0/1554561926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 msgr2=0x563e37fe3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fe3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 -- 172.21.15.150:0/1554561926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 msgr2=0x563e37fe9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 -- 172.21.15.150:0/1554561926 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e37ebe510 con 0x563e37fda860 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8465093700 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.287+0000 7f8464892700 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe3e90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f84540037d0 tx=0x7f8454003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922665b0e525599f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f844b7fe700 1 -- 172.21.15.150:0/1554561926 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454098520 con 0x563e37fda860 2024-05-10T04:38:15.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f844b7fe700 1 -- 172.21.15.150:0/1554561926 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84540a2200 con 0x563e37fda860 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f844b7fe700 1 -- 172.21.15.150:0/1554561926 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845409a580 con 0x563e37fda860 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 -- 172.21.15.150:0/1554561926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 msgr2=0x563e37fe3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe3e90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f84540037d0 tx=0x7f8454003680 comp rx=0 tx=0).stop 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 -- 172.21.15.150:0/1554561926 shutdown_connections 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdc080 0x563e37fe9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fda860 0x563e37fe3e90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/1554561926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fdb730 0x563e37fe3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 -- 172.21.15.150:0/1554561926 >> 172.21.15.150:0/1554561926 conn(0x563e37f3aa10 msgr2=0x563e37fe2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.288+0000 7f8476e3ab80 1 -- 172.21.15.150:0/1554561926 shutdown_connections 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 -- 172.21.15.150:0/1554561926 wait complete. 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 -- start start 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 Processor -- start 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fda860 0x563e37fe7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fdc080 0x563e37fe8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fe9360 con 0x563e37fdc080 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fe94d0 con 0x563e37fdb730 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e37fe9690 con 0x563e37fda860 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8476e3ab80 10 client.0 _ll_register_callbacks cb 0x563e37e9dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8465093700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fdc080 0x563e37fe8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8465093700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fdc080 0x563e37fe8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54290/0 (socket says 172.21.15.150:54290) 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.289+0000 7f8465093700 1 -- 172.21.15.150:0/835128245 learned_addr learned my addr 172.21.15.150:0/835128245 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8464892700 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fda860 0x563e37fe7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f845ffff700 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8465093700 1 -- 172.21.15.150:0/835128245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fda860 msgr2=0x563e37fe7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8465093700 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fda860 0x563e37fe7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8465093700 1 -- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 msgr2=0x563e37fe8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8465093700 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8465093700 1 -- 172.21.15.150:0/835128245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e37d25b10 con 0x563e37fdc080 2024-05-10T04:38:15.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8465093700 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fdc080 0x563e37fe8e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f845800b0a0 tx=0x7f84580077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f02ca91c4b1483 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f843ffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f843ffff700 1 -- 172.21.15.150:0/835128245 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800e040 con 0x563e37fdc080 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f843ffff700 1 -- 172.21.15.150:0/835128245 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8458004a50 con 0x563e37fdc080 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f845409a0a0 con 0x563e37fdc080 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f843ffff700 1 -- 172.21.15.150:0/835128245 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8458004c70 con 0x563e37fdc080 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8476e3ab80 10 client.14799 fetch_fsmap learned FSMap version 17 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x563e37d7dc80 con 0x563e37fdc080 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f843ffff700 1 -- 172.21.15.150:0/835128245 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8458031080 con 0x563e37fdc080 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f843ffff700 1 -- 172.21.15.150:0/835128245 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8458024030 con 0x563e37fdc080 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.290+0000 7f8476e3ab80 10 client.14799 fetch_fsmap finished waiting for FSMap version 17 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f8476e3ab80 10 client.14799 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f8476e3ab80 20 client.14799 populate_metadata read hostname 'smithi150' 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e38048510 0x563e38046a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f8476e3ab80 4 client.14799 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:38:15.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563e37ef0e70 con 0x563e38048510 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f8464892700 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e38048510 0x563e38046a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f843ffff700 10 client.14799 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.291+0000 7f8464892700 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e38048510 0x563e38046a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d663d9331399e5 server_cookie=9b153adcb1448581 in_seq=0 out_seq=0 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f843ffff700 1 -- 172.21.15.150:0/835128245 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563e37ef0e70 con 0x563e38048510 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f843ffff700 10 client.14799 handle_command_reply: tid=0 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 client.14799 shutdown 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fdc080 msgr2=0x563e37fe8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fdc080 0x563e37fe8e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f845800b0a0 tx=0x7f84580077f0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 shutdown_connections 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563e38048510 0x563e38046a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e37fdc080 0x563e37fe8e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563e37fda860 0x563e37fe7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f843ffff700 0 client.14799 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 --2- 172.21.15.150:0/835128245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563e37fdb730 0x563e37fe8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 >> 172.21.15.150:0/835128245 conn(0x563e37f3aa10 msgr2=0x563e37fe2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 shutdown_connections 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 1 -- 172.21.15.150:0/835128245 wait complete. 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8476e3ab80 20 client.14799 trim_cache size 0 max 16384 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24548, 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:38:15.300 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:38:15.301 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 6.5397699950000003, 2024-05-10T04:38:15.301 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 17, 2024-05-10T04:38:15.301 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 86, 2024-05-10T04:38:15.301 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 85, 2024-05-10T04:38:15.301 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 12.514395111000001 2024-05-10T04:38:15.301 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84500a11c0 msgr2=0x7f84500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8466af6700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84500a11c0 0x7f84500a36d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8460197f90 tx=0x7f844c009260 comp rx=0 tx=0).stop 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460100ac0 msgr2=0x7f8460196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.292+0000 7f8466af6700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460100ac0 0x7f8460196ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f845400e940 tx=0x7f845400ed00 comp rx=0 tx=0).stop 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 shutdown_connections 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84500a11c0 0x7f84500a36d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8460102580 0x7f846019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8460100ac0 0x7f8460196ef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 --2- 172.21.15.150:0/1177975127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8460101be0 0x7f8460197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 >> 172.21.15.150:0/1177975127 conn(0x7f84600fc0d0 msgr2=0x7f8460104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 shutdown_connections 2024-05-10T04:38:15.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:15.293+0000 7f8466af6700 1 -- 172.21.15.150:0/1177975127 wait complete. 2024-05-10T04:38:15.307 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T04:38:15.307 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:15.307 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T04:38:15.525 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:15.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:15 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3069192860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:15.912 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:15 smithi150 ceph-mon[42404]: pgmap v228: 129 pgs: 3 creating+peering, 118 active+clean, 8 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-05-10T04:38:16.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:15 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3069192860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:16.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:15 smithi161 ceph-mon[46912]: pgmap v228: 129 pgs: 3 creating+peering, 118 active+clean, 8 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-05-10T04:38:16.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:15 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3069192860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:16.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:15 smithi181 ceph-mon[47314]: pgmap v228: 129 pgs: 3 creating+peering, 118 active+clean, 8 unknown; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 -- 172.21.15.150:0/486572188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 msgr2=0x7fa13810d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/486572188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa13810d1b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b0a0 tx=0x7fa13000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 -- 172.21.15.150:0/486572188 shutdown_connections 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/486572188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa13810d1b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/486572188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1381005d0 0x7fa1381009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/486572188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa138104f00 0x7fa138105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 -- 172.21.15.150:0/486572188 >> 172.21.15.150:0/486572188 conn(0x7fa1380fbee0 msgr2=0x7fa1380fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 -- 172.21.15.150:0/486572188 shutdown_connections 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 -- 172.21.15.150:0/486572188 wait complete. 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.475+0000 7fa13e2c2700 1 Processor -- start 2024-05-10T04:38:16.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13e2c2700 1 -- start start 2024-05-10T04:38:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13e2c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1381005d0 0x7fa1381995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13e2c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138104f00 0x7fa138199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13e2c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa1381a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13e2c2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13819a2a0 con 0x7fa138105850 2024-05-10T04:38:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13e2c2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1381a49c0 con 0x7fa138104f00 2024-05-10T04:38:16.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13e2c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1381a4b80 con 0x7fa1381005d0 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa1381a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa1381a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:54318/0 (socket says 172.21.15.150:54318) 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 -- 172.21.15.150:0/2805489327 learned_addr learned my addr 172.21.15.150:0/2805489327 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa137fff700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1381005d0 0x7fa1381995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa1377fe700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138104f00 0x7fa138199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 -- 172.21.15.150:0/2805489327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1381005d0 msgr2=0x7fa1381995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1381005d0 0x7fa1381995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 -- 172.21.15.150:0/2805489327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138104f00 msgr2=0x7fa138199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138104f00 0x7fa138199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.476+0000 7fa13c85f700 1 -- 172.21.15.150:0/2805489327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa130009d00 con 0x7fa138105850 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.477+0000 7fa13c85f700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa1381a4480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b070 tx=0x7fa130007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.477+0000 7fa1357fa700 1 -- 172.21.15.150:0/2805489327 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000e040 con 0x7fa138105850 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.477+0000 7fa1357fa700 1 -- 172.21.15.150:0/2805489327 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa130009440 con 0x7fa138105850 2024-05-10T04:38:16.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.477+0000 7fa1357fa700 1 -- 172.21.15.150:0/2805489327 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13001b450 con 0x7fa138105850 2024-05-10T04:38:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.477+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1381a4e00 con 0x7fa138105850 2024-05-10T04:38:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.477+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1381a51c0 con 0x7fa138105850 2024-05-10T04:38:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.478+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa138108900 con 0x7fa138105850 2024-05-10T04:38:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.479+0000 7fa1357fa700 1 -- 172.21.15.150:0/2805489327 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa130004cf0 con 0x7fa138105850 2024-05-10T04:38:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.479+0000 7fa1357fa700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1200a1120 0x7fa1200a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:16.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.480+0000 7fa137fff700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1200a1120 0x7fa1200a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:16.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.480+0000 7fa1357fa700 1 -- 172.21.15.150:0/2805489327 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa1300ce7c0 con 0x7fa138105850 2024-05-10T04:38:16.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.480+0000 7fa137fff700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1200a1120 0x7fa1200a3630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa128005950 tx=0x7fa12800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:16.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.483+0000 7fa1357fa700 1 -- 172.21.15.150:0/2805489327 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa130097df0 con 0x7fa138105850 2024-05-10T04:38:16.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.723+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa13810e2b0 con 0x7fa138105850 2024-05-10T04:38:16.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.724+0000 7fa1357fa700 1 -- 172.21.15.150:0/2805489327 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa130011390 con 0x7fa138105850 2024-05-10T04:38:16.725 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T04:38:16.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1200a1120 msgr2=0x7fa1200a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:16.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1200a1120 0x7fa1200a3630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa128005950 tx=0x7fa12800b410 comp rx=0 tx=0).stop 2024-05-10T04:38:16.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 msgr2=0x7fa1381a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:16.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa1381a4480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b070 tx=0x7fa130007870 comp rx=0 tx=0).stop 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 shutdown_connections 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa138105850 0x7fa1381a4480 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1200a1120 0x7fa1200a3630 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1381005d0 0x7fa1381995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 --2- 172.21.15.150:0/2805489327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138104f00 0x7fa138199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 >> 172.21.15.150:0/2805489327 conn(0x7fa1380fbee0 msgr2=0x7fa138102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 shutdown_connections 2024-05-10T04:38:16.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:16.726+0000 7fa13e2c2700 1 -- 172.21.15.150:0/2805489327 wait complete. 2024-05-10T04:38:17.023 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:16 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2805489327' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:38:17.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:16 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2805489327' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:38:17.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:16 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2805489327' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T04:38:17.190 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T04:38:17.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.191 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-05-10T04:38:17.209 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T04:38:17.209 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T04:38:17.209 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T04:38:17.209 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:38:17.209 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-05-10T04:38:17.209 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:38:17.209 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 41871sec preferred_lft 41871sec 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::c92:6035:7edc:b85a/64 scope link noprefixroute 2024-05-10T04:38:17.210 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:38:17.211 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T04:38:17.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.211 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:38:17.211 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:38:17.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-05-10T04:38:17.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-05-10T04:38:17.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-05-10T04:38:17.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T04:38:17.211 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:38:17.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.394 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T04:38:17.422 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-05-10T04:38:17.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.424 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-05-10T04:38:17.493 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-05-10T04:38:17.494 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T04:38:17.494 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T04:38:17.494 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T04:38:17.494 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T04:38:17.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.494 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:38:17.494 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:38:17.494 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T04:38:17.494 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T04:38:17.495 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T04:38:17.495 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:38:17.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.773 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:38:17.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.791 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-05-10T04:38:17.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.848 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:38:17.848 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:38:17.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T04:38:17.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-10T04:38:17.848 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:38:17.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:17 smithi150 ceph-mon[42404]: pgmap v229: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-10T04:38:17.945 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-10T04:38:17.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:17.945 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:38:17.945 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:38:17.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-10T04:38:17.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T04:38:17.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-10T04:38:17.946 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-10T04:38:17.946 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-10T04:38:17.946 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:38:18.085 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:17 smithi161 ceph-mon[46912]: pgmap v229: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-10T04:38:18.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:17 smithi181 ceph-mon[47314]: pgmap v229: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-10T04:38:18.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:18.219 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:38:18.219 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:38:18.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-05-10T04:38:18.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T04:38:18.219 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:38:18.317 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-10T04:38:18.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:38:18.318 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:18.335 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-10T04:38:18.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:38:18.335 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:18.392 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2024-05-10T04:38:18.459 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-10T04:38:18.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-10T04:38:18.527 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-10T04:38:18.528 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-10T04:38:18.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:18.528 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2024-05-10T04:38:18.586 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-10T04:38:18.586 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-10T04:38:18.628 DEBUG:teuthology.orchestra.run.smithi150:> sudo modprobe fuse 2024-05-10T04:38:18.655 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-10T04:38:18.655 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-10T04:38:18.726 INFO:teuthology.orchestra.run.smithi150.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-10T04:38:18.727 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-10T04:38:18.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:18.727 DEBUG:teuthology.orchestra.run.smithi150:> ls /sys/fs/fuse/connections 2024-05-10T04:38:18.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:2024-05-10T04:38:18.726+0000 7f4349a1d5c0 -1 init, newargv = 0x7f432c004ba0 newargc=15 2024-05-10T04:38:18.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:ceph-fuse[95567]: starting ceph client 2024-05-10T04:38:18.735 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:ceph-fuse[95567]: starting fuse 2024-05-10T04:38:18.746 INFO:teuthology.orchestra.run.smithi150.stdout:124 2024-05-10T04:38:18.746 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2024-05-10T04:38:18.746 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-10T04:38:18.746 DEBUG:teuthology.orchestra.run.smithi150:> import glob 2024-05-10T04:38:18.746 DEBUG:teuthology.orchestra.run.smithi150:> import re 2024-05-10T04:38:18.746 DEBUG:teuthology.orchestra.run.smithi150:> import os 2024-05-10T04:38:18.746 DEBUG:teuthology.orchestra.run.smithi150:> import subprocess 2024-05-10T04:38:18.746 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> def _find_admin_socket(client_name): 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> files = glob.glob(asok_path) 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> # Given a non-glob path, it better be there 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> if "*" not in asok_path: 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> assert(len(files) == 1) 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> return files[0] 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> for f in files: 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-10T04:38:18.747 DEBUG:teuthology.orchestra.run.smithi150:> contents = proc_f.read() 2024-05-10T04:38:18.748 DEBUG:teuthology.orchestra.run.smithi150:> if mountpoint in contents: 2024-05-10T04:38:18.748 DEBUG:teuthology.orchestra.run.smithi150:> return f 2024-05-10T04:38:18.748 DEBUG:teuthology.orchestra.run.smithi150:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-10T04:38:18.748 DEBUG:teuthology.orchestra.run.smithi150:> 2024-05-10T04:38:18.748 DEBUG:teuthology.orchestra.run.smithi150:> print(_find_admin_socket("client.admin")) 2024-05-10T04:38:18.748 DEBUG:teuthology.orchestra.run.smithi150:> ' 2024-05-10T04:38:18.870 INFO:teuthology.orchestra.run.smithi150.stdout:/var/run/ceph/ceph-client.admin.95567.asok 2024-05-10T04:38:19.053 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.95567.asok 2024-05-10T04:38:19.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:19.053 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.95567.asok status 2024-05-10T04:38:19.220 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "metadata": { 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_sha1": "50174015a7e79f951b988949abe236791f52d4da", 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.2.2-1330-g50174015 (50174015a7e79f951b988949abe236791f52d4da) reef (stable)", 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "entity_id": "admin", 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": "smithi150", 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "pid": "95567", 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "root": "/" 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_count": 0, 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "dentry_pinned_count": 0, 2024-05-10T04:38:19.221 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 14811, 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "inst": { 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "name": { 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "client", 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "num": 14811 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 51874402 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": { 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-05-10T04:38:19.222 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "192.168.0.1:0", 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 51874402 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "inst_str": "client.14811 192.168.0.1:0/51874402", 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "addr_str": "192.168.0.1:0/51874402", 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "inode_count": 1, 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_epoch": 17, 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch": 86, 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_epoch_barrier": 0, 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "blocklisted": false, 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs" 2024-05-10T04:38:19.223 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:38:19.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:19.231 DEBUG:teuthology.orchestra.run.smithi150:> stat --file-system '--printf=%T 2024-05-10T04:38:19.231 DEBUG:teuthology.orchestra.run.smithi150:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:19.248 INFO:teuthology.orchestra.run.smithi150.stdout:fuseblk 2024-05-10T04:38:19.249 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:19.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:19.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:19.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:19.317 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-10T04:38:19.425 INFO:teuthology.orchestra.run.smithi150.stdout:# file: . 2024-05-10T04:38:19.425 INFO:teuthology.orchestra.run.smithi150.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-10T04:38:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:19.425 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:38:19.426 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:38:19.445 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:19.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/2659502508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe64100570 msgr2=0x7fbe6410a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/2659502508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe64100570 0x7fbe6410a190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00b0a0 tx=0x7fbe5c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/2659502508 shutdown_connections 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/2659502508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe64100570 0x7fbe6410a190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/2659502508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe641031e0 0x7fbe641035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/2659502508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe640ffc20 0x7fbe64100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/2659502508 >> 172.21.15.150:0/2659502508 conn(0x7fbe640fb8f0 msgr2=0x7fbe640fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/2659502508 shutdown_connections 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/2659502508 wait complete. 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 Processor -- start 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.616+0000 7fbe6a9c5700 1 -- start start 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe6a9c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe640ffc20 0x7fbe641980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe6a9c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64100570 0x7fbe641985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe6a9c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe641031e0 0x7fbe6419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe6a9c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419fb80 con 0x7fbe640ffc20 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe6a9c5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419fcf0 con 0x7fbe641031e0 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe6a9c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6419feb0 con 0x7fbe64100570 2024-05-10T04:38:19.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe640ffc20 0x7fbe641980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe640ffc20 0x7fbe641980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57392/0 (socket says 172.21.15.150:57392) 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 -- 172.21.15.150:0/4036809082 learned_addr learned my addr 172.21.15.150:0/4036809082 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe637fe700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64100570 0x7fbe641985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe68f62700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe641031e0 0x7fbe6419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 -- 172.21.15.150:0/4036809082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64100570 msgr2=0x7fbe641985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64100570 0x7fbe641985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 -- 172.21.15.150:0/4036809082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe641031e0 msgr2=0x7fbe6419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe641031e0 0x7fbe6419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 -- 172.21.15.150:0/4036809082 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c009d00 con 0x7fbe640ffc20 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe68f62700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe641031e0 0x7fbe6419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe63fff700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe640ffc20 0x7fbe641980a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e940 tx=0x7fbe5400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.617+0000 7fbe617fa700 1 -- 172.21.15.150:0/4036809082 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400cbd0 con 0x7fbe640ffc20 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.618+0000 7fbe617fa700 1 -- 172.21.15.150:0/4036809082 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe5400ee90 con 0x7fbe640ffc20 2024-05-10T04:38:19.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.618+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe641a0190 con 0x7fbe640ffc20 2024-05-10T04:38:19.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.618+0000 7fbe617fa700 1 -- 172.21.15.150:0/4036809082 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54003cb0 con 0x7fbe640ffc20 2024-05-10T04:38:19.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.618+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe641a06a0 con 0x7fbe640ffc20 2024-05-10T04:38:19.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.618+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe64109030 con 0x7fbe640ffc20 2024-05-10T04:38:19.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.620+0000 7fbe617fa700 1 -- 172.21.15.150:0/4036809082 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fbe5400cd30 con 0x7fbe640ffc20 2024-05-10T04:38:19.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.620+0000 7fbe617fa700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbe500a1120 0x7fbe500a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.620+0000 7fbe617fa700 1 -- 172.21.15.150:0/4036809082 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbe54014070 con 0x7fbe640ffc20 2024-05-10T04:38:19.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.620+0000 7fbe637fe700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbe500a1120 0x7fbe500a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.621+0000 7fbe637fe700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbe500a1120 0x7fbe500a3630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe64199140 tx=0x7fbe4c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:19.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.624+0000 7fbe617fa700 1 -- 172.21.15.150:0/4036809082 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbe54096330 con 0x7fbe640ffc20 2024-05-10T04:38:19.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.823+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe640678b0 con 0x7fbe640ffc20 2024-05-10T04:38:19.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.825+0000 7fbe617fa700 1 -- 172.21.15.150:0/4036809082 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7fbe54099950 con 0x7fbe640ffc20 2024-05-10T04:38:19.826 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:19.827 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbe500a1120 msgr2=0x7fbe500a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbe500a1120 0x7fbe500a3630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe64199140 tx=0x7fbe4c0073d0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe640ffc20 msgr2=0x7fbe641980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe640ffc20 0x7fbe641980a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e940 tx=0x7fbe5400ec50 comp rx=0 tx=0).stop 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 shutdown_connections 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe641031e0 0x7fbe6419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe640ffc20 0x7fbe641980a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbe500a1120 0x7fbe500a3630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 --2- 172.21.15.150:0/4036809082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64100570 0x7fbe641985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 >> 172.21.15.150:0/4036809082 conn(0x7fbe640fb8f0 msgr2=0x7fbe64106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:19.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 shutdown_connections 2024-05-10T04:38:19.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.827+0000 7fbe6a9c5700 1 -- 172.21.15.150:0/4036809082 wait complete. 2024-05-10T04:38:19.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:38:19.842 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:38:19.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 -- 172.21.15.150:0/4273584526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc100ac0 msgr2=0x7f8dfc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 --2- 172.21.15.150:0/4273584526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc100ac0 0x7f8dfc100ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009990 tx=0x7f8dec009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 -- 172.21.15.150:0/4273584526 shutdown_connections 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 --2- 172.21.15.150:0/4273584526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dfc102580 0x7f8dfc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 --2- 172.21.15.150:0/4273584526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc100ac0 0x7f8dfc100ed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 --2- 172.21.15.150:0/4273584526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dfc101be0 0x7f8dfc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 -- 172.21.15.150:0/4273584526 >> 172.21.15.150:0/4273584526 conn(0x7f8dfc0fc0d0 msgr2=0x7f8dfc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 -- 172.21.15.150:0/4273584526 shutdown_connections 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.976+0000 7f8e04383700 1 -- 172.21.15.150:0/4273584526 wait complete. 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 Processor -- start 2024-05-10T04:38:19.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 -- start start 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dfc100ac0 0x7f8dfc196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101be0 0x7f8dfc197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dfc102580 0x7f8dfc19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc19e8d0 con 0x7f8dfc101be0 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc19ea40 con 0x7f8dfc100ac0 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e04383700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc19ec00 con 0x7f8dfc102580 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101be0 0x7f8dfc197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101be0 0x7f8dfc197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57416/0 (socket says 172.21.15.150:57416) 2024-05-10T04:38:19.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 -- 172.21.15.150:0/3069016970 learned_addr learned my addr 172.21.15.150:0/3069016970 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:19.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e02920700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dfc102580 0x7f8dfc19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0211f700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dfc100ac0 0x7f8dfc196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 -- 172.21.15.150:0/3069016970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dfc102580 msgr2=0x7f8dfc19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dfc102580 0x7f8dfc19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 -- 172.21.15.150:0/3069016970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dfc100ac0 msgr2=0x7f8dfc196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dfc100ac0 0x7f8dfc196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.977+0000 7f8e0191e700 1 -- 172.21.15.150:0/3069016970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec0095f0 con 0x7f8dfc101be0 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.978+0000 7f8e0191e700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101be0 0x7f8dfc197340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e9a0 tx=0x7f8df800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.978+0000 7f8df37fe700 1 -- 172.21.15.150:0/3069016970 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8009790 con 0x7f8dfc101be0 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.978+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc19eee0 con 0x7f8dfc101be0 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.978+0000 7f8df37fe700 1 -- 172.21.15.150:0/3069016970 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df800cbd0 con 0x7f8dfc101be0 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.978+0000 7f8df37fe700 1 -- 172.21.15.150:0/3069016970 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8018680 con 0x7f8dfc101be0 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.978+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc19f300 con 0x7f8dfc101be0 2024-05-10T04:38:19.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.979+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc107cd0 con 0x7f8dfc101be0 2024-05-10T04:38:19.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.980+0000 7f8df37fe700 1 -- 172.21.15.150:0/3069016970 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8df800cd40 con 0x7f8dfc101be0 2024-05-10T04:38:19.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.981+0000 7f8df37fe700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8de80a11c0 0x7f8de80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:19.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.981+0000 7f8e0211f700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8de80a11c0 0x7f8de80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:19.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.981+0000 7f8df37fe700 1 -- 172.21.15.150:0/3069016970 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8df8014070 con 0x7f8dfc101be0 2024-05-10T04:38:19.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.982+0000 7f8e0211f700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8de80a11c0 0x7f8de80a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc1981f0 tx=0x7f8dec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:19.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:19.985+0000 7f8df37fe700 1 -- 172.21.15.150:0/3069016970 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8df8096820 con 0x7f8dfc101be0 2024-05-10T04:38:20.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.181+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dfc10c2a0 con 0x7f8dfc101be0 2024-05-10T04:38:20.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.182+0000 7f8df37fe700 1 -- 172.21.15.150:0/3069016970 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8df8099e40 con 0x7f8dfc101be0 2024-05-10T04:38:20.183 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8de80a11c0 msgr2=0x7f8de80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8de80a11c0 0x7f8de80a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc1981f0 tx=0x7f8dec01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101be0 msgr2=0x7f8dfc197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101be0 0x7f8dfc197340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e9a0 tx=0x7f8df800ed60 comp rx=0 tx=0).stop 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 shutdown_connections 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8de80a11c0 0x7f8de80a36d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dfc102580 0x7f8dfc19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dfc100ac0 0x7f8dfc196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 --2- 172.21.15.150:0/3069016970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101be0 0x7f8dfc197340 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 >> 172.21.15.150:0/3069016970 conn(0x7f8dfc0fc0d0 msgr2=0x7f8dfc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:20.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 shutdown_connections 2024-05-10T04:38:20.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.184+0000 7f8e04383700 1 -- 172.21.15.150:0/3069016970 wait complete. 2024-05-10T04:38:20.198 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 -- 172.21.15.150:0/2948900912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec105760 msgr2=0x7f87ec105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 --2- 172.21.15.150:0/2948900912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec105760 0x7f87ec105b70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b0a0 tx=0x7f87e400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 -- 172.21.15.150:0/2948900912 shutdown_connections 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 --2- 172.21.15.150:0/2948900912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec105760 0x7f87ec105b70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 --2- 172.21.15.150:0/2948900912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ec1004a0 0x7f87ec1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 --2- 172.21.15.150:0/2948900912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec102e00 0x7f87ec105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 -- 172.21.15.150:0/2948900912 >> 172.21.15.150:0/2948900912 conn(0x7f87ec0fc0d0 msgr2=0x7f87ec0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 -- 172.21.15.150:0/2948900912 shutdown_connections 2024-05-10T04:38:20.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.333+0000 7f87f0d55700 1 -- 172.21.15.150:0/2948900912 wait complete. 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 Processor -- start 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 -- start start 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec1004a0 0x7f87ec19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec102e00 0x7f87ec19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ec105760 0x7f87ec1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a6150 con 0x7f87ec1004a0 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a62c0 con 0x7f87ec105760 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87f0d55700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a6480 con 0x7f87ec102e00 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87ea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec1004a0 0x7f87ec19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87ea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec1004a0 0x7f87ec19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57430/0 (socket says 172.21.15.150:57430) 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87ea59c700 1 -- 172.21.15.150:0/3052280319 learned_addr learned my addr 172.21.15.150:0/3052280319 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.334+0000 7f87e9d9b700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec102e00 0x7f87ec19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87ead9d700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ec105760 0x7f87ec1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87ea59c700 1 -- 172.21.15.150:0/3052280319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec102e00 msgr2=0x7f87ec19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87ea59c700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec102e00 0x7f87ec19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87ea59c700 1 -- 172.21.15.150:0/3052280319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ec105760 msgr2=0x7f87ec1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87ea59c700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ec105760 0x7f87ec1a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87ea59c700 1 -- 172.21.15.150:0/3052280319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4009d00 con 0x7f87ec1004a0 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87ea59c700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec1004a0 0x7f87ec19e670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e970 tx=0x7f87e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87db7fe700 1 -- 172.21.15.150:0/3052280319 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0009790 con 0x7f87ec1004a0 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87db7fe700 1 -- 172.21.15.150:0/3052280319 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e000eec0 con 0x7f87ec1004a0 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87db7fe700 1 -- 172.21.15.150:0/3052280319 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0005590 con 0x7f87ec1004a0 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1a6760 con 0x7f87ec1004a0 2024-05-10T04:38:20.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.335+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a6c20 con 0x7f87ec1004a0 2024-05-10T04:38:20.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.336+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec0fdc80 con 0x7f87ec1004a0 2024-05-10T04:38:20.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.338+0000 7f87db7fe700 1 -- 172.21.15.150:0/3052280319 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f87e000cbd0 con 0x7f87ec1004a0 2024-05-10T04:38:20.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.339+0000 7f87db7fe700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87d40a11c0 0x7f87d40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.339+0000 7f87db7fe700 1 -- 172.21.15.150:0/3052280319 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f87e0014070 con 0x7f87ec1004a0 2024-05-10T04:38:20.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.339+0000 7f87e9d9b700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87d40a11c0 0x7f87d40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.339+0000 7f87e9d9b700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87d40a11c0 0x7f87d40a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87ec19f710 tx=0x7f87dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:20.343 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.342+0000 7f87db7fe700 1 -- 172.21.15.150:0/3052280319 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f87e00964e0 con 0x7f87ec1004a0 2024-05-10T04:38:20.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.539+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87ec113a10 con 0x7f87ec1004a0 2024-05-10T04:38:20.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.540+0000 7f87db7fe700 1 -- 172.21.15.150:0/3052280319 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87e0004ef0 con 0x7f87ec1004a0 2024-05-10T04:38:20.541 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87d40a11c0 msgr2=0x7f87d40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87d40a11c0 0x7f87d40a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87ec19f710 tx=0x7f87dc009260 comp rx=0 tx=0).stop 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec1004a0 msgr2=0x7f87ec19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec1004a0 0x7f87ec19e670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e970 tx=0x7f87e000ec80 comp rx=0 tx=0).stop 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 shutdown_connections 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ec105760 0x7f87ec1a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87ec1004a0 0x7f87ec19e670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f87d40a11c0 0x7f87d40a36d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 --2- 172.21.15.150:0/3052280319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec102e00 0x7f87ec19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 >> 172.21.15.150:0/3052280319 conn(0x7f87ec0fc0d0 msgr2=0x7f87ec1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:20.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 shutdown_connections 2024-05-10T04:38:20.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.542+0000 7f87f0d55700 1 -- 172.21.15.150:0/3052280319 wait complete. 2024-05-10T04:38:20.555 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:20 smithi150 ceph-mon[42404]: pgmap v230: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-10T04:38:20.555 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:20 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4036809082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:20.555 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:20 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3069016970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:20.556 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:38:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:20 smithi181 ceph-mon[47314]: pgmap v230: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-10T04:38:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:20 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4036809082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:20 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3069016970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:20.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 -- 172.21.15.150:0/2022490834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a541013e0 msgr2=0x7f8a541017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 --2- 172.21.15.150:0/2022490834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a541013e0 0x7f8a541017f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000aaa0 tx=0x7f8a4000adb0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 -- 172.21.15.150:0/2022490834 shutdown_connections 2024-05-10T04:38:20.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 --2- 172.21.15.150:0/2022490834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a54101d80 0x7f8a54108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 --2- 172.21.15.150:0/2022490834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a541002c0 0x7f8a541006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 --2- 172.21.15.150:0/2022490834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a541013e0 0x7f8a541017f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 -- 172.21.15.150:0/2022490834 >> 172.21.15.150:0/2022490834 conn(0x7f8a540fb8f0 msgr2=0x7f8a540fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:20.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 -- 172.21.15.150:0/2022490834 shutdown_connections 2024-05-10T04:38:20.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 -- 172.21.15.150:0/2022490834 wait complete. 2024-05-10T04:38:20.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 Processor -- start 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 -- start start 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a541002c0 0x7f8a54197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a541013e0 0x7f8a54195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.690+0000 7f8a5af27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a54101d80 0x7f8a54196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a5af27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54196710 con 0x7f8a54101d80 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a5af27700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54196850 con 0x7f8a541013e0 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a5af27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a541a2050 con 0x7f8a541002c0 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a54101d80 0x7f8a54196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a54101d80 0x7f8a54196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57442/0 (socket says 172.21.15.150:57442) 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 -- 172.21.15.150:0/1514015148 learned_addr learned my addr 172.21.15.150:0/1514015148 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a58cc3700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a541002c0 0x7f8a54197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a53fff700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a541013e0 0x7f8a54195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 -- 172.21.15.150:0/1514015148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a541002c0 msgr2=0x7f8a54197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a541002c0 0x7f8a54197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 -- 172.21.15.150:0/1514015148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a541013e0 msgr2=0x7f8a54195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a541013e0 0x7f8a54195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 -- 172.21.15.150:0/1514015148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4000a700 con 0x7f8a54101d80 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a594c4700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a54101d80 0x7f8a54196060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c007ee0 tx=0x7f8a4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a51ffb700 1 -- 172.21.15.150:0/1514015148 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c017070 con 0x7f8a54101d80 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a51ffb700 1 -- 172.21.15.150:0/1514015148 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a4c00f040 con 0x7f8a54101d80 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a541a2250 con 0x7f8a54101d80 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.691+0000 7f8a51ffb700 1 -- 172.21.15.150:0/1514015148 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c005580 con 0x7f8a54101d80 2024-05-10T04:38:20.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.692+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a541a2780 con 0x7f8a54101d80 2024-05-10T04:38:20.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.693+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a541074d0 con 0x7f8a54101d80 2024-05-10T04:38:20.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.694+0000 7f8a51ffb700 1 -- 172.21.15.150:0/1514015148 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8a4c00db20 con 0x7f8a54101d80 2024-05-10T04:38:20.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.694+0000 7f8a51ffb700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8a440a1240 0x7f8a440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:20.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.694+0000 7f8a51ffb700 1 -- 172.21.15.150:0/1514015148 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8a4c0cc6b0 con 0x7f8a54101d80 2024-05-10T04:38:20.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.694+0000 7f8a58cc3700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8a440a1240 0x7f8a440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:20.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.695+0000 7f8a58cc3700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8a440a1240 0x7f8a440a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8a541012e0 tx=0x7f8a4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:20.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.698+0000 7f8a51ffb700 1 -- 172.21.15.150:0/1514015148 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8a4c095c50 con 0x7f8a54101d80 2024-05-10T04:38:20.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:20 smithi161 ceph-mon[46912]: pgmap v230: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-10T04:38:20.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:20 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4036809082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:20.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:20 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3069016970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:20.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.901+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a5410b2a0 con 0x7f8a54101d80 2024-05-10T04:38:20.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.901+0000 7f8a51ffb700 1 -- 172.21.15.150:0/1514015148 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a4c099270 con 0x7f8a54101d80 2024-05-10T04:38:20.903 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:38:20.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8a440a1240 msgr2=0x7f8a440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8a440a1240 0x7f8a440a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8a541012e0 tx=0x7f8a4800b410 comp rx=0 tx=0).stop 2024-05-10T04:38:20.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a54101d80 msgr2=0x7f8a54196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a54101d80 0x7f8a54196060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c007ee0 tx=0x7f8a4c00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 shutdown_connections 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a54101d80 0x7f8a54196060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8a440a1240 0x7f8a440a3750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a541002c0 0x7f8a54197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 --2- 172.21.15.150:0/1514015148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a541013e0 0x7f8a54195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 >> 172.21.15.150:0/1514015148 conn(0x7f8a540fb8f0 msgr2=0x7f8a54104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 shutdown_connections 2024-05-10T04:38:20.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:20.904+0000 7f8a5af27700 1 -- 172.21.15.150:0/1514015148 wait complete. 2024-05-10T04:38:20.918 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:38:21.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 -- 172.21.15.150:0/50953744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac1002c0 msgr2=0x7f09ac1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 --2- 172.21.15.150:0/50953744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac1002c0 0x7f09ac1006d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f099c009990 tx=0x7f099c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 -- 172.21.15.150:0/50953744 shutdown_connections 2024-05-10T04:38:21.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 --2- 172.21.15.150:0/50953744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09ac101d80 0x7f09ac108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 --2- 172.21.15.150:0/50953744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac1002c0 0x7f09ac1006d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 --2- 172.21.15.150:0/50953744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac1013e0 0x7f09ac1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 -- 172.21.15.150:0/50953744 >> 172.21.15.150:0/50953744 conn(0x7f09ac0fb8f0 msgr2=0x7f09ac0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 -- 172.21.15.150:0/50953744 shutdown_connections 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.053+0000 7f09b35c9700 1 -- 172.21.15.150:0/50953744 wait complete. 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 Processor -- start 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 -- start start 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac1002c0 0x7f09ac195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09ac1013e0 0x7f09ac196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac101d80 0x7f09ac19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac19d7c0 con 0x7f09ac101d80 2024-05-10T04:38:21.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac19d930 con 0x7f09ac1002c0 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b35c9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac19daf0 con 0x7f09ac1013e0 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac101d80 0x7f09ac19d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1365700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac1002c0 0x7f09ac195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1365700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac1002c0 0x7f09ac195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:36700/0 (socket says 172.21.15.150:36700) 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac101d80 0x7f09ac19d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57460/0 (socket says 172.21.15.150:57460) 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1365700 1 -- 172.21.15.150:0/2950935952 learned_addr learned my addr 172.21.15.150:0/2950935952 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b0b64700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09ac1013e0 0x7f09ac196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 -- 172.21.15.150:0/2950935952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09ac1013e0 msgr2=0x7f09ac196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09ac1013e0 0x7f09ac196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 -- 172.21.15.150:0/2950935952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac1002c0 msgr2=0x7f09ac195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac1002c0 0x7f09ac195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 -- 172.21.15.150:0/2950935952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c0095f0 con 0x7f09ac101d80 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b0b64700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09ac1013e0 0x7f09ac196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.054+0000 7f09b1b66700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac101d80 0x7f09ac19d180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09a400d0c0 tx=0x7f09a400d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.055+0000 7f09a27fc700 1 -- 172.21.15.150:0/2950935952 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4019070 con 0x7f09ac101d80 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.055+0000 7f09a27fc700 1 -- 172.21.15.150:0/2950935952 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09a4010040 con 0x7f09ac101d80 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.055+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac19dd70 con 0x7f09ac101d80 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.055+0000 7f09a27fc700 1 -- 172.21.15.150:0/2950935952 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a400f020 con 0x7f09ac101d80 2024-05-10T04:38:21.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.055+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac19e1d0 con 0x7f09ac101d80 2024-05-10T04:38:21.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.056+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac1074d0 con 0x7f09ac101d80 2024-05-10T04:38:21.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.058+0000 7f09a27fc700 1 -- 172.21.15.150:0/2950935952 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f09a400da80 con 0x7f09ac101d80 2024-05-10T04:38:21.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.058+0000 7f09a27fc700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f09980a1240 0x7f09980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.058+0000 7f09a27fc700 1 -- 172.21.15.150:0/2950935952 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f09a40cdff0 con 0x7f09ac101d80 2024-05-10T04:38:21.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.058+0000 7f09b1365700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f09980a1240 0x7f09980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.059+0000 7f09b1365700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f09980a1240 0x7f09980a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b2e0 tx=0x7f099c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:21.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.062+0000 7f09a27fc700 1 -- 172.21.15.150:0/2950935952 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f09a4097620 con 0x7f09ac101d80 2024-05-10T04:38:21.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.259+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f09ac02ce20 con 0x7f09ac101d80 2024-05-10T04:38:21.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.259+0000 7f09a27fc700 1 -- 172.21.15.150:0/2950935952 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f09a409ac40 con 0x7f09ac101d80 2024-05-10T04:38:21.261 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:38:21.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f09980a1240 msgr2=0x7f09980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f09980a1240 0x7f09980a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b2e0 tx=0x7f099c01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:21.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac101d80 msgr2=0x7f09ac19d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac101d80 0x7f09ac19d180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09a400d0c0 tx=0x7f09a400d3d0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 shutdown_connections 2024-05-10T04:38:21.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ac101d80 0x7f09ac19d180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09ac1002c0 0x7f09ac195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f09980a1240 0x7f09980a3750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 --2- 172.21.15.150:0/2950935952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09ac1013e0 0x7f09ac196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 >> 172.21.15.150:0/2950935952 conn(0x7f09ac0fb8f0 msgr2=0x7f09ac104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:21.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 shutdown_connections 2024-05-10T04:38:21.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.262+0000 7f09b35c9700 1 -- 172.21.15.150:0/2950935952 wait complete. 2024-05-10T04:38:21.276 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T04:38:21.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.409+0000 7f38700bb700 1 -- 172.21.15.150:0/1158185455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868101c00 msgr2=0x7f3868102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.409+0000 7f38700bb700 1 --2- 172.21.15.150:0/1158185455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868101c00 0x7f3868102010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f385c009990 tx=0x7f385c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.409+0000 7f38700bb700 1 -- 172.21.15.150:0/1158185455 shutdown_connections 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.409+0000 7f38700bb700 1 --2- 172.21.15.150:0/1158185455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38681025a0 0x7f3868108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.409+0000 7f38700bb700 1 --2- 172.21.15.150:0/1158185455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3868100ae0 0x7f3868100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.409+0000 7f38700bb700 1 --2- 172.21.15.150:0/1158185455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868101c00 0x7f3868102010 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.409+0000 7f38700bb700 1 -- 172.21.15.150:0/1158185455 >> 172.21.15.150:0/1158185455 conn(0x7f38680fc0f0 msgr2=0x7f38680fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 -- 172.21.15.150:0/1158185455 shutdown_connections 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 -- 172.21.15.150:0/1158185455 wait complete. 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 Processor -- start 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 -- start start 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868100ae0 0x7f3868197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3868101c00 0x7f3868198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38681025a0 0x7f386819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386819fa00 con 0x7f3868100ae0 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386819fb70 con 0x7f38681025a0 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f38700bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386819fd30 con 0x7f3868101c00 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.410+0000 7f386de57700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868100ae0 0x7f3868197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868100ae0 0x7f3868197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57478/0 (socket says 172.21.15.150:57478) 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 -- 172.21.15.150:0/4173921927 learned_addr learned my addr 172.21.15.150:0/4173921927 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386d656700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3868101c00 0x7f3868198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386e658700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38681025a0 0x7f386819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 -- 172.21.15.150:0/4173921927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3868101c00 msgr2=0x7f3868198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3868101c00 0x7f3868198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 -- 172.21.15.150:0/4173921927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38681025a0 msgr2=0x7f386819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38681025a0 0x7f386819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 -- 172.21.15.150:0/4173921927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c0095f0 con 0x7f3868100ae0 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386e658700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38681025a0 0x7f386819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f386de57700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868100ae0 0x7f3868197f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f386400d950 tx=0x7f386400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f385affd700 1 -- 172.21.15.150:0/4173921927 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864004220 con 0x7f3868100ae0 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f385affd700 1 -- 172.21.15.150:0/4173921927 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3864004380 con 0x7f3868100ae0 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386819ffb0 con 0x7f3868100ae0 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f385affd700 1 -- 172.21.15.150:0/4173921927 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864003bf0 con 0x7f3868100ae0 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.411+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38681a0460 con 0x7f3868100ae0 2024-05-10T04:38:21.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.412+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3868107cf0 con 0x7f3868100ae0 2024-05-10T04:38:21.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.413+0000 7f385affd700 1 -- 172.21.15.150:0/4173921927 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3864003d50 con 0x7f3868100ae0 2024-05-10T04:38:21.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.414+0000 7f385affd700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f38540a11c0 0x7f38540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.414+0000 7f386d656700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f38540a11c0 0x7f38540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.414+0000 7f385affd700 1 -- 172.21.15.150:0/4173921927 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f38640ccd60 con 0x7f3868100ae0 2024-05-10T04:38:21.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.414+0000 7f386d656700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f38540a11c0 0x7f38540a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f385c00b5c0 tx=0x7f385c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:21.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.418+0000 7f385affd700 1 -- 172.21.15.150:0/4173921927 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3864096300 con 0x7f3868100ae0 2024-05-10T04:38:21.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.630+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f386810d470 con 0x7f3868100ae0 2024-05-10T04:38:21.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.630+0000 7f385affd700 1 -- 172.21.15.150:0/4173921927 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+599 (secure 0 0 0) 0x7f3864017020 con 0x7f3868100ae0 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/1928870080 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3285068402 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/881426482 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/618215791 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/2517418767 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3236490116 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/881426482 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/2970915543 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/2970915543 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3890124032 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f38540a11c0 msgr2=0x7f38540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f38540a11c0 0x7f38540a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f385c00b5c0 tx=0x7f385c01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868100ae0 msgr2=0x7f3868197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868100ae0 0x7f3868197f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f386400d950 tx=0x7f386400dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 shutdown_connections 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38681025a0 0x7f386819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3868100ae0 0x7f3868197f20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f38540a11c0 0x7f38540a36d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 --2- 172.21.15.150:0/4173921927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3868101c00 0x7f3868198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.632+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 >> 172.21.15.150:0/4173921927 conn(0x7f38680fc0f0 msgr2=0x7f3868104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:21.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.633+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 shutdown_connections 2024-05-10T04:38:21.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.633+0000 7f38700bb700 1 -- 172.21.15.150:0/4173921927 wait complete. 2024-05-10T04:38:21.635 INFO:teuthology.orchestra.run.smithi150.stderr:listed 10 entries 2024-05-10T04:38:21.645 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:21 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3052280319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:21 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1514015148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.646 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:21 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2950935952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.647 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:38:21.647 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:38:21.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:21 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3052280319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:21 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1514015148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:21 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2950935952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.665 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:21.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:21 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3052280319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:21 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1514015148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:21 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2950935952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:38:21.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 -- 172.21.15.150:0/2966981177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 msgr2=0x7f2fcc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 --2- 172.21.15.150:0/2966981177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc108e30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400b0a0 tx=0x7f2fc400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 -- 172.21.15.150:0/2966981177 shutdown_connections 2024-05-10T04:38:21.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 --2- 172.21.15.150:0/2966981177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc108e30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 --2- 172.21.15.150:0/2966981177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fcc100ac0 0x7f2fcc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 --2- 172.21.15.150:0/2966981177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fcc101be0 0x7f2fcc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 -- 172.21.15.150:0/2966981177 >> 172.21.15.150:0/2966981177 conn(0x7f2fcc0fc0d0 msgr2=0x7f2fcc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:21.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 -- 172.21.15.150:0/2966981177 shutdown_connections 2024-05-10T04:38:21.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.836+0000 7f2fd3205700 1 -- 172.21.15.150:0/2966981177 wait complete. 2024-05-10T04:38:21.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 Processor -- start 2024-05-10T04:38:21.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 -- start start 2024-05-10T04:38:21.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fcc100ac0 0x7f2fcc068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fcc101be0 0x7f2fcc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc19fb80 con 0x7f2fcc102580 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc19fcf0 con 0x7f2fcc101be0 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd3205700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fcc19feb0 con 0x7f2fcc100ac0 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc19f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57494/0 (socket says 172.21.15.150:57494) 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 -- 172.21.15.150:0/1942284407 learned_addr learned my addr 172.21.15.150:0/1942284407 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd0fa1700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fcc100ac0 0x7f2fcc068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fcbfff700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fcc101be0 0x7f2fcc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 -- 172.21.15.150:0/1942284407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fcc100ac0 msgr2=0x7f2fcc068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fcc100ac0 0x7f2fcc068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 -- 172.21.15.150:0/1942284407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fcc101be0 msgr2=0x7f2fcc19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fcc101be0 0x7f2fcc19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:21.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd17a2700 1 -- 172.21.15.150:0/1942284407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc4009d00 con 0x7f2fcc102580 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fd0fa1700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fcc100ac0 0x7f2fcc068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.837+0000 7f2fcbfff700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fcc101be0 0x7f2fcc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.838+0000 7f2fd17a2700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc19f480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400b070 tx=0x7f2fc4007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.838+0000 7f2fc9ffb700 1 -- 172.21.15.150:0/1942284407 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400e040 con 0x7f2fcc102580 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.838+0000 7f2fc9ffb700 1 -- 172.21.15.150:0/1942284407 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc4004c50 con 0x7f2fcc102580 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.838+0000 7f2fc9ffb700 1 -- 172.21.15.150:0/1942284407 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40052a0 con 0x7f2fcc102580 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.838+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fcc1a0130 con 0x7f2fcc102580 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.838+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fcc1a0560 con 0x7f2fcc102580 2024-05-10T04:38:21.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.839+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fcc107cd0 con 0x7f2fcc102580 2024-05-10T04:38:21.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.840+0000 7f2fc9ffb700 1 -- 172.21.15.150:0/1942284407 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2fc401d470 con 0x7f2fcc102580 2024-05-10T04:38:21.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.841+0000 7f2fc9ffb700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2fbc0a11a0 0x7f2fbc0a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:21.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.841+0000 7f2fd0fa1700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2fbc0a11a0 0x7f2fbc0a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:21.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.841+0000 7f2fc9ffb700 1 -- 172.21.15.150:0/1942284407 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2fc40ce590 con 0x7f2fcc102580 2024-05-10T04:38:21.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.841+0000 7f2fd0fa1700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2fbc0a11a0 0x7f2fbc0a36b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc101ae0 tx=0x7f2fc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:21.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:21.844+0000 7f2fc9ffb700 1 -- 172.21.15.150:0/1942284407 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2fc4097be0 con 0x7f2fcc102580 2024-05-10T04:38:22.044 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.043+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fcc10c320 con 0x7f2fcc102580 2024-05-10T04:38:22.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.045+0000 7f2fc9ffb700 1 -- 172.21.15.150:0/1942284407 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f2fc4021090 con 0x7f2fcc102580 2024-05-10T04:38:22.046 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:22.046 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2fbc0a11a0 msgr2=0x7f2fbc0a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2fbc0a11a0 0x7f2fbc0a36b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc101ae0 tx=0x7f2fc0009260 comp rx=0 tx=0).stop 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 msgr2=0x7f2fcc19f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc19f480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400b070 tx=0x7f2fc4007950 comp rx=0 tx=0).stop 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 shutdown_connections 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fcc102580 0x7f2fcc19f480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2fbc0a11a0 0x7f2fbc0a36b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fcc100ac0 0x7f2fcc068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 --2- 172.21.15.150:0/1942284407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fcc101be0 0x7f2fcc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 >> 172.21.15.150:0/1942284407 conn(0x7f2fcc0fc0d0 msgr2=0x7f2fcc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:22.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 shutdown_connections 2024-05-10T04:38:22.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.047+0000 7f2fd3205700 1 -- 172.21.15.150:0/1942284407 wait complete. 2024-05-10T04:38:22.062 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:38:22.062 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 -- 172.21.15.150:0/1003477937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30101be0 msgr2=0x7f7c30101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 --2- 172.21.15.150:0/1003477937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30101be0 0x7f7c30101ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009990 tx=0x7f7c20009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 -- 172.21.15.150:0/1003477937 shutdown_connections 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 --2- 172.21.15.150:0/1003477937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c30102580 0x7f7c30108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 --2- 172.21.15.150:0/1003477937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c30100ac0 0x7f7c30100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 --2- 172.21.15.150:0/1003477937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30101be0 0x7f7c30101ff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 -- 172.21.15.150:0/1003477937 >> 172.21.15.150:0/1003477937 conn(0x7f7c300fc0d0 msgr2=0x7f7c300fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:22.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 -- 172.21.15.150:0/1003477937 shutdown_connections 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.196+0000 7f7c36036700 1 -- 172.21.15.150:0/1003477937 wait complete. 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 Processor -- start 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 -- start start 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c30100ac0 0x7f7c30196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c30101be0 0x7f7c301972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30102580 0x7f7c3019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019e890 con 0x7f7c30102580 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019ea00 con 0x7f7c30101be0 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c36036700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019ebc0 con 0x7f7c30100ac0 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30102580 0x7f7c3019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30102580 0x7f7c3019e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57518/0 (socket says 172.21.15.150:57518) 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.197+0000 7f7c2ffff700 1 -- 172.21.15.150:0/2891455806 learned_addr learned my addr 172.21.15.150:0/2891455806 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2f7fe700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c30100ac0 0x7f7c30196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2effd700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c30101be0 0x7f7c301972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2ffff700 1 -- 172.21.15.150:0/2891455806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c30100ac0 msgr2=0x7f7c30196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2ffff700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c30100ac0 0x7f7c30196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2ffff700 1 -- 172.21.15.150:0/2891455806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c30101be0 msgr2=0x7f7c301972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2ffff700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c30101be0 0x7f7c301972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2ffff700 1 -- 172.21.15.150:0/2891455806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c200095f0 con 0x7f7c30102580 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2effd700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c30101be0 0x7f7c301972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2ffff700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30102580 0x7f7c3019e280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800ef30 tx=0x7f7c2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2cff9700 1 -- 172.21.15.150:0/2891455806 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28018070 con 0x7f7c30102580 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2cff9700 1 -- 172.21.15.150:0/2891455806 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2800f040 con 0x7f7c30102580 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3019ee40 con 0x7f7c30102580 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c2cff9700 1 -- 172.21.15.150:0/2891455806 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800cdb0 con 0x7f7c30102580 2024-05-10T04:38:22.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.198+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3019f270 con 0x7f7c30102580 2024-05-10T04:38:22.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.199+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c30107cd0 con 0x7f7c30102580 2024-05-10T04:38:22.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.201+0000 7f7c2cff9700 1 -- 172.21.15.150:0/2891455806 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7c28003a00 con 0x7f7c30102580 2024-05-10T04:38:22.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.201+0000 7f7c2cff9700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c180a11c0 0x7f7c180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.202+0000 7f7c2f7fe700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c180a11c0 0x7f7c180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.202+0000 7f7c2cff9700 1 -- 172.21.15.150:0/2891455806 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7c280ccfa0 con 0x7f7c30102580 2024-05-10T04:38:22.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.202+0000 7f7c2f7fe700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c180a11c0 0x7f7c180a36d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c30101ae0 tx=0x7f7c24007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:22.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.205+0000 7f7c2cff9700 1 -- 172.21.15.150:0/2891455806 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c280965d0 con 0x7f7c30102580 2024-05-10T04:38:22.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.407+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c3010c2c0 con 0x7f7c30102580 2024-05-10T04:38:22.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.408+0000 7f7c2cff9700 1 -- 172.21.15.150:0/2891455806 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c28099bf0 con 0x7f7c30102580 2024-05-10T04:38:22.409 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:38:22.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c180a11c0 msgr2=0x7f7c180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c180a11c0 0x7f7c180a36d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c30101ae0 tx=0x7f7c24007400 comp rx=0 tx=0).stop 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30102580 msgr2=0x7f7c3019e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30102580 0x7f7c3019e280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800ef30 tx=0x7f7c2800c3a0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 shutdown_connections 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c30102580 0x7f7c3019e280 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c180a11c0 0x7f7c180a36d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c30100ac0 0x7f7c30196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 --2- 172.21.15.150:0/2891455806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c30101be0 0x7f7c301972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 >> 172.21.15.150:0/2891455806 conn(0x7f7c300fc0d0 msgr2=0x7f7c30104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 shutdown_connections 2024-05-10T04:38:22.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.410+0000 7f7c36036700 1 -- 172.21.15.150:0/2891455806 wait complete. 2024-05-10T04:38:22.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:38:22.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 -- 172.21.15.150:0/3876802623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 msgr2=0x7fae341035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/3876802623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae341035f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae24009990 tx=0x7fae24009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 -- 172.21.15.150:0/3876802623 shutdown_connections 2024-05-10T04:38:22.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/3876802623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae34100570 0x7fae3410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/3876802623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae341035f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/3876802623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae340ffc20 0x7fae34100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 -- 172.21.15.150:0/3876802623 >> 172.21.15.150:0/3876802623 conn(0x7fae340fb8f0 msgr2=0x7fae340fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:22.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 -- 172.21.15.150:0/3876802623 shutdown_connections 2024-05-10T04:38:22.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.558+0000 7fae3b5a1700 1 -- 172.21.15.150:0/3876802623 wait complete. 2024-05-10T04:38:22.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 Processor -- start 2024-05-10T04:38:22.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 -- start start 2024-05-10T04:38:22.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae340ffc20 0x7fae34197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae34100570 0x7fae34198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae3419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419fa10 con 0x7fae341031e0 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419fb80 con 0x7fae340ffc20 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.559+0000 7fae3b5a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3419fd40 con 0x7fae34100570 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae3419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae3419f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57528/0 (socket says 172.21.15.150:57528) 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 -- 172.21.15.150:0/1512655489 learned_addr learned my addr 172.21.15.150:0/1512655489 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae3933d700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae340ffc20 0x7fae34197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae38b3c700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae34100570 0x7fae34198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 -- 172.21.15.150:0/1512655489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae34100570 msgr2=0x7fae34198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae34100570 0x7fae34198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 -- 172.21.15.150:0/1512655489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae340ffc20 msgr2=0x7fae34197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae340ffc20 0x7fae34197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 -- 172.21.15.150:0/1512655489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240095f0 con 0x7fae341031e0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae38b3c700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae34100570 0x7fae34198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae39b3e700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae3419f400 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00b970 tx=0x7fae2c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae2a7fc700 1 -- 172.21.15.150:0/1512655489 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00c930 con 0x7fae341031e0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae2a7fc700 1 -- 172.21.15.150:0/1512655489 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae2c014440 con 0x7fae341031e0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3419ffc0 con 0x7fae341031e0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae2a7fc700 1 -- 172.21.15.150:0/1512655489 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0075d0 con 0x7fae341031e0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.560+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae341a03f0 con 0x7fae341031e0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.561+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae34109030 con 0x7fae341031e0 2024-05-10T04:38:22.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.562+0000 7fae2a7fc700 1 -- 172.21.15.150:0/1512655489 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fae2c0145b0 con 0x7fae341031e0 2024-05-10T04:38:22.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.563+0000 7fae2a7fc700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fae200a11c0 0x7fae200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.563+0000 7fae2a7fc700 1 -- 172.21.15.150:0/1512655489 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fae2c006e40 con 0x7fae341031e0 2024-05-10T04:38:22.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.563+0000 7fae3933d700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fae200a11c0 0x7fae200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.563+0000 7fae3933d700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fae200a11c0 0x7fae200a36d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae2400b5c0 tx=0x7fae24011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:22.569 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.568+0000 7fae2a7fc700 1 -- 172.21.15.150:0/1512655489 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fae2c0ffd60 con 0x7fae341031e0 2024-05-10T04:38:22.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:22 smithi181 ceph-mon[47314]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 960 B/s wr, 3 op/s 2024-05-10T04:38:22.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:22 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4173921927' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:38:22.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:22 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1942284407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:22 smithi150 ceph-mon[42404]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 960 B/s wr, 3 op/s 2024-05-10T04:38:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:22 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4173921927' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:38:22.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:22 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1942284407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:22.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.763+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae3402ce20 con 0x7fae341031e0 2024-05-10T04:38:22.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.764+0000 7fae2a7fc700 1 -- 172.21.15.150:0/1512655489 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae2c095710 con 0x7fae341031e0 2024-05-10T04:38:22.765 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fae200a11c0 msgr2=0x7fae200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fae200a11c0 0x7fae200a36d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae2400b5c0 tx=0x7fae24011040 comp rx=0 tx=0).stop 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 msgr2=0x7fae3419f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae3419f400 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00b970 tx=0x7fae2c00bc80 comp rx=0 tx=0).stop 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 shutdown_connections 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae341031e0 0x7fae3419f400 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae340ffc20 0x7fae34197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fae200a11c0 0x7fae200a36d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 --2- 172.21.15.150:0/1512655489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae34100570 0x7fae34198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 >> 172.21.15.150:0/1512655489 conn(0x7fae340fb8f0 msgr2=0x7fae34106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 shutdown_connections 2024-05-10T04:38:22.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.766+0000 7fae3b5a1700 1 -- 172.21.15.150:0/1512655489 wait complete. 2024-05-10T04:38:22.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:38:22.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:22 smithi161 ceph-mon[46912]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 960 B/s wr, 3 op/s 2024-05-10T04:38:22.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:22 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4173921927' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:38:22.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:22 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1942284407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:22.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 -- 172.21.15.150:0/3589489821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c01078a0 msgr2=0x7f02c0109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3589489821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c01078a0 0x7f02c0109d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f02b800b0a0 tx=0x7f02b800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 -- 172.21.15.150:0/3589489821 shutdown_connections 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3589489821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c01078a0 0x7f02c0109d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3589489821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0069050 0x7f02c0104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3589489821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0104d30 0x7f02c0107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 -- 172.21.15.150:0/3589489821 >> 172.21.15.150:0/3589489821 conn(0x7f02c00fc0d0 msgr2=0x7f02c00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 -- 172.21.15.150:0/3589489821 shutdown_connections 2024-05-10T04:38:22.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.915+0000 7f02c4e27700 1 -- 172.21.15.150:0/3589489821 wait complete. 2024-05-10T04:38:22.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 Processor -- start 2024-05-10T04:38:22.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 -- start start 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0069050 0x7f02c0101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0104d30 0x7f02c0100660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c01078a0 0x7f02c0100ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0101280 con 0x7f02c0069050 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c01013f0 con 0x7f02c0104d30 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02c4e27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c01a20a0 con 0x7f02c01078a0 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0069050 0x7f02c0101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0069050 0x7f02c0101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57546/0 (socket says 172.21.15.150:57546) 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 -- 172.21.15.150:0/3008277991 learned_addr learned my addr 172.21.15.150:0/3008277991 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02bdd9b700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0104d30 0x7f02c0100660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02bed9d700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c01078a0 0x7f02c0100ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 -- 172.21.15.150:0/3008277991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c01078a0 msgr2=0x7f02c0100ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c01078a0 0x7f02c0100ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 -- 172.21.15.150:0/3008277991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0104d30 msgr2=0x7f02c0100660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0104d30 0x7f02c0100660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02be59c700 1 -- 172.21.15.150:0/3008277991 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b8009d00 con 0x7f02c0069050 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.916+0000 7f02bed9d700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c01078a0 0x7f02c0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.917+0000 7f02be59c700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0069050 0x7f02c0101fc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f02b000d980 tx=0x7f02b000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.917+0000 7f02af7fe700 1 -- 172.21.15.150:0/3008277991 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0009b40 con 0x7f02c0069050 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.917+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c01a2300 con 0x7f02c0069050 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.917+0000 7f02af7fe700 1 -- 172.21.15.150:0/3008277991 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b000de60 con 0x7f02c0069050 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.917+0000 7f02af7fe700 1 -- 172.21.15.150:0/3008277991 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000f620 con 0x7f02c0069050 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.917+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c01a27e0 con 0x7f02c0069050 2024-05-10T04:38:22.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.918+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c00fdc80 con 0x7f02c0069050 2024-05-10T04:38:22.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.920+0000 7f02af7fe700 1 -- 172.21.15.150:0/3008277991 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f02b0010460 con 0x7f02c0069050 2024-05-10T04:38:22.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.920+0000 7f02af7fe700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02a80a11c0 0x7f02a80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:22.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.920+0000 7f02bdd9b700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02a80a11c0 0x7f02a80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:22.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.920+0000 7f02af7fe700 1 -- 172.21.15.150:0/3008277991 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f02b00cc690 con 0x7f02c0069050 2024-05-10T04:38:22.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.921+0000 7f02bdd9b700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02a80a11c0 0x7f02a80a36d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f02b4005950 tx=0x7f02b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:22.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:22.924+0000 7f02af7fe700 1 -- 172.21.15.150:0/3008277991 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f02b0095c30 con 0x7f02c0069050 2024-05-10T04:38:23.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.121+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02c010d3a0 con 0x7f02c0069050 2024-05-10T04:38:23.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.121+0000 7f02af7fe700 1 -- 172.21.15.150:0/3008277991 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02b0020020 con 0x7f02c0069050 2024-05-10T04:38:23.122 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02a80a11c0 msgr2=0x7f02a80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02a80a11c0 0x7f02a80a36d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f02b4005950 tx=0x7f02b40058e0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0069050 msgr2=0x7f02c0101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0069050 0x7f02c0101fc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f02b000d980 tx=0x7f02b000dd40 comp rx=0 tx=0).stop 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 shutdown_connections 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02a80a11c0 0x7f02a80a36d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c01078a0 0x7f02c0100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0069050 0x7f02c0101fc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 --2- 172.21.15.150:0/3008277991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0104d30 0x7f02c0100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 >> 172.21.15.150:0/3008277991 conn(0x7f02c00fc0d0 msgr2=0x7f02c0103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 shutdown_connections 2024-05-10T04:38:23.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.123+0000 7f02c4e27700 1 -- 172.21.15.150:0/3008277991 wait complete. 2024-05-10T04:38:23.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/363246010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8100ac0 msgr2=0x7f8dd8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/363246010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8100ac0 0x7f8dd8100ed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009990 tx=0x7f8dcc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/363246010 shutdown_connections 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/363246010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd8102580 0x7f8dd8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/363246010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8100ac0 0x7f8dd8100ed0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/363246010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8101be0 0x7f8dd8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/363246010 >> 172.21.15.150:0/363246010 conn(0x7f8dd80fc0d0 msgr2=0x7f8dd80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/363246010 shutdown_connections 2024-05-10T04:38:23.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.271+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/363246010 wait complete. 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 Processor -- start 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 -- start start 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd8100ac0 0x7f8dd8068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8101be0 0x7f8dd819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8102580 0x7f8dd819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd819fa50 con 0x7f8dd8101be0 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd819fbc0 con 0x7f8dd8100ac0 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8ddcd6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd819fd80 con 0x7f8dd8102580 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8dd5d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8101be0 0x7f8dd819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8dd5d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8101be0 0x7f8dd819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57560/0 (socket says 172.21.15.150:57560) 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.272+0000 7f8dd5d9b700 1 -- 172.21.15.150:0/1286860411 learned_addr learned my addr 172.21.15.150:0/1286860411 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd6d9d700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8102580 0x7f8dd819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd659c700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd8100ac0 0x7f8dd8068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd5d9b700 1 -- 172.21.15.150:0/1286860411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8102580 msgr2=0x7f8dd819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd5d9b700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8102580 0x7f8dd819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd5d9b700 1 -- 172.21.15.150:0/1286860411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd8100ac0 msgr2=0x7f8dd8068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd5d9b700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd8100ac0 0x7f8dd8068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd5d9b700 1 -- 172.21.15.150:0/1286860411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc0095f0 con 0x7f8dd8101be0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dd5d9b700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8101be0 0x7f8dd819ccc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800d950 tx=0x7f8dc800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dc77fe700 1 -- 172.21.15.150:0/1286860411 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8009940 con 0x7f8dd8101be0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dc77fe700 1 -- 172.21.15.150:0/1286860411 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dc8010460 con 0x7f8dd8101be0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8dc77fe700 1 -- 172.21.15.150:0/1286860411 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800f620 con 0x7f8dd8101be0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd819ffa0 con 0x7f8dd8101be0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.273+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd81a03d0 con 0x7f8dd8101be0 2024-05-10T04:38:23.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.274+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd8107cd0 con 0x7f8dd8101be0 2024-05-10T04:38:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.275+0000 7f8dc77fe700 1 -- 172.21.15.150:0/1286860411 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8dc80108d0 con 0x7f8dd8101be0 2024-05-10T04:38:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.276+0000 7f8dc77fe700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8dc00a11c0 0x7f8dc00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.276+0000 7f8dc77fe700 1 -- 172.21.15.150:0/1286860411 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8dc80cc940 con 0x7f8dd8101be0 2024-05-10T04:38:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.276+0000 7f8dd659c700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8dc00a11c0 0x7f8dc00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.276+0000 7f8dd659c700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8dc00a11c0 0x7f8dc00a36d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc006010 tx=0x7f8dcc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:23.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.280+0000 7f8dc77fe700 1 -- 172.21.15.150:0/1286860411 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8dc8095ee0 con 0x7f8dd8101be0 2024-05-10T04:38:23.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:23 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2891455806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:23 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1512655489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:23 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3008277991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.480+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8dd8069470 con 0x7f8dd8101be0 2024-05-10T04:38:23.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.480+0000 7f8dc77fe700 1 -- 172.21.15.150:0/1286860411 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8dc80160a0 con 0x7f8dd8101be0 2024-05-10T04:38:23.482 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:38:23.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8dc00a11c0 msgr2=0x7f8dc00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8dc00a11c0 0x7f8dc00a36d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc006010 tx=0x7f8dcc01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:23.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8101be0 msgr2=0x7f8dd819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8101be0 0x7f8dd819ccc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800d950 tx=0x7f8dc800dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 shutdown_connections 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8dc00a11c0 0x7f8dc00a36d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8102580 0x7f8dd819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd8100ac0 0x7f8dd8068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 --2- 172.21.15.150:0/1286860411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dd8101be0 0x7f8dd819ccc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 >> 172.21.15.150:0/1286860411 conn(0x7f8dd80fc0d0 msgr2=0x7f8dd8104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 shutdown_connections 2024-05-10T04:38:23.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.483+0000 7f8ddcd6e700 1 -- 172.21.15.150:0/1286860411 wait complete. 2024-05-10T04:38:23.497 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T04:38:23.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.628+0000 7f9d32751700 1 -- 172.21.15.150:0/2202336764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 msgr2=0x7f9d2c108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.628+0000 7f9d32751700 1 --2- 172.21.15.150:0/2202336764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c108ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400b0a0 tx=0x7f9d2400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 -- 172.21.15.150:0/2202336764 shutdown_connections 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 --2- 172.21.15.150:0/2202336764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c108ee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 --2- 172.21.15.150:0/2202336764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c100b70 0x7f9d2c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 --2- 172.21.15.150:0/2202336764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2c101c90 0x7f9d2c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 -- 172.21.15.150:0/2202336764 >> 172.21.15.150:0/2202336764 conn(0x7f9d2c0fc180 msgr2=0x7f9d2c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 -- 172.21.15.150:0/2202336764 shutdown_connections 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 -- 172.21.15.150:0/2202336764 wait complete. 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 Processor -- start 2024-05-10T04:38:23.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.629+0000 7f9d32751700 1 -- start start 2024-05-10T04:38:23.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d32751700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2c100b70 0x7f9d2c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d32751700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c101c90 0x7f9d2c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d32751700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d32751700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c19fb30 con 0x7f9d2c102630 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d32751700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c19fca0 con 0x7f9d2c100b70 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d32751700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c19fe60 con 0x7f9d2c101c90 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c19f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57578/0 (socket says 172.21.15.150:57578) 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 -- 172.21.15.150:0/3983473934 learned_addr learned my addr 172.21.15.150:0/3983473934 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d2b7fe700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c101c90 0x7f9d2c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d2bfff700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2c100b70 0x7f9d2c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 -- 172.21.15.150:0/3983473934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c101c90 msgr2=0x7f9d2c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c101c90 0x7f9d2c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 -- 172.21.15.150:0/3983473934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2c100b70 msgr2=0x7f9d2c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2c100b70 0x7f9d2c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 -- 172.21.15.150:0/3983473934 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24009d00 con 0x7f9d2c102630 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d2bfff700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2c100b70 0x7f9d2c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d30cee700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c19f520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d24006070 tx=0x7f9d24009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d297fa700 1 -- 172.21.15.150:0/3983473934 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400e040 con 0x7f9d2c102630 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c1a00e0 con 0x7f9d2c102630 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d297fa700 1 -- 172.21.15.150:0/3983473934 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d24004d60 con 0x7f9d2c102630 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.630+0000 7f9d297fa700 1 -- 172.21.15.150:0/3983473934 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2401d480 con 0x7f9d2c102630 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.631+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c1a05b0 con 0x7f9d2c102630 2024-05-10T04:38:23.635 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.631+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c0678b0 con 0x7f9d2c102630 2024-05-10T04:38:23.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.633+0000 7f9d297fa700 1 -- 172.21.15.150:0/3983473934 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9d24003a00 con 0x7f9d2c102630 2024-05-10T04:38:23.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.633+0000 7f9d297fa700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d140a11c0 0x7f9d140a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:23.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.633+0000 7f9d2bfff700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d140a11c0 0x7f9d140a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:23.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.633+0000 7f9d297fa700 1 -- 172.21.15.150:0/3983473934 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9d240ce570 con 0x7f9d2c102630 2024-05-10T04:38:23.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.634+0000 7f9d2bfff700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d140a11c0 0x7f9d140a36d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c005950 tx=0x7f9d1c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:23.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.637+0000 7f9d297fa700 1 -- 172.21.15.150:0/3983473934 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d24097b40 con 0x7f9d2c102630 2024-05-10T04:38:23.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:23 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2891455806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:23 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1512655489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:23 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3008277991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:23 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2891455806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:23 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1512655489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:23 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3008277991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:38:23.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.849+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d2c10c310 con 0x7f9d2c102630 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.849+0000 7f9d297fa700 1 -- 172.21.15.150:0/3983473934 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+599 (secure 0 0 0) 0x7f9d2409b160 con 0x7f9d2c102630 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/1928870080 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3285068402 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/881426482 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/618215791 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/2517418767 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3236490116 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/881426482 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/2970915543 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/2970915543 2024-05-11T04:32:48.043455+0000 2024-05-10T04:38:23.851 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3890124032 2024-05-11T04:33:11.238386+0000 2024-05-10T04:38:23.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d140a11c0 msgr2=0x7f9d140a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d140a11c0 0x7f9d140a36d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c005950 tx=0x7f9d1c0058e0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 msgr2=0x7f9d2c19f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:23.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c19f520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d24006070 tx=0x7f9d24009440 comp rx=0 tx=0).stop 2024-05-10T04:38:23.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 shutdown_connections 2024-05-10T04:38:23.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c102630 0x7f9d2c19f520 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d2c100b70 0x7f9d2c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d140a11c0 0x7f9d140a36d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 --2- 172.21.15.150:0/3983473934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d2c101c90 0x7f9d2c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:23.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 >> 172.21.15.150:0/3983473934 conn(0x7f9d2c0fc180 msgr2=0x7f9d2c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:23.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 shutdown_connections 2024-05-10T04:38:23.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:23.852+0000 7f9d32751700 1 -- 172.21.15.150:0/3983473934 wait complete. 2024-05-10T04:38:23.854 INFO:teuthology.orchestra.run.smithi150.stderr:listed 10 entries 2024-05-10T04:38:23.866 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi150.front.sepia.ceph.com... 2024-05-10T04:38:23.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:23.866 DEBUG:teuthology.orchestra.run.smithi150:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-10T04:38:23.903 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi150.stderr:ceph-fuse[95567]: fuse finished with error 0 and tester_r 0 2024-05-10T04:38:23.904 INFO:teuthology.orchestra.run:waiting for 300 2024-05-10T04:38:24.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:24 smithi181 ceph-mon[47314]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 B/s wr, 0 op/s 2024-05-10T04:38:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:24 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1286860411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:38:24.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:24 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3983473934' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:38:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:24 smithi150 ceph-mon[42404]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 B/s wr, 0 op/s 2024-05-10T04:38:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:24 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1286860411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:38:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:24 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3983473934' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:38:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:24 smithi161 ceph-mon[46912]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 B/s wr, 0 op/s 2024-05-10T04:38:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:24 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1286860411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:38:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:24 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3983473934' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:38:26.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:26 smithi181 ceph-mon[47314]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 0 op/s 2024-05-10T04:38:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:26 smithi150 ceph-mon[42404]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 0 op/s 2024-05-10T04:38:26.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:26 smithi161 ceph-mon[46912]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 0 op/s 2024-05-10T04:38:28.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:28 smithi181 ceph-mon[47314]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:38:28.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:28 smithi150 ceph-mon[42404]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:38:28.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:28 smithi161 ceph-mon[46912]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-10T04:38:30.006 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:38:30.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:38:30.006 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-10T04:38:30.042 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:30.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 -- 172.21.15.150:0/215086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c0ffc20 msgr2=0x7fdf2c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 --2- 172.21.15.150:0/215086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c0ffc20 0x7fdf2c100030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009990 tx=0x7fdf1c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 -- 172.21.15.150:0/215086046 shutdown_connections 2024-05-10T04:38:30.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 --2- 172.21.15.150:0/215086046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf2c100570 0x7fdf2c10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 --2- 172.21.15.150:0/215086046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1031e0 0x7fdf2c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 --2- 172.21.15.150:0/215086046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c0ffc20 0x7fdf2c100030 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 -- 172.21.15.150:0/215086046 >> 172.21.15.150:0/215086046 conn(0x7fdf2c0fb8f0 msgr2=0x7fdf2c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 -- 172.21.15.150:0/215086046 shutdown_connections 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.225+0000 7fdf322dd700 1 -- 172.21.15.150:0/215086046 wait complete. 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 Processor -- start 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 -- start start 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c0ffc20 0x7fdf2c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf2c100570 0x7fdf2c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c1031e0 0x7fdf2c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c19fa70 con 0x7fdf2c1031e0 2024-05-10T04:38:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c19fbb0 con 0x7fdf2c100570 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf322dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c19fd40 con 0x7fdf2c0ffc20 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf3087a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c1031e0 0x7fdf2c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf3087a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c1031e0 0x7fdf2c19f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50314/0 (socket says 172.21.15.150:50314) 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.226+0000 7fdf3087a700 1 -- 172.21.15.150:0/2589123060 learned_addr learned my addr 172.21.15.150:0/2589123060 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf2b7fe700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf2c100570 0x7fdf2c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf2bfff700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c0ffc20 0x7fdf2c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf3087a700 1 -- 172.21.15.150:0/2589123060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c0ffc20 msgr2=0x7fdf2c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf3087a700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c0ffc20 0x7fdf2c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf3087a700 1 -- 172.21.15.150:0/2589123060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf2c100570 msgr2=0x7fdf2c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf3087a700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf2c100570 0x7fdf2c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf3087a700 1 -- 172.21.15.150:0/2589123060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0095f0 con 0x7fdf2c1031e0 2024-05-10T04:38:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf2bfff700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c0ffc20 0x7fdf2c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf2b7fe700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf2c100570 0x7fdf2c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf3087a700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c1031e0 0x7fdf2c19f4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007da0 tx=0x7fdf2400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf297fa700 1 -- 172.21.15.150:0/2589123060 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400ad00 con 0x7fdf2c1031e0 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf297fa700 1 -- 172.21.15.150:0/2589123060 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf2400ae60 con 0x7fdf2c1031e0 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c19ffc0 con 0x7fdf2c1031e0 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf297fa700 1 -- 172.21.15.150:0/2589123060 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24011780 con 0x7fdf2c1031e0 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.227+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c1a0440 con 0x7fdf2c1031e0 2024-05-10T04:38:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.228+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c109030 con 0x7fdf2c1031e0 2024-05-10T04:38:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.230+0000 7fdf297fa700 1 -- 172.21.15.150:0/2589123060 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fdf24012a70 con 0x7fdf2c1031e0 2024-05-10T04:38:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.230+0000 7fdf297fa700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdf180a1120 0x7fdf180a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.231+0000 7fdf2bfff700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdf180a1120 0x7fdf180a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.231+0000 7fdf297fa700 1 -- 172.21.15.150:0/2589123060 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdf240cd280 con 0x7fdf2c1031e0 2024-05-10T04:38:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.231+0000 7fdf2bfff700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdf180a1120 0x7fdf180a3630 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005d90 tx=0x7fdf14005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:30.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.234+0000 7fdf297fa700 1 -- 172.21.15.150:0/2589123060 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdf240967c0 con 0x7fdf2c1031e0 2024-05-10T04:38:30.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.434+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2c10d3d0 con 0x7fdf2c1031e0 2024-05-10T04:38:30.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.436+0000 7fdf297fa700 1 -- 172.21.15.150:0/2589123060 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7fdf24099de0 con 0x7fdf2c1031e0 2024-05-10T04:38:30.437 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:30.437 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdf180a1120 msgr2=0x7fdf180a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdf180a1120 0x7fdf180a3630 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005d90 tx=0x7fdf14005e60 comp rx=0 tx=0).stop 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c1031e0 msgr2=0x7fdf2c19f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c1031e0 0x7fdf2c19f4a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007da0 tx=0x7fdf2400a340 comp rx=0 tx=0).stop 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 shutdown_connections 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf2c1031e0 0x7fdf2c19f4a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdf180a1120 0x7fdf180a3630 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c0ffc20 0x7fdf2c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 --2- 172.21.15.150:0/2589123060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf2c100570 0x7fdf2c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 >> 172.21.15.150:0/2589123060 conn(0x7fdf2c0fb8f0 msgr2=0x7fdf2c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 shutdown_connections 2024-05-10T04:38:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.438+0000 7fdf322dd700 1 -- 172.21.15.150:0/2589123060 wait complete. 2024-05-10T04:38:30.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:30.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.584+0000 7f0c20339700 1 -- 172.21.15.150:0/1052230884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18100ac0 msgr2=0x7f0c18100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.584+0000 7f0c20339700 1 --2- 172.21.15.150:0/1052230884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18100ac0 0x7f0c18100ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009990 tx=0x7f0c14009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 -- 172.21.15.150:0/1052230884 shutdown_connections 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 --2- 172.21.15.150:0/1052230884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c18102580 0x7f0c18108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 --2- 172.21.15.150:0/1052230884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18100ac0 0x7f0c18100ed0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 --2- 172.21.15.150:0/1052230884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c18101be0 0x7f0c18101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 -- 172.21.15.150:0/1052230884 >> 172.21.15.150:0/1052230884 conn(0x7f0c180fc0d0 msgr2=0x7f0c180fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 -- 172.21.15.150:0/1052230884 shutdown_connections 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 -- 172.21.15.150:0/1052230884 wait complete. 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 Processor -- start 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 -- start start 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.585+0000 7f0c20339700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c18100ac0 0x7f0c18068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c20339700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c18101be0 0x7f0c1819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c20339700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18102580 0x7f0c1819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c20339700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1819fb70 con 0x7f0c18102580 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c20339700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1819fce0 con 0x7f0c18101be0 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c20339700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1819fea0 con 0x7f0c18100ac0 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18102580 0x7f0c1819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18102580 0x7f0c1819f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50330/0 (socket says 172.21.15.150:50330) 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 -- 172.21.15.150:0/2570400780 learned_addr learned my addr 172.21.15.150:0/2570400780 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e0d5700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c18100ac0 0x7f0c18068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1d8d4700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c18101be0 0x7f0c1819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 -- 172.21.15.150:0/2570400780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c18100ac0 msgr2=0x7f0c18068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c18100ac0 0x7f0c18068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 -- 172.21.15.150:0/2570400780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c18101be0 msgr2=0x7f0c1819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c18101be0 0x7f0c1819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 -- 172.21.15.150:0/2570400780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140095f0 con 0x7f0c18102580 2024-05-10T04:38:30.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1d8d4700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c18101be0 0x7f0c1819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:30.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c1e8d6700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18102580 0x7f0c1819f420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c10007ee0 tx=0x7f0c1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:30.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c0f7fe700 1 -- 172.21.15.150:0/2570400780 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000ddd0 con 0x7f0c18102580 2024-05-10T04:38:30.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c181a0120 con 0x7f0c18102580 2024-05-10T04:38:30.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c0f7fe700 1 -- 172.21.15.150:0/2570400780 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c1000f040 con 0x7f0c18102580 2024-05-10T04:38:30.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c0f7fe700 1 -- 172.21.15.150:0/2570400780 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100048e0 con 0x7f0c18102580 2024-05-10T04:38:30.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.586+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c181a05e0 con 0x7f0c18102580 2024-05-10T04:38:30.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.587+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c18107cd0 con 0x7f0c18102580 2024-05-10T04:38:30.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.589+0000 7f0c0f7fe700 1 -- 172.21.15.150:0/2570400780 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0c10003a00 con 0x7f0c18102580 2024-05-10T04:38:30.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.589+0000 7f0c0f7fe700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c040a1120 0x7f0c040a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.589+0000 7f0c1e0d5700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c040a1120 0x7f0c040a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.589+0000 7f0c0f7fe700 1 -- 172.21.15.150:0/2570400780 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0c100cc310 con 0x7f0c18102580 2024-05-10T04:38:30.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.590+0000 7f0c1e0d5700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c040a1120 0x7f0c040a3630 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006010 tx=0x7f0c1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:30.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.594+0000 7f0c0f7fe700 1 -- 172.21.15.150:0/2570400780 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0c100958b0 con 0x7f0c18102580 2024-05-10T04:38:30.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:30 smithi181 ceph-mon[47314]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:38:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:30 smithi150 ceph-mon[42404]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:38:30.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.791+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c18069470 con 0x7f0c18102580 2024-05-10T04:38:30.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.793+0000 7f0c0f7fe700 1 -- 172.21.15.150:0/2570400780 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f0c100096c0 con 0x7f0c18102580 2024-05-10T04:38:30.794 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:30.794 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:30.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c040a1120 msgr2=0x7f0c040a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c040a1120 0x7f0c040a3630 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006010 tx=0x7f0c1401a040 comp rx=0 tx=0).stop 2024-05-10T04:38:30.796 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18102580 msgr2=0x7f0c1819f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18102580 0x7f0c1819f420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c10007ee0 tx=0x7f0c1000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 shutdown_connections 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c18102580 0x7f0c1819f420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c040a1120 0x7f0c040a3630 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c18100ac0 0x7f0c18068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 --2- 172.21.15.150:0/2570400780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c18101be0 0x7f0c1819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 >> 172.21.15.150:0/2570400780 conn(0x7f0c180fc0d0 msgr2=0x7f0c18104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 shutdown_connections 2024-05-10T04:38:30.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.795+0000 7f0c20339700 1 -- 172.21.15.150:0/2570400780 wait complete. 2024-05-10T04:38:30.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-10T04:38:30.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:30 smithi161 ceph-mon[46912]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:38:30.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 -- 172.21.15.150:0/2559416791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec102590 msgr2=0x7f9cec108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/2559416791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec102590 0x7f9cec108e40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b0a0 tx=0x7f9ce400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 -- 172.21.15.150:0/2559416791 shutdown_connections 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/2559416791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec102590 0x7f9cec108e40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/2559416791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec100ad0 0x7f9cec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/2559416791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cec101bf0 0x7f9cec102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 -- 172.21.15.150:0/2559416791 >> 172.21.15.150:0/2559416791 conn(0x7f9cec0fc0c0 msgr2=0x7f9cec0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 -- 172.21.15.150:0/2559416791 shutdown_connections 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 -- 172.21.15.150:0/2559416791 wait complete. 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.944+0000 7f9cf17b8700 1 Processor -- start 2024-05-10T04:38:30.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 -- start start 2024-05-10T04:38:30.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec100ad0 0x7f9cec06a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cec101bf0 0x7f9cec19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec102590 0x7f9cec19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec19fac0 con 0x7f9cec100ad0 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec19fc30 con 0x7f9cec101bf0 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec19fdf0 con 0x7f9cec102590 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec100ad0 0x7f9cec06a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec100ad0 0x7f9cec06a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50358/0 (socket says 172.21.15.150:50358) 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 -- 172.21.15.150:0/3662953769 learned_addr learned my addr 172.21.15.150:0/3662953769 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cea7fc700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cec101bf0 0x7f9cec19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceb7fe700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec102590 0x7f9cec19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 -- 172.21.15.150:0/3662953769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec102590 msgr2=0x7f9cec19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec102590 0x7f9cec19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 -- 172.21.15.150:0/3662953769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cec101bf0 msgr2=0x7f9cec19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cec101bf0 0x7f9cec19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 -- 172.21.15.150:0/3662953769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4009d00 con 0x7f9cec100ad0 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9ceaffd700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec100ad0 0x7f9cec06a240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00d950 tx=0x7f9cdc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:30.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cd3fff700 1 -- 172.21.15.150:0/3662953769 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004220 con 0x7f9cec100ad0 2024-05-10T04:38:30.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec1a00d0 con 0x7f9cec100ad0 2024-05-10T04:38:30.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cd3fff700 1 -- 172.21.15.150:0/3662953769 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc004380 con 0x7f9cec100ad0 2024-05-10T04:38:30.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.945+0000 7f9cd3fff700 1 -- 172.21.15.150:0/3662953769 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc003bf0 con 0x7f9cec100ad0 2024-05-10T04:38:30.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.946+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec1a05e0 con 0x7f9cec100ad0 2024-05-10T04:38:30.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.946+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec107ce0 con 0x7f9cec100ad0 2024-05-10T04:38:30.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.948+0000 7f9cd3fff700 1 -- 172.21.15.150:0/3662953769 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9cdc003d50 con 0x7f9cec100ad0 2024-05-10T04:38:30.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.948+0000 7f9cd3fff700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cd40a11c0 0x7f9cd40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:30.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.948+0000 7f9cea7fc700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cd40a11c0 0x7f9cd40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:30.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.948+0000 7f9cd3fff700 1 -- 172.21.15.150:0/3662953769 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9cdc0ccd70 con 0x7f9cec100ad0 2024-05-10T04:38:30.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.949+0000 7f9cea7fc700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cd40a11c0 0x7f9cd40a36d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9cec069be0 tx=0x7f9ce000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:30.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:30.952+0000 7f9cd3fff700 1 -- 172.21.15.150:0/3662953769 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9cdc096310 con 0x7f9cec100ad0 2024-05-10T04:38:31.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.195+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9cec0678b0 con 0x7f9cec100ad0 2024-05-10T04:38:31.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.389+0000 7f9cd3fff700 1 -- 172.21.15.150:0/3662953769 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9cdc017020 con 0x7f9cec100ad0 2024-05-10T04:38:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.391+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cd40a11c0 msgr2=0x7f9cd40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.391+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cd40a11c0 0x7f9cd40a36d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9cec069be0 tx=0x7f9ce000b410 comp rx=0 tx=0).stop 2024-05-10T04:38:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.391+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec100ad0 msgr2=0x7f9cec06a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.391+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec100ad0 0x7f9cec06a240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00d950 tx=0x7f9cdc00dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 shutdown_connections 2024-05-10T04:38:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cd40a11c0 0x7f9cd40a36d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cec102590 0x7f9cec19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cec100ad0 0x7f9cec06a240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 --2- 172.21.15.150:0/3662953769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cec101bf0 0x7f9cec19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 >> 172.21.15.150:0/3662953769 conn(0x7f9cec0fc0c0 msgr2=0x7f9cec104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 shutdown_connections 2024-05-10T04:38:31.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.392+0000 7f9cf17b8700 1 -- 172.21.15.150:0/3662953769 wait complete. 2024-05-10T04:38:31.406 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-10T04:38:31.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-10T04:38:31.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 -- 172.21.15.150:0/1952343762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 msgr2=0x7fd080109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 --2- 172.21.15.150:0/1952343762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 0x7fd080109050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b0a0 tx=0x7fd07800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 -- 172.21.15.150:0/1952343762 shutdown_connections 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 --2- 172.21.15.150:0/1952343762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 0x7fd080109050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 --2- 172.21.15.150:0/1952343762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 0x7fd0801014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 --2- 172.21.15.150:0/1952343762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd080101a70 0x7fd080101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 -- 172.21.15.150:0/1952343762 >> 172.21.15.150:0/1952343762 conn(0x7fd0800fb8f0 msgr2=0x7fd0800fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 -- 172.21.15.150:0/1952343762 shutdown_connections 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.541+0000 7fd084e57700 1 -- 172.21.15.150:0/1952343762 wait complete. 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 Processor -- start 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 -- start start 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 0x7fd080104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:31.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd080101a70 0x7fd080103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 0x7fd080103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080103d60 con 0x7fd0801025d0 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080103ed0 con 0x7fd080101090 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd084e57700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0801a20b0 con 0x7fd080101a70 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 0x7fd080103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 0x7fd080104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 0x7fd080104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:36462/0 (socket says 172.21.15.150:36462) 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07e59c700 1 -- 172.21.15.150:0/3791210687 learned_addr learned my addr 172.21.15.150:0/3791210687 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07dd9b700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd080101a70 0x7fd080103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07ed9d700 1 -- 172.21.15.150:0/3791210687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd080101a70 msgr2=0x7fd080103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07ed9d700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd080101a70 0x7fd080103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07ed9d700 1 -- 172.21.15.150:0/3791210687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 msgr2=0x7fd080104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07ed9d700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 0x7fd080104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:31.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07ed9d700 1 -- 172.21.15.150:0/3791210687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078009d00 con 0x7fd0801025d0 2024-05-10T04:38:31.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.542+0000 7fd07e59c700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 0x7fd080104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:38:31.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.543+0000 7fd07ed9d700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 0x7fd080103680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd078007b90 tx=0x7fd0780078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:31.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.543+0000 7fd06f7fe700 1 -- 172.21.15.150:0/3791210687 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07800e040 con 0x7fd0801025d0 2024-05-10T04:38:31.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.543+0000 7fd06f7fe700 1 -- 172.21.15.150:0/3791210687 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd078003c00 con 0x7fd0801025d0 2024-05-10T04:38:31.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.543+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0801a22b0 con 0x7fd0801025d0 2024-05-10T04:38:31.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.543+0000 7fd06f7fe700 1 -- 172.21.15.150:0/3791210687 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07801b6e0 con 0x7fd0801025d0 2024-05-10T04:38:31.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.543+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0801a26d0 con 0x7fd0801025d0 2024-05-10T04:38:31.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.544+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0800678b0 con 0x7fd0801025d0 2024-05-10T04:38:31.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.545+0000 7fd06f7fe700 1 -- 172.21.15.150:0/3791210687 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd07801d470 con 0x7fd0801025d0 2024-05-10T04:38:31.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.545+0000 7fd06f7fe700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd0680a11c0 0x7fd0680a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:31.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.546+0000 7fd06f7fe700 1 -- 172.21.15.150:0/3791210687 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd0780cdc70 con 0x7fd0801025d0 2024-05-10T04:38:31.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.546+0000 7fd07e59c700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd0680a11c0 0x7fd0680a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:31.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.546+0000 7fd07e59c700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd0680a11c0 0x7fd0680a36d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd070011550 tx=0x7fd070009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:31.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.549+0000 7fd06f7fe700 1 -- 172.21.15.150:0/3791210687 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd0780972a0 con 0x7fd0801025d0 2024-05-10T04:38:31.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2589123060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:31.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2570400780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:31.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3662953769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T04:38:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2589123060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2570400780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3662953769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T04:38:31.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:31.793+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fd0801a29c0 con 0x7fd0801025d0 2024-05-10T04:38:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2589123060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2570400780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3662953769' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-10T04:38:32.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.396+0000 7fd06f7fe700 1 -- 172.21.15.150:0/3791210687 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fd078003d70 con 0x7fd0801025d0 2024-05-10T04:38:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd0680a11c0 msgr2=0x7fd0680a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd0680a11c0 0x7fd0680a36d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd070011550 tx=0x7fd070009260 comp rx=0 tx=0).stop 2024-05-10T04:38:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 msgr2=0x7fd080103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:32.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 0x7fd080103680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd078007b90 tx=0x7fd0780078e0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 shutdown_connections 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0801025d0 0x7fd080103680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd080101090 0x7fd080104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd0680a11c0 0x7fd0680a36d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 --2- 172.21.15.150:0/3791210687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd080101a70 0x7fd080103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 >> 172.21.15.150:0/3791210687 conn(0x7fd0800fb8f0 msgr2=0x7fd0800fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 shutdown_connections 2024-05-10T04:38:32.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.399+0000 7fd084e57700 1 -- 172.21.15.150:0/3791210687 wait complete. 2024-05-10T04:38:32.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-10T04:38:32.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 -- 172.21.15.150:0/3691999593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 msgr2=0x7fe13c109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3691999593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c109250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe13400b0a0 tx=0x7fe13400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 -- 172.21.15.150:0/3691999593 shutdown_connections 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3691999593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c109250 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3691999593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c101110 0x7fe13c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3691999593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe13c102000 0x7fe13c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 -- 172.21.15.150:0/3691999593 >> 172.21.15.150:0/3691999593 conn(0x7fe13c0fb8f0 msgr2=0x7fe13c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 -- 172.21.15.150:0/3691999593 shutdown_connections 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.556+0000 7fe1437e9700 1 -- 172.21.15.150:0/3691999593 wait complete. 2024-05-10T04:38:32.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 Processor -- start 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 -- start start 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c101110 0x7fe13c1007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe13c102000 0x7fe13c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c19fcc0 con 0x7fe13c102950 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c19fe30 con 0x7fe13c102000 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe1437e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c19fff0 con 0x7fe13c101110 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50386/0 (socket says 172.21.15.150:50386) 2024-05-10T04:38:32.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 -- 172.21.15.150:0/3813050781 learned_addr learned my addr 172.21.15.150:0/3813050781 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe140d84700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe13c102000 0x7fe13c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141585700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c101110 0x7fe13c1007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 -- 172.21.15.150:0/3813050781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c101110 msgr2=0x7fe13c1007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c101110 0x7fe13c1007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 -- 172.21.15.150:0/3813050781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe13c102000 msgr2=0x7fe13c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe13c102000 0x7fe13c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.557+0000 7fe141d86700 1 -- 172.21.15.150:0/3813050781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe134009d00 con 0x7fe13c102950 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.558+0000 7fe141d86700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c19f5c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe134006070 tx=0x7fe13400bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.558+0000 7fe1327fc700 1 -- 172.21.15.150:0/3813050781 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13400e040 con 0x7fe13c102950 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.558+0000 7fe1327fc700 1 -- 172.21.15.150:0/3813050781 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe13400bde0 con 0x7fe13c102950 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.558+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c1a0270 con 0x7fe13c102950 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.558+0000 7fe1327fc700 1 -- 172.21.15.150:0/3813050781 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13401b810 con 0x7fe13c102950 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.558+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c1a06f0 con 0x7fe13c102950 2024-05-10T04:38:32.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.559+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c1080f0 con 0x7fe13c102950 2024-05-10T04:38:32.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.561+0000 7fe1327fc700 1 -- 172.21.15.150:0/3813050781 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe13401da40 con 0x7fe13c102950 2024-05-10T04:38:32.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.561+0000 7fe1327fc700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1280a11c0 0x7fe1280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:32.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.562+0000 7fe141585700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1280a11c0 0x7fe1280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:32.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.562+0000 7fe1327fc700 1 -- 172.21.15.150:0/3813050781 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe1340ce740 con 0x7fe13c102950 2024-05-10T04:38:32.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.562+0000 7fe141585700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1280a11c0 0x7fe1280a36d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe13c101f90 tx=0x7fe12c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:32.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.565+0000 7fe1327fc700 1 -- 172.21.15.150:0/3813050781 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe134097d70 con 0x7fe13c102950 2024-05-10T04:38:32.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:32 smithi181 ceph-mon[47314]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:38:32.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:32 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3662953769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T04:38:32.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:32 smithi181 ceph-mon[47314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:32.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:32 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3791210687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T04:38:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:32 smithi150 ceph-mon[42404]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:38:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3662953769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T04:38:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:32 smithi150 ceph-mon[42404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3791210687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T04:38:32.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.813+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe13c0681a0 con 0x7fe13c102950 2024-05-10T04:38:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:32 smithi161 ceph-mon[46912]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-10T04:38:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:32 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3662953769' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-10T04:38:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:32 smithi161 ceph-mon[46912]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:32 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3791210687' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-10T04:38:32.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.935+0000 7fe1327fc700 1 -- 172.21.15.150:0/3813050781 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe13409b390 con 0x7fe13c102950 2024-05-10T04:38:32.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1280a11c0 msgr2=0x7fe1280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:32.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1280a11c0 0x7fe1280a36d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe13c101f90 tx=0x7fe12c009260 comp rx=0 tx=0).stop 2024-05-10T04:38:32.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 msgr2=0x7fe13c19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:32.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c19f5c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe134006070 tx=0x7fe13400bb00 comp rx=0 tx=0).stop 2024-05-10T04:38:32.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 shutdown_connections 2024-05-10T04:38:32.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe13c102950 0x7fe13c19f5c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1280a11c0 0x7fe1280a36d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe13c101110 0x7fe13c1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 --2- 172.21.15.150:0/3813050781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe13c102000 0x7fe13c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:32.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 >> 172.21.15.150:0/3813050781 conn(0x7fe13c0fb8f0 msgr2=0x7fe13c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:32.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 shutdown_connections 2024-05-10T04:38:32.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:32.938+0000 7fe1437e9700 1 -- 172.21.15.150:0/3813050781 wait complete. 2024-05-10T04:38:32.956 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 -- 172.21.15.150:0/2562717477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 msgr2=0x7f60d0108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 --2- 172.21.15.150:0/2562717477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d0108e60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f60c800b0a0 tx=0x7f60c800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 -- 172.21.15.150:0/2562717477 shutdown_connections 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 --2- 172.21.15.150:0/2562717477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d0108e60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 --2- 172.21.15.150:0/2562717477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d0100af0 0x7f60d0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 --2- 172.21.15.150:0/2562717477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d0101c10 0x7f60d0102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 -- 172.21.15.150:0/2562717477 >> 172.21.15.150:0/2562717477 conn(0x7f60d00fc0e0 msgr2=0x7f60d00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 -- 172.21.15.150:0/2562717477 shutdown_connections 2024-05-10T04:38:33.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.091+0000 7f60d753a700 1 -- 172.21.15.150:0/2562717477 wait complete. 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 Processor -- start 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 -- start start 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d0100af0 0x7f60d0196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d0101c10 0x7f60d0197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d019e9c0 con 0x7f60d01025b0 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d019eb30 con 0x7f60d0101c10 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d753a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d019ecf0 con 0x7f60d0100af0 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d5ad7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d5ad7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d019e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50408/0 (socket says 172.21.15.150:50408) 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d5ad7700 1 -- 172.21.15.150:0/1795648062 learned_addr learned my addr 172.21.15.150:0/1795648062 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d4ad5700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d0101c10 0x7f60d0197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.092+0000 7f60d52d6700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d0100af0 0x7f60d0196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:33.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d5ad7700 1 -- 172.21.15.150:0/1795648062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d0100af0 msgr2=0x7f60d0196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d5ad7700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d0100af0 0x7f60d0196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d5ad7700 1 -- 172.21.15.150:0/1795648062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d0101c10 msgr2=0x7f60d0197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d5ad7700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d0101c10 0x7f60d0197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d5ad7700 1 -- 172.21.15.150:0/1795648062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c8009d00 con 0x7f60d01025b0 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d5ad7700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d019e3b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60c8006070 tx=0x7f60c80078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60c67fc700 1 -- 172.21.15.150:0/1795648062 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800e040 con 0x7f60d01025b0 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60c67fc700 1 -- 172.21.15.150:0/1795648062 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60c801e070 con 0x7f60d01025b0 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60c67fc700 1 -- 172.21.15.150:0/1795648062 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c801ae70 con 0x7f60d01025b0 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d0073440 con 0x7f60d01025b0 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.093+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d0073900 con 0x7f60d01025b0 2024-05-10T04:38:33.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.094+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d0068730 con 0x7f60d01025b0 2024-05-10T04:38:33.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.095+0000 7f60c67fc700 1 -- 172.21.15.150:0/1795648062 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f60c8003a00 con 0x7f60d01025b0 2024-05-10T04:38:33.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.096+0000 7f60c67fc700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f60bc0a11c0 0x7f60bc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:33.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.096+0000 7f60c67fc700 1 -- 172.21.15.150:0/1795648062 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f60c8020070 con 0x7f60d01025b0 2024-05-10T04:38:33.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.096+0000 7f60d52d6700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f60bc0a11c0 0x7f60bc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:33.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.097+0000 7f60d52d6700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f60bc0a11c0 0x7f60bc0a36d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60d0101b10 tx=0x7f60cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:33.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.100+0000 7f60c67fc700 1 -- 172.21.15.150:0/1795648062 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f60c8099340 con 0x7f60d01025b0 2024-05-10T04:38:33.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.346+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f60d00742f0 con 0x7f60d01025b0 2024-05-10T04:38:33.585 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: 2024-05-10T04:38:33.287+0000 7fa2dad01700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3791210687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3813050781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] up:active 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3813050781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-05-10T04:38:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-05-10T04:38:33.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-10T04:38:33.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1795648062' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T04:38:33.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3791210687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3813050781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] up:active 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3813050781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T04:38:33.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-05-10T04:38:33.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-05-10T04:38:33.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-10T04:38:33.656 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1795648062' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T04:38:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-10T04:38:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3791210687' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-10T04:38:33.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3813050781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] up:active 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3813050781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-10T04:38:33.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-05-10T04:38:33.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-05-10T04:38:33.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-10T04:38:33.663 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1795648062' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-10T04:38:33.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.945+0000 7f60c67fc700 1 -- 172.21.15.150:0/1795648062 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f60c8003cb0 con 0x7f60d01025b0 2024-05-10T04:38:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.948+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f60bc0a11c0 msgr2=0x7f60bc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.948+0000 7f60d753a700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f60bc0a11c0 0x7f60bc0a36d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60d0101b10 tx=0x7f60cc009260 comp rx=0 tx=0).stop 2024-05-10T04:38:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 msgr2=0x7f60d019e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d019e3b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60c8006070 tx=0x7f60c80078e0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 shutdown_connections 2024-05-10T04:38:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d01025b0 0x7f60d019e3b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f60bc0a11c0 0x7f60bc0a36d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d0100af0 0x7f60d0196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 --2- 172.21.15.150:0/1795648062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60d0101c10 0x7f60d0197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:33.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 >> 172.21.15.150:0/1795648062 conn(0x7f60d00fc0e0 msgr2=0x7f60d0104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:33.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 shutdown_connections 2024-05-10T04:38:33.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:33.949+0000 7f60d753a700 1 -- 172.21.15.150:0/1795648062 wait complete. 2024-05-10T04:38:33.970 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-10T04:38:33.974 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-10T04:38:34.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 -- 172.21.15.150:0/205370266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024101700 msgr2=0x7f7024109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:34.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 --2- 172.21.15.150:0/205370266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024101700 0x7f7024109010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f701c00b0a0 tx=0x7f701c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 -- 172.21.15.150:0/205370266 shutdown_connections 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 --2- 172.21.15.150:0/205370266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024101700 0x7f7024109010 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 --2- 172.21.15.150:0/205370266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70241020a0 0x7f70241024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 --2- 172.21.15.150:0/205370266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7024100db0 0x7f70241011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 -- 172.21.15.150:0/205370266 >> 172.21.15.150:0/205370266 conn(0x7f70240fb8f0 msgr2=0x7f70240fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 -- 172.21.15.150:0/205370266 shutdown_connections 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.114+0000 7f702b352700 1 -- 172.21.15.150:0/205370266 wait complete. 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 Processor -- start 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 -- start start 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024100db0 0x7f7024196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:34.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7024101700 0x7f7024197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70241020a0 0x7f702419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702419e9b0 con 0x7f7024100db0 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702419eb20 con 0x7f7024101700 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f702b352700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702419ece0 con 0x7f70241020a0 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024100db0 0x7f7024196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024100db0 0x7f7024196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50418/0 (socket says 172.21.15.150:50418) 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 -- 172.21.15.150:0/1596425845 learned_addr learned my addr 172.21.15.150:0/1596425845 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70288ed700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7024101700 0x7f7024197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70298ef700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70241020a0 0x7f702419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 -- 172.21.15.150:0/1596425845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70241020a0 msgr2=0x7f702419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70241020a0 0x7f702419e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 -- 172.21.15.150:0/1596425845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7024101700 msgr2=0x7f7024197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7024101700 0x7f7024197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70290ee700 1 -- 172.21.15.150:0/1596425845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c009d00 con 0x7f7024100db0 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.115+0000 7f70288ed700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7024101700 0x7f7024197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.116+0000 7f70290ee700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024100db0 0x7f7024196ed0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f702000c8f0 tx=0x7f702000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.116+0000 7f701a7fc700 1 -- 172.21.15.150:0/1596425845 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020007b00 con 0x7f7024100db0 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.116+0000 7f701a7fc700 1 -- 172.21.15.150:0/1596425845 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7020007c60 con 0x7f7024100db0 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.116+0000 7f701a7fc700 1 -- 172.21.15.150:0/1596425845 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020018660 con 0x7f7024100db0 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.116+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702419efc0 con 0x7f7024100db0 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.116+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702419f3f0 con 0x7f7024100db0 2024-05-10T04:38:34.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.117+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7024107eb0 con 0x7f7024100db0 2024-05-10T04:38:34.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.118+0000 7f701a7fc700 1 -- 172.21.15.150:0/1596425845 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f702000f450 con 0x7f7024100db0 2024-05-10T04:38:34.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.118+0000 7f701a7fc700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70100a1240 0x7f70100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:34.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.119+0000 7f70288ed700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70100a1240 0x7f70100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:34.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.119+0000 7f701a7fc700 1 -- 172.21.15.150:0/1596425845 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f70200cca50 con 0x7f7024100db0 2024-05-10T04:38:34.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.119+0000 7f70288ed700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70100a1240 0x7f70100a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7014005950 tx=0x7f7014005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:34.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.122+0000 7f701a7fc700 1 -- 172.21.15.150:0/1596425845 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f70200960a0 con 0x7f7024100db0 2024-05-10T04:38:34.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.354+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f702402cf30 con 0x7f70100a1240 2024-05-10T04:38:34.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:34 smithi181 ceph-mon[47314]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-05-10T04:38:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] up:active 2024-05-10T04:38:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:34 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1795648062' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T04:38:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:34 smithi181 ceph-mon[47314]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T04:38:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:34 smithi181 ceph-mon[47314]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T04:38:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:34 smithi181 ceph-mon[47314]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=k=up:creating} 2 up:standby-replay 6 up:standby 2024-05-10T04:38:34.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:34 smithi181 ceph-mon[47314]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-10T04:38:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:34 smithi150 ceph-mon[42404]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-05-10T04:38:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] up:active 2024-05-10T04:38:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1795648062' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T04:38:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:34 smithi150 ceph-mon[42404]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T04:38:34.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:34 smithi150 ceph-mon[42404]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T04:38:34.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:34 smithi150 ceph-mon[42404]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=k=up:creating} 2 up:standby-replay 6 up:standby 2024-05-10T04:38:34.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:34 smithi150 ceph-mon[42404]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-10T04:38:34.662 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:38:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:38:34.360+0000 7f782487d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:38:34.662 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:38:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:38:34.360+0000 7f782487d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:38:34.662 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:38:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:38:34.360+0000 7f782487d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:38:34.662 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:38:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:38:34.360+0000 7f782487d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:38:34.662 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:38:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:38:34.360+0000 7f782487d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-10T04:38:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:34 smithi161 ceph-mon[46912]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-05-10T04:38:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] up:active 2024-05-10T04:38:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:34 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1795648062' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-10T04:38:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:34 smithi161 ceph-mon[46912]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-10T04:38:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:34 smithi161 ceph-mon[46912]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-05-10T04:38:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:34 smithi161 ceph-mon[46912]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=k=up:creating} 2 up:standby-replay 6 up:standby 2024-05-10T04:38:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:34 smithi161 ceph-mon[46912]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-10T04:38:34.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.977+0000 7f701a7fc700 1 -- 172.21.15.150:0/1596425845 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f702402cf30 con 0x7f70100a1240 2024-05-10T04:38:34.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70100a1240 msgr2=0x7f70100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:34.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70100a1240 0x7f70100a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7014005950 tx=0x7f7014005f50 comp rx=0 tx=0).stop 2024-05-10T04:38:34.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024100db0 msgr2=0x7f7024196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:34.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024100db0 0x7f7024196ed0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f702000c8f0 tx=0x7f702000ccb0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 shutdown_connections 2024-05-10T04:38:34.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70100a1240 0x7f70100a3750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70241020a0 0x7f702419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7024100db0 0x7f7024196ed0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 --2- 172.21.15.150:0/1596425845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7024101700 0x7f7024197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:34.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 >> 172.21.15.150:0/1596425845 conn(0x7f70240fb8f0 msgr2=0x7f7024104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:34.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 shutdown_connections 2024-05-10T04:38:34.982 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:34.980+0000 7f702b352700 1 -- 172.21.15.150:0/1596425845 wait complete. 2024-05-10T04:38:34.995 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-10T04:38:35.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 -- 172.21.15.150:0/463336636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0641036a0 msgr2=0x7fb064109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 --2- 172.21.15.150:0/463336636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0641036a0 0x7fb064109f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00b0a0 tx=0x7fb05c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 -- 172.21.15.150:0/463336636 shutdown_connections 2024-05-10T04:38:35.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 --2- 172.21.15.150:0/463336636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0641036a0 0x7fb064109f50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 --2- 172.21.15.150:0/463336636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb064101be0 0x7fb064101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 --2- 172.21.15.150:0/463336636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb064102d00 0x7fb064103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 -- 172.21.15.150:0/463336636 >> 172.21.15.150:0/463336636 conn(0x7fb0640fd210 msgr2=0x7fb0640ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:35.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 -- 172.21.15.150:0/463336636 shutdown_connections 2024-05-10T04:38:35.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 -- 172.21.15.150:0/463336636 wait complete. 2024-05-10T04:38:35.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.135+0000 7fb0692d7700 1 Processor -- start 2024-05-10T04:38:35.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0692d7700 1 -- start start 2024-05-10T04:38:35.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0692d7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb064101be0 0x7fb064197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0692d7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb064102d00 0x7fb064198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0692d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0641036a0 0x7fb06419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0692d7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419fad0 con 0x7fb064102d00 2024-05-10T04:38:35.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0692d7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419fc40 con 0x7fb064101be0 2024-05-10T04:38:35.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0692d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419fe00 con 0x7fb0641036a0 2024-05-10T04:38:35.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb064102d00 0x7fb064198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb064102d00 0x7fb064198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50446/0 (socket says 172.21.15.150:50446) 2024-05-10T04:38:35.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 -- 172.21.15.150:0/3427192572 learned_addr learned my addr 172.21.15.150:0/3427192572 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:35.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0637fe700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0641036a0 0x7fb06419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb062ffd700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb064101be0 0x7fb064197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 -- 172.21.15.150:0/3427192572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0641036a0 msgr2=0x7fb06419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0641036a0 0x7fb06419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 -- 172.21.15.150:0/3427192572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb064101be0 msgr2=0x7fb064197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb064101be0 0x7fb064197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 -- 172.21.15.150:0/3427192572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c009d00 con 0x7fb064102d00 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.136+0000 7fb0627fc700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb064102d00 0x7fb064198530 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d950 tx=0x7fb05800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.137+0000 7fb04bfff700 1 -- 172.21.15.150:0/3427192572 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800f890 con 0x7fb064102d00 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.137+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0641a00e0 con 0x7fb064102d00 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.137+0000 7fb04bfff700 1 -- 172.21.15.150:0/3427192572 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb058010460 con 0x7fb064102d00 2024-05-10T04:38:35.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.137+0000 7fb04bfff700 1 -- 172.21.15.150:0/3427192572 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800e610 con 0x7fb064102d00 2024-05-10T04:38:35.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.137+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0641a0640 con 0x7fb064102d00 2024-05-10T04:38:35.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.138+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb064108df0 con 0x7fb064102d00 2024-05-10T04:38:35.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.139+0000 7fb04bfff700 1 -- 172.21.15.150:0/3427192572 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb05800f9f0 con 0x7fb064102d00 2024-05-10T04:38:35.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.139+0000 7fb04bfff700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb04c0a11c0 0x7fb04c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.139+0000 7fb062ffd700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb04c0a11c0 0x7fb04c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.139+0000 7fb04bfff700 1 -- 172.21.15.150:0/3427192572 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb0580cc880 con 0x7fb064102d00 2024-05-10T04:38:35.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.140+0000 7fb062ffd700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb04c0a11c0 0x7fb04c0a36d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb064102c00 tx=0x7fb05400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:35.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.144+0000 7fb04bfff700 1 -- 172.21.15.150:0/3427192572 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb058095e20 con 0x7fb064102d00 2024-05-10T04:38:35.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.376+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb064061650 con 0x7fb04c0a11c0 2024-05-10T04:38:35.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.401+0000 7fb04bfff700 1 -- 172.21.15.150:0/3427192572 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb064061650 con 0x7fb04c0a11c0 2024-05-10T04:38:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb04c0a11c0 msgr2=0x7fb04c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb04c0a11c0 0x7fb04c0a36d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb064102c00 tx=0x7fb05400b410 comp rx=0 tx=0).stop 2024-05-10T04:38:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb064102d00 msgr2=0x7fb064198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb064102d00 0x7fb064198530 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d950 tx=0x7fb05800dc60 comp rx=0 tx=0).stop 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 shutdown_connections 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb04c0a11c0 0x7fb04c0a36d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0641036a0 0x7fb06419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb064101be0 0x7fb064197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 --2- 172.21.15.150:0/3427192572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb064102d00 0x7fb064198530 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 >> 172.21.15.150:0/3427192572 conn(0x7fb0640fd210 msgr2=0x7fb064105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 shutdown_connections 2024-05-10T04:38:35.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.404+0000 7fb0692d7700 1 -- 172.21.15.150:0/3427192572 wait complete. 2024-05-10T04:38:35.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-10T04:38:35.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 -- 172.21.15.150:0/157302573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0101be0 msgr2=0x7f1cf0101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/157302573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0101be0 0x7f1cf0101ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009990 tx=0x7f1ce0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 -- 172.21.15.150:0/157302573 shutdown_connections 2024-05-10T04:38:35.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/157302573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf0102580 0x7f1cf0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/157302573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0100ac0 0x7f1cf0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/157302573 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0101be0 0x7f1cf0101ff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 -- 172.21.15.150:0/157302573 >> 172.21.15.150:0/157302573 conn(0x7f1cf00fc0d0 msgr2=0x7f1cf00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:35.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 -- 172.21.15.150:0/157302573 shutdown_connections 2024-05-10T04:38:35.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 -- 172.21.15.150:0/157302573 wait complete. 2024-05-10T04:38:35.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.555+0000 7f1cf4a34700 1 Processor -- start 2024-05-10T04:38:35.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cf4a34700 1 -- start start 2024-05-10T04:38:35.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cf4a34700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf0100ac0 0x7f1cf0068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cf4a34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0101be0 0x7f1cf019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cf4a34700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0102580 0x7f1cf019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cf4a34700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf019f8e0 con 0x7f1cf0102580 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cf4a34700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf019fa50 con 0x7f1cf0100ac0 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cf4a34700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf019fc10 con 0x7f1cf0101be0 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0102580 0x7f1cf019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0102580 0x7f1cf019f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50470/0 (socket says 172.21.15.150:50470) 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 -- 172.21.15.150:0/1188256044 learned_addr learned my addr 172.21.15.150:0/1188256044 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cee59c700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf0100ac0 0x7f1cf0068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cedd9b700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0101be0 0x7f1cf019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 -- 172.21.15.150:0/1188256044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0101be0 msgr2=0x7f1cf019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0101be0 0x7f1cf019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 -- 172.21.15.150:0/1188256044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf0100ac0 msgr2=0x7f1cf0068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf0100ac0 0x7f1cf0068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 -- 172.21.15.150:0/1188256044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00095f0 con 0x7f1cf0102580 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cedd9b700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0101be0 0x7f1cf019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1ceed9d700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0102580 0x7f1cf019f1e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8007ee0 tx=0x7f1ce800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.556+0000 7f1cdf7fe700 1 -- 172.21.15.150:0/1188256044 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8017070 con 0x7f1cf0102580 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.557+0000 7f1cdf7fe700 1 -- 172.21.15.150:0/1188256044 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce800f040 con 0x7f1cf0102580 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.557+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf019fef0 con 0x7f1cf0102580 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.557+0000 7f1cdf7fe700 1 -- 172.21.15.150:0/1188256044 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8005740 con 0x7f1cf0102580 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.557+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf01a0320 con 0x7f1cf0102580 2024-05-10T04:38:35.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.557+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf0107cd0 con 0x7f1cf0102580 2024-05-10T04:38:35.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.559+0000 7f1cdf7fe700 1 -- 172.21.15.150:0/1188256044 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1ce800dc60 con 0x7f1cf0102580 2024-05-10T04:38:35.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.559+0000 7f1cdf7fe700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1cd80a11c0 0x7f1cd80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.559+0000 7f1cdf7fe700 1 -- 172.21.15.150:0/1188256044 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1ce80cc8d0 con 0x7f1cf0102580 2024-05-10T04:38:35.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.559+0000 7f1cee59c700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1cd80a11c0 0x7f1cd80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.560+0000 7f1cee59c700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1cd80a11c0 0x7f1cd80a36d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0101ae0 tx=0x7f1ce400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:35.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.563+0000 7f1cdf7fe700 1 -- 172.21.15.150:0/1188256044 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ce8095e70 con 0x7f1cf0102580 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] up:active 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: Cluster is now healthy 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=k=up:active} 2 up:standby-replay 6 up:standby 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-05-10T04:38:35.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:35 smithi181 ceph-mon[47314]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] up:active 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: Cluster is now healthy 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=k=up:active} 2 up:standby-replay 6 up:standby 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-05-10T04:38:35.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:35 smithi150 ceph-mon[42404]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-05-10T04:38:35.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.792+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cf002ce90 con 0x7f1cd80a11c0 2024-05-10T04:38:35.797 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.796+0000 7f1cdf7fe700 1 -- 172.21.15.150:0/1188256044 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f1cf002ce90 con 0x7f1cd80a11c0 2024-05-10T04:38:35.798 INFO:teuthology.orchestra.run.smithi150.stdout:/volumes/qa/sv_0/3e2ace8d-a11d-4e4f-8f53-b5a77d14069d 2024-05-10T04:38:35.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1cd80a11c0 msgr2=0x7f1cd80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1cd80a11c0 0x7f1cd80a36d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0101ae0 tx=0x7f1ce400a3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0102580 msgr2=0x7f1cf019f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0102580 0x7f1cf019f1e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8007ee0 tx=0x7f1ce800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 shutdown_connections 2024-05-10T04:38:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cf0102580 0x7f1cf019f1e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cf0100ac0 0x7f1cf0068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1cd80a11c0 0x7f1cd80a36d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 --2- 172.21.15.150:0/1188256044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0101be0 0x7f1cf019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 >> 172.21.15.150:0/1188256044 conn(0x7f1cf00fc0d0 msgr2=0x7f1cf0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 shutdown_connections 2024-05-10T04:38:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.799+0000 7f1cf4a34700 1 -- 172.21.15.150:0/1188256044 wait complete. 2024-05-10T04:38:35.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-10T04:38:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] up:active 2024-05-10T04:38:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-10T04:38:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-10T04:38:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: Cluster is now healthy 2024-05-10T04:38:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=k=up:active} 2 up:standby-replay 6 up:standby 2024-05-10T04:38:35.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-05-10T04:38:35.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:35 smithi161 ceph-mon[46912]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 -- 172.21.15.150:0/2822420171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88101be0 msgr2=0x7efc88101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2822420171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88101be0 0x7efc88101ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc78009990 tx=0x7efc78009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 -- 172.21.15.150:0/2822420171 shutdown_connections 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2822420171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc88102580 0x7efc88108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2822420171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc88100ac0 0x7efc88100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2822420171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88101be0 0x7efc88101ff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 -- 172.21.15.150:0/2822420171 >> 172.21.15.150:0/2822420171 conn(0x7efc880fc0d0 msgr2=0x7efc880fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:35.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 -- 172.21.15.150:0/2822420171 shutdown_connections 2024-05-10T04:38:35.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 -- 172.21.15.150:0/2822420171 wait complete. 2024-05-10T04:38:35.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.950+0000 7efc8dc38700 1 Processor -- start 2024-05-10T04:38:35.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc8dc38700 1 -- start start 2024-05-10T04:38:35.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc8dc38700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88100ac0 0x7efc88196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc8dc38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc88101be0 0x7efc88197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc8dc38700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc88102580 0x7efc8819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc8dc38700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8819e8d0 con 0x7efc88100ac0 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc8dc38700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8819ea40 con 0x7efc88102580 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc8dc38700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8819ec00 con 0x7efc88101be0 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88100ac0 0x7efc88196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88100ac0 0x7efc88196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50494/0 (socket says 172.21.15.150:50494) 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 -- 172.21.15.150:0/2506124444 learned_addr learned my addr 172.21.15.150:0/2506124444 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc87fff700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc88102580 0x7efc8819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc86ffd700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc88101be0 0x7efc88197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 -- 172.21.15.150:0/2506124444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc88101be0 msgr2=0x7efc88197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc88101be0 0x7efc88197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 -- 172.21.15.150:0/2506124444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc88102580 msgr2=0x7efc8819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:35.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc88102580 0x7efc8819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 -- 172.21.15.150:0/2506124444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc780095f0 con 0x7efc88100ac0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.951+0000 7efc877fe700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88100ac0 0x7efc88196df0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e940 tx=0x7efc7c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.952+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00cbd0 con 0x7efc88100ac0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.952+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc7c00ee90 con 0x7efc88100ac0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.952+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8819eee0 con 0x7efc88100ac0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.952+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c003cb0 con 0x7efc88100ac0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.952+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8819f310 con 0x7efc88100ac0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.953+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8810c310 con 0x7efc88100ac0 2024-05-10T04:38:35.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.954+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135874+0+0 (secure 0 0 0) 0x7efc7c00cd30 con 0x7efc88100ac0 2024-05-10T04:38:35.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.954+0000 7efc84ff9700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc700a11c0 0x7efc700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:35.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.955+0000 7efc86ffd700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc700a11c0 0x7efc700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:35.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.955+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc7c014070 con 0x7efc88100ac0 2024-05-10T04:38:35.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.955+0000 7efc86ffd700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc700a11c0 0x7efc700a36d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efc78006010 tx=0x7efc7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:35.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.959+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc7c0963f0 con 0x7efc88100ac0 2024-05-10T04:38:35.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:35.977+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7efc7c0967c0 con 0x7efc88100ac0 2024-05-10T04:38:36.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.201+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7efc881079e0 con 0x7efc700a11c0 2024-05-10T04:38:36.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.225+0000 7efc84ff9700 1 -- 172.21.15.150:0/2506124444 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efc881079e0 con 0x7efc700a11c0 2024-05-10T04:38:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc700a11c0 msgr2=0x7efc700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc700a11c0 0x7efc700a36d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efc78006010 tx=0x7efc7801a040 comp rx=0 tx=0).stop 2024-05-10T04:38:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88100ac0 msgr2=0x7efc88196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88100ac0 0x7efc88196df0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e940 tx=0x7efc7c00ec50 comp rx=0 tx=0).stop 2024-05-10T04:38:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 shutdown_connections 2024-05-10T04:38:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc88102580 0x7efc8819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc88100ac0 0x7efc88196df0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc700a11c0 0x7efc700a36d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 --2- 172.21.15.150:0/2506124444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc88101be0 0x7efc88197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 >> 172.21.15.150:0/2506124444 conn(0x7efc880fc0d0 msgr2=0x7efc88104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 shutdown_connections 2024-05-10T04:38:36.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.227+0000 7efc8dc38700 1 -- 172.21.15.150:0/2506124444 wait complete. 2024-05-10T04:38:36.242 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-10T04:38:36.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 -- 172.21.15.150:0/2252570533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4069090 msgr2=0x7fabb4104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 --2- 172.21.15.150:0/2252570533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4069090 0x7fabb4104720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009990 tx=0x7fabb0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 -- 172.21.15.150:0/2252570533 shutdown_connections 2024-05-10T04:38:36.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 --2- 172.21.15.150:0/2252570533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb4107880 0x7fabb4109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 --2- 172.21.15.150:0/2252570533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4069090 0x7fabb4104720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 --2- 172.21.15.150:0/2252570533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb4104d10 0x7fabb41071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 -- 172.21.15.150:0/2252570533 >> 172.21.15.150:0/2252570533 conn(0x7fabb40fc0d0 msgr2=0x7fabb40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 -- 172.21.15.150:0/2252570533 shutdown_connections 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.380+0000 7fabbb625700 1 -- 172.21.15.150:0/2252570533 wait complete. 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 Processor -- start 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 -- start start 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb4069090 0x7fabb4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4104d10 0x7fabb4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb4107880 0x7fabb419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb419f9a0 con 0x7fabb4104d10 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb419fae0 con 0x7fabb4069090 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabbb625700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb419fc70 con 0x7fabb4107880 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabb8bc0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4104d10 0x7fabb4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.381+0000 7fabb8bc0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4104d10 0x7fabb4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50504/0 (socket says 172.21.15.150:50504) 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb8bc0700 1 -- 172.21.15.150:0/140156534 learned_addr learned my addr 172.21.15.150:0/140156534 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb93c1700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb4069090 0x7fabb4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb9bc2700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb4107880 0x7fabb419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb8bc0700 1 -- 172.21.15.150:0/140156534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb4107880 msgr2=0x7fabb419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb8bc0700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb4107880 0x7fabb419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb8bc0700 1 -- 172.21.15.150:0/140156534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb4069090 msgr2=0x7fabb4197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb8bc0700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb4069090 0x7fabb4197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb8bc0700 1 -- 172.21.15.150:0/140156534 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb00095f0 con 0x7fabb4104d10 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabb8bc0700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4104d10 0x7fabb4198440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faba800c9e0 tx=0x7faba800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7faba67fc700 1 -- 172.21.15.150:0/140156534 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8007ac0 con 0x7fabb4104d10 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7faba67fc700 1 -- 172.21.15.150:0/140156534 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faba8007c20 con 0x7fabb4104d10 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7faba67fc700 1 -- 172.21.15.150:0/140156534 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8018660 con 0x7fabb4104d10 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb419ff50 con 0x7fabb4104d10 2024-05-10T04:38:36.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.382+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb41a0370 con 0x7fabb4104d10 2024-05-10T04:38:36.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.383+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb40fdc80 con 0x7fabb4104d10 2024-05-10T04:38:36.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.384+0000 7faba67fc700 1 -- 172.21.15.150:0/140156534 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7faba800f450 con 0x7fabb4104d10 2024-05-10T04:38:36.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.385+0000 7faba67fc700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faba00a12e0 0x7faba00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.385+0000 7fabb93c1700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faba00a12e0 0x7faba00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.385+0000 7faba67fc700 1 -- 172.21.15.150:0/140156534 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faba80ccbf0 con 0x7fabb4104d10 2024-05-10T04:38:36.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.386+0000 7fabb93c1700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faba00a12e0 0x7faba00a37f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabb0003730 tx=0x7fabb001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:36.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.390+0000 7faba67fc700 1 -- 172.21.15.150:0/140156534 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faba8096030 con 0x7fabb4104d10 2024-05-10T04:38:36.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.623+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fabb402cee0 con 0x7faba00a12e0 2024-05-10T04:38:36.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.628+0000 7faba67fc700 1 -- 172.21.15.150:0/140156534 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fabb402cee0 con 0x7faba00a12e0 2024-05-10T04:38:36.629 INFO:teuthology.orchestra.run.smithi150.stdout:/volumes/qa/sv_1/013d3174-3ee5-41d3-8986-00bdc9f11cf2 2024-05-10T04:38:36.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faba00a12e0 msgr2=0x7faba00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faba00a12e0 0x7faba00a37f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabb0003730 tx=0x7fabb001a040 comp rx=0 tx=0).stop 2024-05-10T04:38:36.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4104d10 msgr2=0x7fabb4198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4104d10 0x7fabb4198440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faba800c9e0 tx=0x7faba800cda0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 shutdown_connections 2024-05-10T04:38:36.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faba00a12e0 0x7faba00a37f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb4107880 0x7fabb419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fabb4069090 0x7fabb4197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 --2- 172.21.15.150:0/140156534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabb4104d10 0x7fabb4198440 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 >> 172.21.15.150:0/140156534 conn(0x7fabb40fc0d0 msgr2=0x7fabb4103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:36.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 shutdown_connections 2024-05-10T04:38:36.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.630+0000 7fabbb625700 1 -- 172.21.15.150:0/140156534 wait complete. 2024-05-10T04:38:36.644 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:36 smithi150 ceph-mon[42404]: pgmap v238: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-10T04:38:36.644 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:36 smithi150 ceph-mon[42404]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:36.644 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:36 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] up:active 2024-05-10T04:38:36.644 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:36 smithi150 ceph-mon[42404]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-05-10T04:38:36.644 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:36 smithi150 ceph-mon[42404]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T04:38:36.644 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:36 smithi150 ceph-mon[42404]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T04:38:36.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:38:36.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:36 smithi181 ceph-mon[47314]: pgmap v238: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-10T04:38:36.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:36 smithi181 ceph-mon[47314]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:36.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:36 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] up:active 2024-05-10T04:38:36.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:36 smithi181 ceph-mon[47314]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-05-10T04:38:36.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:36 smithi181 ceph-mon[47314]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T04:38:36.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:36 smithi181 ceph-mon[47314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T04:38:36.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 -- 172.21.15.150:0/125556232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8107880 msgr2=0x7fcfb8109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 --2- 172.21.15.150:0/125556232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8107880 0x7fcfb8109d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b0a0 tx=0x7fcfb000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 -- 172.21.15.150:0/125556232 shutdown_connections 2024-05-10T04:38:36.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 --2- 172.21.15.150:0/125556232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8107880 0x7fcfb8109d90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 --2- 172.21.15.150:0/125556232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8069070 0x7fcfb8104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 --2- 172.21.15.150:0/125556232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb8104d10 0x7fcfb81071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 -- 172.21.15.150:0/125556232 >> 172.21.15.150:0/125556232 conn(0x7fcfb80fc0d0 msgr2=0x7fcfb80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:36.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 -- 172.21.15.150:0/125556232 shutdown_connections 2024-05-10T04:38:36.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.777+0000 7fcfbe453700 1 -- 172.21.15.150:0/125556232 wait complete. 2024-05-10T04:38:36.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 Processor -- start 2024-05-10T04:38:36.780 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 -- start start 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8069070 0x7fcfb81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8104d10 0x7fcfb81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb8107880 0x7fcfb819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819fb50 con 0x7fcfb8104d10 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819fc90 con 0x7fcfb8069070 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfbe453700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819fe20 con 0x7fcfb8107880 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfb77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8104d10 0x7fcfb81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfb77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8104d10 0x7fcfb81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50526/0 (socket says 172.21.15.150:50526) 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.778+0000 7fcfb77fe700 1 -- 172.21.15.150:0/1289182560 learned_addr learned my addr 172.21.15.150:0/1289182560 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfbc9f0700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb8107880 0x7fcfb819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb7fff700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8069070 0x7fcfb81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb77fe700 1 -- 172.21.15.150:0/1289182560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb8107880 msgr2=0x7fcfb819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb77fe700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb8107880 0x7fcfb819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb77fe700 1 -- 172.21.15.150:0/1289182560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8069070 msgr2=0x7fcfb81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb77fe700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8069070 0x7fcfb81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb77fe700 1 -- 172.21.15.150:0/1289182560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0009d00 con 0x7fcfb8104d10 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfbc9f0700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb8107880 0x7fcfb819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb7fff700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8069070 0x7fcfb81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb77fe700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8104d10 0x7fcfb81985f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c9e0 tx=0x7fcfa800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb57fa700 1 -- 172.21.15.150:0/1289182560 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8007ac0 con 0x7fcfb8104d10 2024-05-10T04:38:36.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb81a0100 con 0x7fcfb8104d10 2024-05-10T04:38:36.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb57fa700 1 -- 172.21.15.150:0/1289182560 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfa8007c20 con 0x7fcfb8104d10 2024-05-10T04:38:36.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfb57fa700 1 -- 172.21.15.150:0/1289182560 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8018660 con 0x7fcfb8104d10 2024-05-10T04:38:36.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.779+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb81a0610 con 0x7fcfb8104d10 2024-05-10T04:38:36.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.780+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb80fdc80 con 0x7fcfb8104d10 2024-05-10T04:38:36.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.781+0000 7fcfb57fa700 1 -- 172.21.15.150:0/1289182560 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fcfa800f450 con 0x7fcfb8104d10 2024-05-10T04:38:36.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.782+0000 7fcfb57fa700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fcfa00a11b0 0x7fcfa00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:36.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.782+0000 7fcfb7fff700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fcfa00a11b0 0x7fcfa00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:36.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.782+0000 7fcfb57fa700 1 -- 172.21.15.150:0/1289182560 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcfa80cce40 con 0x7fcfb8104d10 2024-05-10T04:38:36.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.782+0000 7fcfb7fff700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fcfa00a11b0 0x7fcfa00a36c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8068d60 tx=0x7fcfac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:36.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:36.786+0000 7fcfb57fa700 1 -- 172.21.15.150:0/1289182560 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcfa8096280 con 0x7fcfb8104d10 2024-05-10T04:38:36.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:36 smithi161 ceph-mon[46912]: pgmap v238: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-10T04:38:36.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:36 smithi161 ceph-mon[46912]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:36.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:36 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] up:active 2024-05-10T04:38:36.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:36 smithi161 ceph-mon[46912]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-05-10T04:38:36.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:36 smithi161 ceph-mon[46912]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-10T04:38:36.837 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:36 smithi161 ceph-mon[46912]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T04:38:37.035 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.034+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfb80694d0 con 0x7fcfb8104d10 2024-05-10T04:38:37.036 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.035+0000 7fcfb57fa700 1 -- 172.21.15.150:0/1289182560 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10933 (secure 0 0 0) 0x7fcfa80998a0 con 0x7fcfb8104d10 2024-05-10T04:38:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:37.037 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:35.978906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fcfa00a11b0 msgr2=0x7fcfa00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fcfa00a11b0 0x7fcfa00a36c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8068d60 tx=0x7fcfac009260 comp rx=0 tx=0).stop 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8104d10 msgr2=0x7fcfb81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8104d10 0x7fcfb81985f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c9e0 tx=0x7fcfa800cda0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 shutdown_connections 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fcfa00a11b0 0x7fcfa00a36c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb8107880 0x7fcfb819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.039 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8069070 0x7fcfb81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 --2- 172.21.15.150:0/1289182560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfb8104d10 0x7fcfb81985f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 >> 172.21.15.150:0/1289182560 conn(0x7fcfb80fc0d0 msgr2=0x7fcfb8102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:37.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 shutdown_connections 2024-05-10T04:38:37.040 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.037+0000 7fcfbe453700 1 -- 172.21.15.150:0/1289182560 wait complete. 2024-05-10T04:38:37.040 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 28 2024-05-10T04:38:37.052 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:38:37.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 -- 172.21.15.150:0/616766047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484102580 msgr2=0x7f8484108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 --2- 172.21.15.150:0/616766047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484102580 0x7f8484108e30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f847c00b0a0 tx=0x7f847c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 -- 172.21.15.150:0/616766047 shutdown_connections 2024-05-10T04:38:37.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 --2- 172.21.15.150:0/616766047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484102580 0x7f8484108e30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 --2- 172.21.15.150:0/616766047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8484100ac0 0x7f8484100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 --2- 172.21.15.150:0/616766047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8484101be0 0x7f8484101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 -- 172.21.15.150:0/616766047 >> 172.21.15.150:0/616766047 conn(0x7f84840fc0d0 msgr2=0x7f84840fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 -- 172.21.15.150:0/616766047 shutdown_connections 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.190+0000 7f848af8b700 1 -- 172.21.15.150:0/616766047 wait complete. 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 Processor -- start 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 -- start start 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8484100ac0 0x7f8484198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484101be0 0x7f84841985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8484102580 0x7f848419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848419fb40 con 0x7f8484101be0 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848419fcb0 con 0x7f8484100ac0 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f848af8b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848419fe70 con 0x7f8484102580 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484101be0 0x7f84841985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484101be0 0x7f84841985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:50544/0 (socket says 172.21.15.150:50544) 2024-05-10T04:38:37.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 -- 172.21.15.150:0/276510158 learned_addr learned my addr 172.21.15.150:0/276510158 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8488d27700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8484100ac0 0x7f8484198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8489528700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8484102580 0x7f848419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 -- 172.21.15.150:0/276510158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8484102580 msgr2=0x7f848419f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8484102580 0x7f848419f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 -- 172.21.15.150:0/276510158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8484100ac0 msgr2=0x7f8484198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8484100ac0 0x7f8484198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.191+0000 7f8483fff700 1 -- 172.21.15.150:0/276510158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009d00 con 0x7f8484101be0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.192+0000 7f8483fff700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484101be0 0x7f84841985a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f847000e9a0 tx=0x7f847000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.192+0000 7f8481ffb700 1 -- 172.21.15.150:0/276510158 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470009790 con 0x7f8484101be0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.192+0000 7f8481ffb700 1 -- 172.21.15.150:0/276510158 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f847000cbd0 con 0x7f8484101be0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.192+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84841a0150 con 0x7f8484101be0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.192+0000 7f8481ffb700 1 -- 172.21.15.150:0/276510158 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470018680 con 0x7f8484101be0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.192+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84841a06b0 con 0x7f8484101be0 2024-05-10T04:38:37.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.193+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8484107cd0 con 0x7f8484101be0 2024-05-10T04:38:37.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.194+0000 7f8481ffb700 1 -- 172.21.15.150:0/276510158 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f847000cd40 con 0x7f8484101be0 2024-05-10T04:38:37.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.195+0000 7f8481ffb700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84740a1230 0x7f84740a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.195+0000 7f8481ffb700 1 -- 172.21.15.150:0/276510158 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8470014070 con 0x7f8484101be0 2024-05-10T04:38:37.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.195+0000 7f8488d27700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84740a1230 0x7f84740a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.195+0000 7f8488d27700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84740a1230 0x7f84740a3740 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8484101ae0 tx=0x7f8478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:37.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.198+0000 7f8481ffb700 1 -- 172.21.15.150:0/276510158 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f84700965b0 con 0x7f8484101be0 2024-05-10T04:38:37.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.400+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84840694d0 con 0x7f8484101be0 2024-05-10T04:38:37.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.404+0000 7f8481ffb700 1 -- 172.21.15.150:0/276510158 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f8470026090 con 0x7f8484101be0 2024-05-10T04:38:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:37.408 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:37.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84740a1230 msgr2=0x7f84740a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84740a1230 0x7f84740a3740 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8484101ae0 tx=0x7f8478009260 comp rx=0 tx=0).stop 2024-05-10T04:38:37.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484101be0 msgr2=0x7f84841985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484101be0 0x7f84841985a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f847000e9a0 tx=0x7f847000ecb0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 shutdown_connections 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f84740a1230 0x7f84740a3740 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8484102580 0x7f848419f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8484100ac0 0x7f8484198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 --2- 172.21.15.150:0/276510158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8484101be0 0x7f84841985a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 >> 172.21.15.150:0/276510158 conn(0x7f84840fc0d0 msgr2=0x7f8484104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 shutdown_connections 2024-05-10T04:38:37.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.407+0000 7f848af8b700 1 -- 172.21.15.150:0/276510158 wait complete. 2024-05-10T04:38:37.423 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-10T04:38:37.423 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:38:37.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 -- 172.21.15.150:0/3095164134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4104d10 msgr2=0x7f22a41071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 --2- 172.21.15.150:0/3095164134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4104d10 0x7f22a41071f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2294009990 tx=0x7f2294009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 -- 172.21.15.150:0/3095164134 shutdown_connections 2024-05-10T04:38:37.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 --2- 172.21.15.150:0/3095164134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4107880 0x7f22a4109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 --2- 172.21.15.150:0/3095164134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4069090 0x7f22a4104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 --2- 172.21.15.150:0/3095164134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4104d10 0x7f22a41071f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 -- 172.21.15.150:0/3095164134 >> 172.21.15.150:0/3095164134 conn(0x7f22a40fc0d0 msgr2=0x7f22a40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 -- 172.21.15.150:0/3095164134 shutdown_connections 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.556+0000 7f22a95d6700 1 -- 172.21.15.150:0/3095164134 wait complete. 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 Processor -- start 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 -- start start 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4069090 0x7f22a4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4104d10 0x7f22a4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4107880 0x7f22a419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419fac0 con 0x7f22a4069090 2024-05-10T04:38:37.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419fc30 con 0x7f22a4107880 2024-05-10T04:38:37.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a95d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419fdf0 con 0x7f22a4104d10 2024-05-10T04:38:37.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4069090 0x7f22a4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4069090 0x7f22a4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57188/0 (socket says 172.21.15.150:57188) 2024-05-10T04:38:37.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.557+0000 7f22a2ffd700 1 -- 172.21.15.150:0/871711841 learned_addr learned my addr 172.21.15.150:0/871711841 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a27fc700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4104d10 0x7f22a4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a37fe700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4107880 0x7f22a419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a2ffd700 1 -- 172.21.15.150:0/871711841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4104d10 msgr2=0x7f22a4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a2ffd700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4104d10 0x7f22a4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a2ffd700 1 -- 172.21.15.150:0/871711841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4107880 msgr2=0x7f22a419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a2ffd700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4107880 0x7f22a419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a2ffd700 1 -- 172.21.15.150:0/871711841 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940095f0 con 0x7f22a4069090 2024-05-10T04:38:37.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a37fe700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4107880 0x7f22a419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a27fc700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4104d10 0x7f22a4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a2ffd700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4069090 0x7f22a4197fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f229800d950 tx=0x7f229800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f228bfff700 1 -- 172.21.15.150:0/871711841 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800bb60 con 0x7f22a4069090 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f228bfff700 1 -- 172.21.15.150:0/871711841 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2298007620 con 0x7f22a4069090 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a41a00d0 con 0x7f22a4069090 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f228bfff700 1 -- 172.21.15.150:0/871711841 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800f620 con 0x7f22a4069090 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.558+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a41a0630 con 0x7f22a4069090 2024-05-10T04:38:37.565 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.559+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a40fdc80 con 0x7f22a4069090 2024-05-10T04:38:37.566 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.560+0000 7f228bfff700 1 -- 172.21.15.150:0/871711841 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2298010460 con 0x7f22a4069090 2024-05-10T04:38:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.561+0000 7f228bfff700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f228c0a1230 0x7f228c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.561+0000 7f22a27fc700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f228c0a1230 0x7f228c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.561+0000 7f228bfff700 1 -- 172.21.15.150:0/871711841 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f22980cce80 con 0x7f22a4069090 2024-05-10T04:38:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.561+0000 7f22a27fc700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f228c0a1230 0x7f228c0a3740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f229400b7b0 tx=0x7f2294005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:37.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.566+0000 7f228bfff700 1 -- 172.21.15.150:0/871711841 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f22980962c0 con 0x7f22a4069090 2024-05-10T04:38:37.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:37 smithi181 ceph-mon[47314]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:37.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:37 smithi181 ceph-mon[47314]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:37.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:37 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1289182560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:37.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:37 smithi150 ceph-mon[42404]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:37.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:37 smithi150 ceph-mon[42404]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:37.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:37 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1289182560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:37.762 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.761+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22a4061650 con 0x7f228c0a1230 2024-05-10T04:38:37.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.774+0000 7f228bfff700 1 -- 172.21.15.150:0/871711841 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422114 (secure 0 0 0) 0x7f22a4061650 con 0x7f228c0a1230 2024-05-10T04:38:37.781 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:37.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f228c0a1230 msgr2=0x7f228c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f228c0a1230 0x7f228c0a3740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f229400b7b0 tx=0x7f2294005fb0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4069090 msgr2=0x7f22a4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4069090 0x7f22a4197fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f229800d950 tx=0x7f229800dd10 comp rx=0 tx=0).stop 2024-05-10T04:38:37.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 shutdown_connections 2024-05-10T04:38:37.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22a4107880 0x7f22a419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22a4069090 0x7f22a4197fe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f228c0a1230 0x7f228c0a3740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 --2- 172.21.15.150:0/871711841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a4104d10 0x7f22a4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 >> 172.21.15.150:0/871711841 conn(0x7f22a40fc0d0 msgr2=0x7f22a4104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:37.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 shutdown_connections 2024-05-10T04:38:37.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.778+0000 7f22a95d6700 1 -- 172.21.15.150:0/871711841 wait complete. 2024-05-10T04:38:37.786 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:38:37.793 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-05-10T04:38:37.310333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598372,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":143,"num_read_kb":123,"num_write":325,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":266,"ondisk_log_size":266,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360008,"kb_used_data":11240,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581816,"statfs":{"total":1151940427776,"available":1151571779584,"internally_reserved":0,"allocated":11509760,"data_stored":7331080,"data_compressed":247101,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19127,"internal_metadata":357021001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3551,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":97,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003825"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787827+0000","last_change":"2024-05-10T04:38:08.784019+0000","last_active":"2024-05-10T04:38:12.787827+0000","last_peered":"2024-05-10T04:38:12.787827+0000","last_clean":"2024-05-10T04:38:12.787827+0000","last_became_active":"2024-05-10T04:38:08.783585+0000","last_became_peered":"2024-05-10T04:38:08.783585+0000","last_unstale":"2024-05-10T04:38:12.787827+0000","last_undegraded":"2024-05-10T04:38:12.787827+0000","last_fullsized":"2024-05-10T04:38:12.787827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:56.963068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788295+0000","last_change":"2024-05-10T04:38:08.777364+0000","last_active":"2024-05-10T04:38:12.788295+0000","last_peered":"2024-05-10T04:38:12.788295+0000","last_clean":"2024-05-10T04:38:12.788295+0000","last_became_active":"2024-05-10T04:38:08.776488+0000","last_became_peered":"2024-05-10T04:38:08.776488+0000","last_unstale":"2024-05-10T04:38:12.788295+0000","last_undegraded":"2024-05-10T04:38:12.788295+0000","last_fullsized":"2024-05-10T04:38:12.788295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:40.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787110+0000","last_change":"2024-05-10T04:38:08.777169+0000","last_active":"2024-05-10T04:38:12.787110+0000","last_peered":"2024-05-10T04:38:12.787110+0000","last_clean":"2024-05-10T04:38:12.787110+0000","last_became_active":"2024-05-10T04:38:08.776713+0000","last_became_peered":"2024-05-10T04:38:08.776713+0000","last_unstale":"2024-05-10T04:38:12.787110+0000","last_undegraded":"2024-05-10T04:38:12.787110+0000","last_fullsized":"2024-05-10T04:38:12.787110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:08:41.947577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791723+0000","last_change":"2024-05-10T04:38:08.788464+0000","last_active":"2024-05-10T04:38:12.791723+0000","last_peered":"2024-05-10T04:38:12.791723+0000","last_clean":"2024-05-10T04:38:12.791723+0000","last_became_active":"2024-05-10T04:38:08.787811+0000","last_became_peered":"2024-05-10T04:38:08.787811+0000","last_unstale":"2024-05-10T04:38:12.791723+0000","last_undegraded":"2024-05-10T04:38:12.791723+0000","last_fullsized":"2024-05-10T04:38:12.791723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:22.574882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156610+0000","last_change":"2024-05-10T04:38:08.772904+0000","last_active":"2024-05-10T04:38:13.156610+0000","last_peered":"2024-05-10T04:38:13.156610+0000","last_clean":"2024-05-10T04:38:13.156610+0000","last_became_active":"2024-05-10T04:38:08.772503+0000","last_became_peered":"2024-05-10T04:38:08.772503+0000","last_unstale":"2024-05-10T04:38:13.156610+0000","last_undegraded":"2024-05-10T04:38:13.156610+0000","last_fullsized":"2024-05-10T04:38:13.156610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:58.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809064+0000","last_change":"2024-05-10T04:38:08.779442+0000","last_active":"2024-05-10T04:38:12.809064+0000","last_peered":"2024-05-10T04:38:12.809064+0000","last_clean":"2024-05-10T04:38:12.809064+0000","last_became_active":"2024-05-10T04:38:08.777348+0000","last_became_peered":"2024-05-10T04:38:08.777348+0000","last_unstale":"2024-05-10T04:38:12.809064+0000","last_undegraded":"2024-05-10T04:38:12.809064+0000","last_fullsized":"2024-05-10T04:38:12.809064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:25.619031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786546+0000","last_change":"2024-05-10T04:38:08.775646+0000","last_active":"2024-05-10T04:38:12.786546+0000","last_peered":"2024-05-10T04:38:12.786546+0000","last_clean":"2024-05-10T04:38:12.786546+0000","last_became_active":"2024-05-10T04:38:08.768229+0000","last_became_peered":"2024-05-10T04:38:08.768229+0000","last_unstale":"2024-05-10T04:38:12.786546+0000","last_undegraded":"2024-05-10T04:38:12.786546+0000","last_fullsized":"2024-05-10T04:38:12.786546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:14.653065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787545+0000","last_change":"2024-05-10T04:38:08.771012+0000","last_active":"2024-05-10T04:38:12.787545+0000","last_peered":"2024-05-10T04:38:12.787545+0000","last_clean":"2024-05-10T04:38:12.787545+0000","last_became_active":"2024-05-10T04:38:08.770674+0000","last_became_peered":"2024-05-10T04:38:08.770674+0000","last_unstale":"2024-05-10T04:38:12.787545+0000","last_undegraded":"2024-05-10T04:38:12.787545+0000","last_fullsized":"2024-05-10T04:38:12.787545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:14.389263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784668+0000","last_change":"2024-05-10T04:38:08.782109+0000","last_active":"2024-05-10T04:38:12.784668+0000","last_peered":"2024-05-10T04:38:12.784668+0000","last_clean":"2024-05-10T04:38:12.784668+0000","last_became_active":"2024-05-10T04:38:08.781717+0000","last_became_peered":"2024-05-10T04:38:08.781717+0000","last_unstale":"2024-05-10T04:38:12.784668+0000","last_undegraded":"2024-05-10T04:38:12.784668+0000","last_fullsized":"2024-05-10T04:38:12.784668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:17.505448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809930+0000","last_change":"2024-05-10T04:38:08.778429+0000","last_active":"2024-05-10T04:38:12.809930+0000","last_peered":"2024-05-10T04:38:12.809930+0000","last_clean":"2024-05-10T04:38:12.809930+0000","last_became_active":"2024-05-10T04:38:08.777874+0000","last_became_peered":"2024-05-10T04:38:08.777874+0000","last_unstale":"2024-05-10T04:38:12.809930+0000","last_undegraded":"2024-05-10T04:38:12.809930+0000","last_fullsized":"2024-05-10T04:38:12.809930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:20:19.704678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786985+0000","last_change":"2024-05-10T04:38:08.780104+0000","last_active":"2024-05-10T04:38:12.786985+0000","last_peered":"2024-05-10T04:38:12.786985+0000","last_clean":"2024-05-10T04:38:12.786985+0000","last_became_active":"2024-05-10T04:38:08.779298+0000","last_became_peered":"2024-05-10T04:38:08.779298+0000","last_unstale":"2024-05-10T04:38:12.786985+0000","last_undegraded":"2024-05-10T04:38:12.786985+0000","last_fullsized":"2024-05-10T04:38:12.786985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:41.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791619+0000","last_change":"2024-05-10T04:38:08.788605+0000","last_active":"2024-05-10T04:38:12.791619+0000","last_peered":"2024-05-10T04:38:12.791619+0000","last_clean":"2024-05-10T04:38:12.791619+0000","last_became_active":"2024-05-10T04:38:08.788207+0000","last_became_peered":"2024-05-10T04:38:08.788207+0000","last_unstale":"2024-05-10T04:38:12.791619+0000","last_undegraded":"2024-05-10T04:38:12.791619+0000","last_fullsized":"2024-05-10T04:38:12.791619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:35.000980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782473+0000","last_change":"2024-05-10T04:38:08.761953+0000","last_active":"2024-05-10T04:38:12.782473+0000","last_peered":"2024-05-10T04:38:12.782473+0000","last_clean":"2024-05-10T04:38:12.782473+0000","last_became_active":"2024-05-10T04:38:08.761585+0000","last_became_peered":"2024-05-10T04:38:08.761585+0000","last_unstale":"2024-05-10T04:38:12.782473+0000","last_undegraded":"2024-05-10T04:38:12.782473+0000","last_fullsized":"2024-05-10T04:38:12.782473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:25.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786743+0000","last_change":"2024-05-10T04:38:08.777656+0000","last_active":"2024-05-10T04:38:12.786743+0000","last_peered":"2024-05-10T04:38:12.786743+0000","last_clean":"2024-05-10T04:38:12.786743+0000","last_became_active":"2024-05-10T04:38:08.777332+0000","last_became_peered":"2024-05-10T04:38:08.777332+0000","last_unstale":"2024-05-10T04:38:12.786743+0000","last_undegraded":"2024-05-10T04:38:12.786743+0000","last_fullsized":"2024-05-10T04:38:12.786743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:17:52.878943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787112+0000","last_change":"2024-05-10T04:38:08.784934+0000","last_active":"2024-05-10T04:38:12.787112+0000","last_peered":"2024-05-10T04:38:12.787112+0000","last_clean":"2024-05-10T04:38:12.787112+0000","last_became_active":"2024-05-10T04:38:08.784644+0000","last_became_peered":"2024-05-10T04:38:08.784644+0000","last_unstale":"2024-05-10T04:38:12.787112+0000","last_undegraded":"2024-05-10T04:38:12.787112+0000","last_fullsized":"2024-05-10T04:38:12.787112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:43.889305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783537+0000","last_change":"2024-05-10T04:38:08.784110+0000","last_active":"2024-05-10T04:38:12.783537+0000","last_peered":"2024-05-10T04:38:12.783537+0000","last_clean":"2024-05-10T04:38:12.783537+0000","last_became_active":"2024-05-10T04:38:08.783779+0000","last_became_peered":"2024-05-10T04:38:08.783779+0000","last_unstale":"2024-05-10T04:38:12.783537+0000","last_undegraded":"2024-05-10T04:38:12.783537+0000","last_fullsized":"2024-05-10T04:38:12.783537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:24.233024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787536+0000","last_change":"2024-05-10T04:38:08.788685+0000","last_active":"2024-05-10T04:38:12.787536+0000","last_peered":"2024-05-10T04:38:12.787536+0000","last_clean":"2024-05-10T04:38:12.787536+0000","last_became_active":"2024-05-10T04:38:08.788122+0000","last_became_peered":"2024-05-10T04:38:08.788122+0000","last_unstale":"2024-05-10T04:38:12.787536+0000","last_undegraded":"2024-05-10T04:38:12.787536+0000","last_fullsized":"2024-05-10T04:38:12.787536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:21:01.098784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802717+0000","last_change":"2024-05-10T04:38:08.776330+0000","last_active":"2024-05-10T04:38:12.802717+0000","last_peered":"2024-05-10T04:38:12.802717+0000","last_clean":"2024-05-10T04:38:12.802717+0000","last_became_active":"2024-05-10T04:38:08.776005+0000","last_became_peered":"2024-05-10T04:38:08.776005+0000","last_unstale":"2024-05-10T04:38:12.802717+0000","last_undegraded":"2024-05-10T04:38:12.802717+0000","last_fullsized":"2024-05-10T04:38:12.802717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:46:14.407766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787933+0000","last_change":"2024-05-10T04:38:08.772811+0000","last_active":"2024-05-10T04:38:12.787933+0000","last_peered":"2024-05-10T04:38:12.787933+0000","last_clean":"2024-05-10T04:38:12.787933+0000","last_became_active":"2024-05-10T04:38:08.772462+0000","last_became_peered":"2024-05-10T04:38:08.772462+0000","last_unstale":"2024-05-10T04:38:12.787933+0000","last_undegraded":"2024-05-10T04:38:12.787933+0000","last_fullsized":"2024-05-10T04:38:12.787933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:39.703137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410816+0000","last_change":"2024-05-10T04:38:12.799483+0000","last_active":"2024-05-10T04:38:13.410816+0000","last_peered":"2024-05-10T04:38:13.410816+0000","last_clean":"2024-05-10T04:38:13.410816+0000","last_became_active":"2024-05-10T04:38:12.799049+0000","last_became_peered":"2024-05-10T04:38:12.799049+0000","last_unstale":"2024-05-10T04:38:13.410816+0000","last_undegraded":"2024-05-10T04:38:13.410816+0000","last_fullsized":"2024-05-10T04:38:13.410816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:00.380645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968383+0000","last_change":"2024-05-10T04:38:08.345935+0000","last_active":"2024-05-10T04:38:34.968383+0000","last_peered":"2024-05-10T04:38:34.968383+0000","last_clean":"2024-05-10T04:38:34.968383+0000","last_became_active":"2024-05-10T04:38:08.345580+0000","last_became_peered":"2024-05-10T04:38:08.345580+0000","last_unstale":"2024-05-10T04:38:34.968383+0000","last_undegraded":"2024-05-10T04:38:34.968383+0000","last_fullsized":"2024-05-10T04:38:34.968383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:11.527710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787511+0000","last_change":"2024-05-10T04:38:08.771072+0000","last_active":"2024-05-10T04:38:12.787511+0000","last_peered":"2024-05-10T04:38:12.787511+0000","last_clean":"2024-05-10T04:38:12.787511+0000","last_became_active":"2024-05-10T04:38:08.770711+0000","last_became_peered":"2024-05-10T04:38:08.770711+0000","last_unstale":"2024-05-10T04:38:12.787511+0000","last_undegraded":"2024-05-10T04:38:12.787511+0000","last_fullsized":"2024-05-10T04:38:12.787511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:44.803219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220274+0000","last_change":"2024-05-10T04:38:12.801374+0000","last_active":"2024-05-10T04:38:13.220274+0000","last_peered":"2024-05-10T04:38:13.220274+0000","last_clean":"2024-05-10T04:38:13.220274+0000","last_became_active":"2024-05-10T04:38:12.800889+0000","last_became_peered":"2024-05-10T04:38:12.800889+0000","last_unstale":"2024-05-10T04:38:13.220274+0000","last_undegraded":"2024-05-10T04:38:13.220274+0000","last_fullsized":"2024-05-10T04:38:13.220274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:35:51.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217645+0000","last_change":"2024-05-10T04:38:08.348806+0000","last_active":"2024-05-10T04:38:13.217645+0000","last_peered":"2024-05-10T04:38:13.217645+0000","last_clean":"2024-05-10T04:38:13.217645+0000","last_became_active":"2024-05-10T04:38:08.348513+0000","last_became_peered":"2024-05-10T04:38:08.348513+0000","last_unstale":"2024-05-10T04:38:13.217645+0000","last_undegraded":"2024-05-10T04:38:13.217645+0000","last_fullsized":"2024-05-10T04:38:13.217645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:45.474403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787353+0000","last_change":"2024-05-10T04:38:08.782738+0000","last_active":"2024-05-10T04:38:12.787353+0000","last_peered":"2024-05-10T04:38:12.787353+0000","last_clean":"2024-05-10T04:38:12.787353+0000","last_became_active":"2024-05-10T04:38:08.782375+0000","last_became_peered":"2024-05-10T04:38:08.782375+0000","last_unstale":"2024-05-10T04:38:12.787353+0000","last_undegraded":"2024-05-10T04:38:12.787353+0000","last_fullsized":"2024-05-10T04:38:12.787353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:16.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220388+0000","last_change":"2024-05-10T04:38:12.801322+0000","last_active":"2024-05-10T04:38:13.220388+0000","last_peered":"2024-05-10T04:38:13.220388+0000","last_clean":"2024-05-10T04:38:13.220388+0000","last_became_active":"2024-05-10T04:38:12.800388+0000","last_became_peered":"2024-05-10T04:38:12.800388+0000","last_unstale":"2024-05-10T04:38:13.220388+0000","last_undegraded":"2024-05-10T04:38:13.220388+0000","last_fullsized":"2024-05-10T04:38:13.220388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:57.678691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950748+0000","last_change":"2024-05-10T04:38:08.347997+0000","last_active":"2024-05-10T04:38:32.950748+0000","last_peered":"2024-05-10T04:38:32.950748+0000","last_clean":"2024-05-10T04:38:32.950748+0000","last_became_active":"2024-05-10T04:38:08.347469+0000","last_became_peered":"2024-05-10T04:38:08.347469+0000","last_unstale":"2024-05-10T04:38:32.950748+0000","last_undegraded":"2024-05-10T04:38:32.950748+0000","last_fullsized":"2024-05-10T04:38:32.950748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:39.122148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783940+0000","last_change":"2024-05-10T04:38:08.781068+0000","last_active":"2024-05-10T04:38:12.783940+0000","last_peered":"2024-05-10T04:38:12.783940+0000","last_clean":"2024-05-10T04:38:12.783940+0000","last_became_active":"2024-05-10T04:38:08.780671+0000","last_became_peered":"2024-05-10T04:38:08.780671+0000","last_unstale":"2024-05-10T04:38:12.783940+0000","last_undegraded":"2024-05-10T04:38:12.783940+0000","last_fullsized":"2024-05-10T04:38:12.783940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220322+0000","last_change":"2024-05-10T04:38:12.801702+0000","last_active":"2024-05-10T04:38:13.220322+0000","last_peered":"2024-05-10T04:38:13.220322+0000","last_clean":"2024-05-10T04:38:13.220322+0000","last_became_active":"2024-05-10T04:38:12.801031+0000","last_became_peered":"2024-05-10T04:38:12.801031+0000","last_unstale":"2024-05-10T04:38:13.220322+0000","last_undegraded":"2024-05-10T04:38:13.220322+0000","last_fullsized":"2024-05-10T04:38:13.220322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:05:07.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786570+0000","last_change":"2024-05-10T04:38:08.347981+0000","last_active":"2024-05-10T04:38:12.786570+0000","last_peered":"2024-05-10T04:38:12.786570+0000","last_clean":"2024-05-10T04:38:12.786570+0000","last_became_active":"2024-05-10T04:38:08.346915+0000","last_became_peered":"2024-05-10T04:38:08.346915+0000","last_unstale":"2024-05-10T04:38:12.786570+0000","last_undegraded":"2024-05-10T04:38:12.786570+0000","last_fullsized":"2024-05-10T04:38:12.786570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:52.383734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780699+0000","last_change":"2024-05-10T04:38:08.779164+0000","last_active":"2024-05-10T04:38:12.780699+0000","last_peered":"2024-05-10T04:38:12.780699+0000","last_clean":"2024-05-10T04:38:12.780699+0000","last_became_active":"2024-05-10T04:38:08.778851+0000","last_became_peered":"2024-05-10T04:38:08.778851+0000","last_unstale":"2024-05-10T04:38:12.780699+0000","last_undegraded":"2024-05-10T04:38:12.780699+0000","last_fullsized":"2024-05-10T04:38:12.780699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:01.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411122+0000","last_change":"2024-05-10T04:38:12.796175+0000","last_active":"2024-05-10T04:38:13.411122+0000","last_peered":"2024-05-10T04:38:13.411122+0000","last_clean":"2024-05-10T04:38:13.411122+0000","last_became_active":"2024-05-10T04:38:12.795752+0000","last_became_peered":"2024-05-10T04:38:12.795752+0000","last_unstale":"2024-05-10T04:38:13.411122+0000","last_undegraded":"2024-05-10T04:38:13.411122+0000","last_fullsized":"2024-05-10T04:38:13.411122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:48.077369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.946490+0000","last_change":"2024-05-10T04:38:08.341633+0000","last_active":"2024-05-10T04:38:33.946490+0000","last_peered":"2024-05-10T04:38:33.946490+0000","last_clean":"2024-05-10T04:38:33.946490+0000","last_became_active":"2024-05-10T04:38:08.341220+0000","last_became_peered":"2024-05-10T04:38:08.341220+0000","last_unstale":"2024-05-10T04:38:33.946490+0000","last_undegraded":"2024-05-10T04:38:33.946490+0000","last_fullsized":"2024-05-10T04:38:33.946490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:36.861191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156596+0000","last_change":"2024-05-10T04:38:08.790969+0000","last_active":"2024-05-10T04:38:13.156596+0000","last_peered":"2024-05-10T04:38:13.156596+0000","last_clean":"2024-05-10T04:38:13.156596+0000","last_became_active":"2024-05-10T04:38:08.790539+0000","last_became_peered":"2024-05-10T04:38:08.790539+0000","last_unstale":"2024-05-10T04:38:13.156596+0000","last_undegraded":"2024-05-10T04:38:13.156596+0000","last_fullsized":"2024-05-10T04:38:13.156596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:50.174798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411137+0000","last_change":"2024-05-10T04:38:12.800901+0000","last_active":"2024-05-10T04:38:13.411137+0000","last_peered":"2024-05-10T04:38:13.411137+0000","last_clean":"2024-05-10T04:38:13.411137+0000","last_became_active":"2024-05-10T04:38:12.800287+0000","last_became_peered":"2024-05-10T04:38:12.800287+0000","last_unstale":"2024-05-10T04:38:13.411137+0000","last_undegraded":"2024-05-10T04:38:13.411137+0000","last_fullsized":"2024-05-10T04:38:13.411137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:23:25.463044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809056+0000","last_change":"2024-05-10T04:38:08.342449+0000","last_active":"2024-05-10T04:38:12.809056+0000","last_peered":"2024-05-10T04:38:12.809056+0000","last_clean":"2024-05-10T04:38:12.809056+0000","last_became_active":"2024-05-10T04:38:08.342043+0000","last_became_peered":"2024-05-10T04:38:08.342043+0000","last_unstale":"2024-05-10T04:38:12.809056+0000","last_undegraded":"2024-05-10T04:38:12.809056+0000","last_fullsized":"2024-05-10T04:38:12.809056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:31.161203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780542+0000","last_change":"2024-05-10T04:38:08.783759+0000","last_active":"2024-05-10T04:38:12.780542+0000","last_peered":"2024-05-10T04:38:12.780542+0000","last_clean":"2024-05-10T04:38:12.780542+0000","last_became_active":"2024-05-10T04:38:08.783419+0000","last_became_peered":"2024-05-10T04:38:08.783419+0000","last_unstale":"2024-05-10T04:38:12.780542+0000","last_undegraded":"2024-05-10T04:38:12.780542+0000","last_fullsized":"2024-05-10T04:38:12.780542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:16.114017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218482+0000","last_change":"2024-05-10T04:38:12.803018+0000","last_active":"2024-05-10T04:38:13.218482+0000","last_peered":"2024-05-10T04:38:13.218482+0000","last_clean":"2024-05-10T04:38:13.218482+0000","last_became_active":"2024-05-10T04:38:12.802546+0000","last_became_peered":"2024-05-10T04:38:12.802546+0000","last_unstale":"2024-05-10T04:38:13.218482+0000","last_undegraded":"2024-05-10T04:38:13.218482+0000","last_fullsized":"2024-05-10T04:38:13.218482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:40.676335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950654+0000","last_change":"2024-05-10T04:38:08.341404+0000","last_active":"2024-05-10T04:38:32.950654+0000","last_peered":"2024-05-10T04:38:32.950654+0000","last_clean":"2024-05-10T04:38:32.950654+0000","last_became_active":"2024-05-10T04:38:08.341013+0000","last_became_peered":"2024-05-10T04:38:08.341013+0000","last_unstale":"2024-05-10T04:38:32.950654+0000","last_undegraded":"2024-05-10T04:38:32.950654+0000","last_fullsized":"2024-05-10T04:38:32.950654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:43.513989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787639+0000","last_change":"2024-05-10T04:38:08.775650+0000","last_active":"2024-05-10T04:38:12.787639+0000","last_peered":"2024-05-10T04:38:12.787639+0000","last_clean":"2024-05-10T04:38:12.787639+0000","last_became_active":"2024-05-10T04:38:08.775320+0000","last_became_peered":"2024-05-10T04:38:08.775320+0000","last_unstale":"2024-05-10T04:38:12.787639+0000","last_undegraded":"2024-05-10T04:38:12.787639+0000","last_fullsized":"2024-05-10T04:38:12.787639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:49:07.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.341020+0000","last_change":"2024-05-10T04:38:12.797168+0000","last_active":"2024-05-10T04:38:13.341020+0000","last_peered":"2024-05-10T04:38:13.341020+0000","last_clean":"2024-05-10T04:38:13.341020+0000","last_became_active":"2024-05-10T04:38:12.796776+0000","last_became_peered":"2024-05-10T04:38:12.796776+0000","last_unstale":"2024-05-10T04:38:13.341020+0000","last_undegraded":"2024-05-10T04:38:13.341020+0000","last_fullsized":"2024-05-10T04:38:13.341020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:59:02.363398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963802+0000","last_change":"2024-05-10T04:38:08.344086+0000","last_active":"2024-05-10T04:38:33.963802+0000","last_peered":"2024-05-10T04:38:33.963802+0000","last_clean":"2024-05-10T04:38:33.963802+0000","last_became_active":"2024-05-10T04:38:08.343744+0000","last_became_peered":"2024-05-10T04:38:08.343744+0000","last_unstale":"2024-05-10T04:38:33.963802+0000","last_undegraded":"2024-05-10T04:38:33.963802+0000","last_fullsized":"2024-05-10T04:38:33.963802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:01.800590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802611+0000","last_change":"2024-05-10T04:38:08.773638+0000","last_active":"2024-05-10T04:38:12.802611+0000","last_peered":"2024-05-10T04:38:12.802611+0000","last_clean":"2024-05-10T04:38:12.802611+0000","last_became_active":"2024-05-10T04:38:08.773218+0000","last_became_peered":"2024-05-10T04:38:08.773218+0000","last_unstale":"2024-05-10T04:38:12.802611+0000","last_undegraded":"2024-05-10T04:38:12.802611+0000","last_fullsized":"2024-05-10T04:38:12.802611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:54:38.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217138+0000","last_change":"2024-05-10T04:38:12.796344+0000","last_active":"2024-05-10T04:38:13.217138+0000","last_peered":"2024-05-10T04:38:13.217138+0000","last_clean":"2024-05-10T04:38:13.217138+0000","last_became_active":"2024-05-10T04:38:12.795958+0000","last_became_peered":"2024-05-10T04:38:12.795958+0000","last_unstale":"2024-05-10T04:38:13.217138+0000","last_undegraded":"2024-05-10T04:38:13.217138+0000","last_fullsized":"2024-05-10T04:38:13.217138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:45.708078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809634+0000","last_change":"2024-05-10T04:38:08.341327+0000","last_active":"2024-05-10T04:38:12.809634+0000","last_peered":"2024-05-10T04:38:12.809634+0000","last_clean":"2024-05-10T04:38:12.809634+0000","last_became_active":"2024-05-10T04:38:08.339606+0000","last_became_peered":"2024-05-10T04:38:08.339606+0000","last_unstale":"2024-05-10T04:38:12.809634+0000","last_undegraded":"2024-05-10T04:38:12.809634+0000","last_fullsized":"2024-05-10T04:38:12.809634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:22:40.212212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784430+0000","last_change":"2024-05-10T04:38:08.781195+0000","last_active":"2024-05-10T04:38:12.784430+0000","last_peered":"2024-05-10T04:38:12.784430+0000","last_clean":"2024-05-10T04:38:12.784430+0000","last_became_active":"2024-05-10T04:38:08.780779+0000","last_became_peered":"2024-05-10T04:38:08.780779+0000","last_unstale":"2024-05-10T04:38:12.784430+0000","last_undegraded":"2024-05-10T04:38:12.784430+0000","last_fullsized":"2024-05-10T04:38:12.784430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:24.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930221+0000","last_change":"2024-05-10T04:38:12.799494+0000","last_active":"2024-05-10T04:38:13.930221+0000","last_peered":"2024-05-10T04:38:13.930221+0000","last_clean":"2024-05-10T04:38:13.930221+0000","last_became_active":"2024-05-10T04:38:12.799071+0000","last_became_peered":"2024-05-10T04:38:12.799071+0000","last_unstale":"2024-05-10T04:38:13.930221+0000","last_undegraded":"2024-05-10T04:38:13.930221+0000","last_fullsized":"2024-05-10T04:38:13.930221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:00.814875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.408813+0000","last_change":"2024-05-10T04:38:08.348569+0000","last_active":"2024-05-10T04:38:32.408813+0000","last_peered":"2024-05-10T04:38:32.408813+0000","last_clean":"2024-05-10T04:38:32.408813+0000","last_became_active":"2024-05-10T04:38:08.347942+0000","last_became_peered":"2024-05-10T04:38:08.347942+0000","last_unstale":"2024-05-10T04:38:32.408813+0000","last_undegraded":"2024-05-10T04:38:32.408813+0000","last_fullsized":"2024-05-10T04:38:32.408813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.865369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787640+0000","last_change":"2024-05-10T04:38:08.784447+0000","last_active":"2024-05-10T04:38:12.787640+0000","last_peered":"2024-05-10T04:38:12.787640+0000","last_clean":"2024-05-10T04:38:12.787640+0000","last_became_active":"2024-05-10T04:38:08.783989+0000","last_became_peered":"2024-05-10T04:38:08.783989+0000","last_unstale":"2024-05-10T04:38:12.787640+0000","last_undegraded":"2024-05-10T04:38:12.787640+0000","last_fullsized":"2024-05-10T04:38:12.787640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:37.511400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220978+0000","last_change":"2024-05-10T04:38:12.807478+0000","last_active":"2024-05-10T04:38:13.220978+0000","last_peered":"2024-05-10T04:38:13.220978+0000","last_clean":"2024-05-10T04:38:13.220978+0000","last_became_active":"2024-05-10T04:38:12.806801+0000","last_became_peered":"2024-05-10T04:38:12.806801+0000","last_unstale":"2024-05-10T04:38:13.220978+0000","last_undegraded":"2024-05-10T04:38:13.220978+0000","last_fullsized":"2024-05-10T04:38:13.220978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:32.168836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.943959+0000","last_change":"2024-05-10T04:38:08.343610+0000","last_active":"2024-05-10T04:38:32.943959+0000","last_peered":"2024-05-10T04:38:32.943959+0000","last_clean":"2024-05-10T04:38:32.943959+0000","last_became_active":"2024-05-10T04:38:08.342880+0000","last_became_peered":"2024-05-10T04:38:08.342880+0000","last_unstale":"2024-05-10T04:38:32.943959+0000","last_undegraded":"2024-05-10T04:38:32.943959+0000","last_fullsized":"2024-05-10T04:38:32.943959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:45:11.502258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784342+0000","last_change":"2024-05-10T04:38:08.780354+0000","last_active":"2024-05-10T04:38:12.784342+0000","last_peered":"2024-05-10T04:38:12.784342+0000","last_clean":"2024-05-10T04:38:12.784342+0000","last_became_active":"2024-05-10T04:38:08.779777+0000","last_became_peered":"2024-05-10T04:38:08.779777+0000","last_unstale":"2024-05-10T04:38:12.784342+0000","last_undegraded":"2024-05-10T04:38:12.784342+0000","last_fullsized":"2024-05-10T04:38:12.784342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:28.243474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931246+0000","last_change":"2024-05-10T04:38:12.800759+0000","last_active":"2024-05-10T04:38:13.931246+0000","last_peered":"2024-05-10T04:38:13.931246+0000","last_clean":"2024-05-10T04:38:13.931246+0000","last_became_active":"2024-05-10T04:38:12.800417+0000","last_became_peered":"2024-05-10T04:38:12.800417+0000","last_unstale":"2024-05-10T04:38:13.931246+0000","last_undegraded":"2024-05-10T04:38:13.931246+0000","last_fullsized":"2024-05-10T04:38:13.931246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:04.734817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:23.906052+0000","last_change":"2024-05-10T04:38:08.343873+0000","last_active":"2024-05-10T04:38:23.906052+0000","last_peered":"2024-05-10T04:38:23.906052+0000","last_clean":"2024-05-10T04:38:23.906052+0000","last_became_active":"2024-05-10T04:38:08.343360+0000","last_became_peered":"2024-05-10T04:38:08.343360+0000","last_unstale":"2024-05-10T04:38:23.906052+0000","last_undegraded":"2024-05-10T04:38:23.906052+0000","last_fullsized":"2024-05-10T04:38:23.906052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:06:10.816343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156650+0000","last_change":"2024-05-10T04:38:08.783955+0000","last_active":"2024-05-10T04:38:13.156650+0000","last_peered":"2024-05-10T04:38:13.156650+0000","last_clean":"2024-05-10T04:38:13.156650+0000","last_became_active":"2024-05-10T04:38:08.779131+0000","last_became_peered":"2024-05-10T04:38:08.779131+0000","last_unstale":"2024-05-10T04:38:13.156650+0000","last_undegraded":"2024-05-10T04:38:13.156650+0000","last_fullsized":"2024-05-10T04:38:13.156650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:58.619571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930607+0000","last_change":"2024-05-10T04:38:12.795140+0000","last_active":"2024-05-10T04:38:13.930607+0000","last_peered":"2024-05-10T04:38:13.930607+0000","last_clean":"2024-05-10T04:38:13.930607+0000","last_became_active":"2024-05-10T04:38:12.794666+0000","last_became_peered":"2024-05-10T04:38:12.794666+0000","last_unstale":"2024-05-10T04:38:13.930607+0000","last_undegraded":"2024-05-10T04:38:13.930607+0000","last_fullsized":"2024-05-10T04:38:13.930607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:44.468321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950141+0000","last_change":"2024-05-10T04:38:08.338866+0000","last_active":"2024-05-10T04:38:32.950141+0000","last_peered":"2024-05-10T04:38:32.950141+0000","last_clean":"2024-05-10T04:38:32.950141+0000","last_became_active":"2024-05-10T04:38:08.338434+0000","last_became_peered":"2024-05-10T04:38:08.338434+0000","last_unstale":"2024-05-10T04:38:32.950141+0000","last_undegraded":"2024-05-10T04:38:32.950141+0000","last_fullsized":"2024-05-10T04:38:32.950141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:49.858034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781395+0000","last_change":"2024-05-10T04:38:08.768998+0000","last_active":"2024-05-10T04:38:12.781395+0000","last_peered":"2024-05-10T04:38:12.781395+0000","last_clean":"2024-05-10T04:38:12.781395+0000","last_became_active":"2024-05-10T04:38:08.768687+0000","last_became_peered":"2024-05-10T04:38:08.768687+0000","last_unstale":"2024-05-10T04:38:12.781395+0000","last_undegraded":"2024-05-10T04:38:12.781395+0000","last_fullsized":"2024-05-10T04:38:12.781395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:19.390746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218211+0000","last_change":"2024-05-10T04:38:12.801223+0000","last_active":"2024-05-10T04:38:13.218211+0000","last_peered":"2024-05-10T04:38:13.218211+0000","last_clean":"2024-05-10T04:38:13.218211+0000","last_became_active":"2024-05-10T04:38:12.800750+0000","last_became_peered":"2024-05-10T04:38:12.800750+0000","last_unstale":"2024-05-10T04:38:13.218211+0000","last_undegraded":"2024-05-10T04:38:13.218211+0000","last_fullsized":"2024-05-10T04:38:13.218211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:33.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963267+0000","last_change":"2024-05-10T04:38:08.344579+0000","last_active":"2024-05-10T04:38:33.963267+0000","last_peered":"2024-05-10T04:38:33.963267+0000","last_clean":"2024-05-10T04:38:33.963267+0000","last_became_active":"2024-05-10T04:38:08.344250+0000","last_became_peered":"2024-05-10T04:38:08.344250+0000","last_unstale":"2024-05-10T04:38:33.963267+0000","last_undegraded":"2024-05-10T04:38:33.963267+0000","last_fullsized":"2024-05-10T04:38:33.963267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:56.491732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786793+0000","last_change":"2024-05-10T04:38:08.784556+0000","last_active":"2024-05-10T04:38:12.786793+0000","last_peered":"2024-05-10T04:38:12.786793+0000","last_clean":"2024-05-10T04:38:12.786793+0000","last_became_active":"2024-05-10T04:38:08.784229+0000","last_became_peered":"2024-05-10T04:38:08.784229+0000","last_unstale":"2024-05-10T04:38:12.786793+0000","last_undegraded":"2024-05-10T04:38:12.786793+0000","last_fullsized":"2024-05-10T04:38:12.786793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:48:46.519837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930516+0000","last_change":"2024-05-10T04:38:12.805795+0000","last_active":"2024-05-10T04:38:13.930516+0000","last_peered":"2024-05-10T04:38:13.930516+0000","last_clean":"2024-05-10T04:38:13.930516+0000","last_became_active":"2024-05-10T04:38:12.805366+0000","last_became_peered":"2024-05-10T04:38:12.805366+0000","last_unstale":"2024-05-10T04:38:13.930516+0000","last_undegraded":"2024-05-10T04:38:13.930516+0000","last_fullsized":"2024-05-10T04:38:13.930516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:38:11.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950525+0000","last_change":"2024-05-10T04:38:08.345396+0000","last_active":"2024-05-10T04:38:32.950525+0000","last_peered":"2024-05-10T04:38:32.950525+0000","last_clean":"2024-05-10T04:38:32.950525+0000","last_became_active":"2024-05-10T04:38:08.345076+0000","last_became_peered":"2024-05-10T04:38:08.345076+0000","last_unstale":"2024-05-10T04:38:32.950525+0000","last_undegraded":"2024-05-10T04:38:32.950525+0000","last_fullsized":"2024-05-10T04:38:32.950525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:38.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787293+0000","last_change":"2024-05-10T04:38:08.791089+0000","last_active":"2024-05-10T04:38:12.787293+0000","last_peered":"2024-05-10T04:38:12.787293+0000","last_clean":"2024-05-10T04:38:12.787293+0000","last_became_active":"2024-05-10T04:38:08.790813+0000","last_became_peered":"2024-05-10T04:38:08.790813+0000","last_unstale":"2024-05-10T04:38:12.787293+0000","last_undegraded":"2024-05-10T04:38:12.787293+0000","last_fullsized":"2024-05-10T04:38:12.787293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:52:45.753196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809384+0000","last_change":"2024-05-10T04:38:12.803896+0000","last_active":"2024-05-10T04:38:12.809384+0000","last_peered":"2024-05-10T04:38:12.809384+0000","last_clean":"2024-05-10T04:38:12.809384+0000","last_became_active":"2024-05-10T04:38:12.803512+0000","last_became_peered":"2024-05-10T04:38:12.803512+0000","last_unstale":"2024-05-10T04:38:12.809384+0000","last_undegraded":"2024-05-10T04:38:12.809384+0000","last_fullsized":"2024-05-10T04:38:12.809384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:01:23.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.952446+0000","last_change":"2024-05-10T04:38:08.348187+0000","last_active":"2024-05-10T04:38:32.952446+0000","last_peered":"2024-05-10T04:38:32.952446+0000","last_clean":"2024-05-10T04:38:32.952446+0000","last_became_active":"2024-05-10T04:38:08.346986+0000","last_became_peered":"2024-05-10T04:38:08.346986+0000","last_unstale":"2024-05-10T04:38:32.952446+0000","last_undegraded":"2024-05-10T04:38:32.952446+0000","last_fullsized":"2024-05-10T04:38:32.952446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:46.245042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780738+0000","last_change":"2024-05-10T04:38:08.782905+0000","last_active":"2024-05-10T04:38:12.780738+0000","last_peered":"2024-05-10T04:38:12.780738+0000","last_clean":"2024-05-10T04:38:12.780738+0000","last_became_active":"2024-05-10T04:38:08.782508+0000","last_became_peered":"2024-05-10T04:38:08.782508+0000","last_unstale":"2024-05-10T04:38:12.780738+0000","last_undegraded":"2024-05-10T04:38:12.780738+0000","last_fullsized":"2024-05-10T04:38:12.780738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:29.026282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930246+0000","last_change":"2024-05-10T04:38:12.796046+0000","last_active":"2024-05-10T04:38:13.930246+0000","last_peered":"2024-05-10T04:38:13.930246+0000","last_clean":"2024-05-10T04:38:13.930246+0000","last_became_active":"2024-05-10T04:38:12.795635+0000","last_became_peered":"2024-05-10T04:38:12.795635+0000","last_unstale":"2024-05-10T04:38:13.930246+0000","last_undegraded":"2024-05-10T04:38:13.930246+0000","last_fullsized":"2024-05-10T04:38:13.930246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:50.822404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.869606+0000","last_change":"2024-05-10T04:38:08.348701+0000","last_active":"2024-05-10T04:38:13.869606+0000","last_peered":"2024-05-10T04:38:13.869606+0000","last_clean":"2024-05-10T04:38:13.869606+0000","last_became_active":"2024-05-10T04:38:08.348345+0000","last_became_peered":"2024-05-10T04:38:08.348345+0000","last_unstale":"2024-05-10T04:38:13.869606+0000","last_undegraded":"2024-05-10T04:38:13.869606+0000","last_fullsized":"2024-05-10T04:38:13.869606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:31.505996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787204+0000","last_change":"2024-05-10T04:38:08.793070+0000","last_active":"2024-05-10T04:38:12.787204+0000","last_peered":"2024-05-10T04:38:12.787204+0000","last_clean":"2024-05-10T04:38:12.787204+0000","last_became_active":"2024-05-10T04:38:08.792694+0000","last_became_peered":"2024-05-10T04:38:08.792694+0000","last_unstale":"2024-05-10T04:38:12.787204+0000","last_undegraded":"2024-05-10T04:38:12.787204+0000","last_fullsized":"2024-05-10T04:38:12.787204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:23.636222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156578+0000","last_change":"2024-05-10T04:38:12.809215+0000","last_active":"2024-05-10T04:38:13.156578+0000","last_peered":"2024-05-10T04:38:13.156578+0000","last_clean":"2024-05-10T04:38:13.156578+0000","last_became_active":"2024-05-10T04:38:12.808717+0000","last_became_peered":"2024-05-10T04:38:12.808717+0000","last_unstale":"2024-05-10T04:38:13.156578+0000","last_undegraded":"2024-05-10T04:38:13.156578+0000","last_fullsized":"2024-05-10T04:38:13.156578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:40.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931223+0000","last_change":"2024-05-10T04:38:08.347770+0000","last_active":"2024-05-10T04:38:13.931223+0000","last_peered":"2024-05-10T04:38:13.931223+0000","last_clean":"2024-05-10T04:38:13.931223+0000","last_became_active":"2024-05-10T04:38:08.347444+0000","last_became_peered":"2024-05-10T04:38:08.347444+0000","last_unstale":"2024-05-10T04:38:13.931223+0000","last_undegraded":"2024-05-10T04:38:13.931223+0000","last_fullsized":"2024-05-10T04:38:13.931223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:49.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809464+0000","last_change":"2024-05-10T04:38:08.765358+0000","last_active":"2024-05-10T04:38:12.809464+0000","last_peered":"2024-05-10T04:38:12.809464+0000","last_clean":"2024-05-10T04:38:12.809464+0000","last_became_active":"2024-05-10T04:38:08.765090+0000","last_became_peered":"2024-05-10T04:38:08.765090+0000","last_unstale":"2024-05-10T04:38:12.809464+0000","last_undegraded":"2024-05-10T04:38:12.809464+0000","last_fullsized":"2024-05-10T04:38:12.809464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:55:25.722331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.798587+0000","last_change":"2024-05-10T04:38:12.794765+0000","last_active":"2024-05-10T04:38:12.798587+0000","last_peered":"2024-05-10T04:38:12.798587+0000","last_clean":"2024-05-10T04:38:12.798587+0000","last_became_active":"2024-05-10T04:38:12.794400+0000","last_became_peered":"2024-05-10T04:38:12.794400+0000","last_unstale":"2024-05-10T04:38:12.798587+0000","last_undegraded":"2024-05-10T04:38:12.798587+0000","last_fullsized":"2024-05-10T04:38:12.798587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:15:16.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930593+0000","last_change":"2024-05-10T04:38:08.348812+0000","last_active":"2024-05-10T04:38:13.930593+0000","last_peered":"2024-05-10T04:38:13.930593+0000","last_clean":"2024-05-10T04:38:13.930593+0000","last_became_active":"2024-05-10T04:38:08.348207+0000","last_became_peered":"2024-05-10T04:38:08.348207+0000","last_unstale":"2024-05-10T04:38:13.930593+0000","last_undegraded":"2024-05-10T04:38:13.930593+0000","last_fullsized":"2024-05-10T04:38:13.930593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:17.350094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809431+0000","last_change":"2024-05-10T04:38:08.785502+0000","last_active":"2024-05-10T04:38:12.809431+0000","last_peered":"2024-05-10T04:38:12.809431+0000","last_clean":"2024-05-10T04:38:12.809431+0000","last_became_active":"2024-05-10T04:38:08.785073+0000","last_became_peered":"2024-05-10T04:38:08.785073+0000","last_unstale":"2024-05-10T04:38:12.809431+0000","last_undegraded":"2024-05-10T04:38:12.809431+0000","last_fullsized":"2024-05-10T04:38:12.809431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:08:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.810257+0000","last_change":"2024-05-10T04:38:12.809599+0000","last_active":"2024-05-10T04:38:12.810257+0000","last_peered":"2024-05-10T04:38:12.810257+0000","last_clean":"2024-05-10T04:38:12.810257+0000","last_became_active":"2024-05-10T04:38:12.809200+0000","last_became_peered":"2024-05-10T04:38:12.809200+0000","last_unstale":"2024-05-10T04:38:12.810257+0000","last_undegraded":"2024-05-10T04:38:12.810257+0000","last_fullsized":"2024-05-10T04:38:12.810257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:54.940517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781249+0000","last_change":"2024-05-10T04:38:08.779837+0000","last_active":"2024-05-10T04:38:12.781249+0000","last_peered":"2024-05-10T04:38:12.781249+0000","last_clean":"2024-05-10T04:38:12.781249+0000","last_became_active":"2024-05-10T04:38:08.779501+0000","last_became_peered":"2024-05-10T04:38:08.779501+0000","last_unstale":"2024-05-10T04:38:12.781249+0000","last_undegraded":"2024-05-10T04:38:12.781249+0000","last_fullsized":"2024-05-10T04:38:12.781249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:00.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.985942+0000","last_change":"2024-05-10T04:38:08.343649+0000","last_active":"2024-05-10T04:38:35.985942+0000","last_peered":"2024-05-10T04:38:35.985942+0000","last_clean":"2024-05-10T04:38:35.985942+0000","last_became_active":"2024-05-10T04:38:08.343292+0000","last_became_peered":"2024-05-10T04:38:08.343292+0000","last_unstale":"2024-05-10T04:38:35.985942+0000","last_undegraded":"2024-05-10T04:38:35.985942+0000","last_fullsized":"2024-05-10T04:38:35.985942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:53.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930867+0000","last_change":"2024-05-10T04:38:12.796552+0000","last_active":"2024-05-10T04:38:13.930867+0000","last_peered":"2024-05-10T04:38:13.930867+0000","last_clean":"2024-05-10T04:38:13.930867+0000","last_became_active":"2024-05-10T04:38:12.796151+0000","last_became_peered":"2024-05-10T04:38:12.796151+0000","last_unstale":"2024-05-10T04:38:13.930867+0000","last_undegraded":"2024-05-10T04:38:13.930867+0000","last_fullsized":"2024-05-10T04:38:13.930867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:05.520859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787088+0000","last_change":"2024-05-10T04:38:08.770957+0000","last_active":"2024-05-10T04:38:12.787088+0000","last_peered":"2024-05-10T04:38:12.787088+0000","last_clean":"2024-05-10T04:38:12.787088+0000","last_became_active":"2024-05-10T04:38:08.770572+0000","last_became_peered":"2024-05-10T04:38:08.770572+0000","last_unstale":"2024-05-10T04:38:12.787088+0000","last_undegraded":"2024-05-10T04:38:12.787088+0000","last_fullsized":"2024-05-10T04:38:12.787088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:21.421033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.940410+0000","last_change":"2024-05-10T04:38:08.340788+0000","last_active":"2024-05-10T04:38:32.940410+0000","last_peered":"2024-05-10T04:38:32.940410+0000","last_clean":"2024-05-10T04:38:32.940410+0000","last_became_active":"2024-05-10T04:38:08.340407+0000","last_became_peered":"2024-05-10T04:38:08.340407+0000","last_unstale":"2024-05-10T04:38:32.940410+0000","last_undegraded":"2024-05-10T04:38:32.940410+0000","last_fullsized":"2024-05-10T04:38:32.940410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:30:05.276951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930055+0000","last_change":"2024-05-10T04:37:20.961282+0000","last_active":"2024-05-10T04:38:13.930055+0000","last_peered":"2024-05-10T04:38:13.930055+0000","last_clean":"2024-05-10T04:38:13.930055+0000","last_became_active":"2024-05-10T04:37:20.960828+0000","last_became_peered":"2024-05-10T04:37:20.960828+0000","last_unstale":"2024-05-10T04:38:13.930055+0000","last_undegraded":"2024-05-10T04:38:13.930055+0000","last_fullsized":"2024-05-10T04:38:13.930055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_clean_scrub_stamp":"2024-05-10T04:35:00.248380+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:08:04.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930338+0000","last_change":"2024-05-10T04:38:12.809899+0000","last_active":"2024-05-10T04:38:13.930338+0000","last_peered":"2024-05-10T04:38:13.930338+0000","last_clean":"2024-05-10T04:38:13.930338+0000","last_became_active":"2024-05-10T04:38:12.809441+0000","last_became_peered":"2024-05-10T04:38:12.809441+0000","last_unstale":"2024-05-10T04:38:13.930338+0000","last_undegraded":"2024-05-10T04:38:13.930338+0000","last_fullsized":"2024-05-10T04:38:13.930338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:48.378890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781228+0000","last_change":"2024-05-10T04:38:08.784838+0000","last_active":"2024-05-10T04:38:12.781228+0000","last_peered":"2024-05-10T04:38:12.781228+0000","last_clean":"2024-05-10T04:38:12.781228+0000","last_became_active":"2024-05-10T04:38:08.784464+0000","last_became_peered":"2024-05-10T04:38:08.784464+0000","last_unstale":"2024-05-10T04:38:12.781228+0000","last_undegraded":"2024-05-10T04:38:12.781228+0000","last_fullsized":"2024-05-10T04:38:12.781228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:00.335350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.952966+0000","last_change":"2024-05-10T04:38:08.348639+0000","last_active":"2024-05-10T04:38:32.952966+0000","last_peered":"2024-05-10T04:38:32.952966+0000","last_clean":"2024-05-10T04:38:32.952966+0000","last_became_active":"2024-05-10T04:38:08.348271+0000","last_became_peered":"2024-05-10T04:38:08.348271+0000","last_unstale":"2024-05-10T04:38:32.952966+0000","last_undegraded":"2024-05-10T04:38:32.952966+0000","last_fullsized":"2024-05-10T04:38:32.952966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:18.486150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218189+0000","last_change":"2024-05-10T04:38:12.801079+0000","last_active":"2024-05-10T04:38:13.218189+0000","last_peered":"2024-05-10T04:38:13.218189+0000","last_clean":"2024-05-10T04:38:13.218189+0000","last_became_active":"2024-05-10T04:38:12.800630+0000","last_became_peered":"2024-05-10T04:38:12.800630+0000","last_unstale":"2024-05-10T04:38:13.218189+0000","last_undegraded":"2024-05-10T04:38:13.218189+0000","last_fullsized":"2024-05-10T04:38:13.218189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:05.213620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.408355+0000","last_change":"2024-05-10T04:38:08.348764+0000","last_active":"2024-05-10T04:38:32.408355+0000","last_peered":"2024-05-10T04:38:32.408355+0000","last_clean":"2024-05-10T04:38:32.408355+0000","last_became_active":"2024-05-10T04:38:08.347939+0000","last_became_peered":"2024-05-10T04:38:08.347939+0000","last_unstale":"2024-05-10T04:38:32.408355+0000","last_undegraded":"2024-05-10T04:38:32.408355+0000","last_fullsized":"2024-05-10T04:38:32.408355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:01.308255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782312+0000","last_change":"2024-05-10T04:38:08.771382+0000","last_active":"2024-05-10T04:38:12.782312+0000","last_peered":"2024-05-10T04:38:12.782312+0000","last_clean":"2024-05-10T04:38:12.782312+0000","last_became_active":"2024-05-10T04:38:08.771108+0000","last_became_peered":"2024-05-10T04:38:08.771108+0000","last_unstale":"2024-05-10T04:38:12.782312+0000","last_undegraded":"2024-05-10T04:38:12.782312+0000","last_fullsized":"2024-05-10T04:38:12.782312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:32.664640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217051+0000","last_change":"2024-05-10T04:38:12.809582+0000","last_active":"2024-05-10T04:38:13.217051+0000","last_peered":"2024-05-10T04:38:13.217051+0000","last_clean":"2024-05-10T04:38:13.217051+0000","last_became_active":"2024-05-10T04:38:12.809134+0000","last_became_peered":"2024-05-10T04:38:12.809134+0000","last_unstale":"2024-05-10T04:38:13.217051+0000","last_undegraded":"2024-05-10T04:38:13.217051+0000","last_fullsized":"2024-05-10T04:38:13.217051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:31:07.694368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.965023+0000","last_change":"2024-05-10T04:38:08.345903+0000","last_active":"2024-05-10T04:38:33.965023+0000","last_peered":"2024-05-10T04:38:33.965023+0000","last_clean":"2024-05-10T04:38:33.965023+0000","last_became_active":"2024-05-10T04:38:08.345371+0000","last_became_peered":"2024-05-10T04:38:08.345371+0000","last_unstale":"2024-05-10T04:38:33.965023+0000","last_undegraded":"2024-05-10T04:38:33.965023+0000","last_fullsized":"2024-05-10T04:38:33.965023+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:03.581112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789703+0000","last_change":"2024-05-10T04:38:08.779853+0000","last_active":"2024-05-10T04:38:12.789703+0000","last_peered":"2024-05-10T04:38:12.789703+0000","last_clean":"2024-05-10T04:38:12.789703+0000","last_became_active":"2024-05-10T04:38:08.779531+0000","last_became_peered":"2024-05-10T04:38:08.779531+0000","last_unstale":"2024-05-10T04:38:12.789703+0000","last_undegraded":"2024-05-10T04:38:12.789703+0000","last_fullsized":"2024-05-10T04:38:12.789703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:05.079540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410643+0000","last_change":"2024-05-10T04:38:12.795970+0000","last_active":"2024-05-10T04:38:13.410643+0000","last_peered":"2024-05-10T04:38:13.410643+0000","last_clean":"2024-05-10T04:38:13.410643+0000","last_became_active":"2024-05-10T04:38:12.795497+0000","last_became_peered":"2024-05-10T04:38:12.795497+0000","last_unstale":"2024-05-10T04:38:13.410643+0000","last_undegraded":"2024-05-10T04:38:13.410643+0000","last_fullsized":"2024-05-10T04:38:13.410643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:11.174672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.409250+0000","last_change":"2024-05-10T04:38:08.343268+0000","last_active":"2024-05-10T04:38:32.409250+0000","last_peered":"2024-05-10T04:38:32.409250+0000","last_clean":"2024-05-10T04:38:32.409250+0000","last_became_active":"2024-05-10T04:38:08.342952+0000","last_became_peered":"2024-05-10T04:38:08.342952+0000","last_unstale":"2024-05-10T04:38:32.409250+0000","last_undegraded":"2024-05-10T04:38:32.409250+0000","last_fullsized":"2024-05-10T04:38:32.409250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:49.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788390+0000","last_change":"2024-05-10T04:38:08.784276+0000","last_active":"2024-05-10T04:38:12.788390+0000","last_peered":"2024-05-10T04:38:12.788390+0000","last_clean":"2024-05-10T04:38:12.788390+0000","last_became_active":"2024-05-10T04:38:08.783951+0000","last_became_peered":"2024-05-10T04:38:08.783951+0000","last_unstale":"2024-05-10T04:38:12.788390+0000","last_undegraded":"2024-05-10T04:38:12.788390+0000","last_fullsized":"2024-05-10T04:38:12.788390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:45:43.992567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220551+0000","last_change":"2024-05-10T04:38:12.811689+0000","last_active":"2024-05-10T04:38:13.220551+0000","last_peered":"2024-05-10T04:38:13.220551+0000","last_clean":"2024-05-10T04:38:13.220551+0000","last_became_active":"2024-05-10T04:38:12.811138+0000","last_became_peered":"2024-05-10T04:38:12.811138+0000","last_unstale":"2024-05-10T04:38:13.220551+0000","last_undegraded":"2024-05-10T04:38:13.220551+0000","last_fullsized":"2024-05-10T04:38:13.220551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:09.291137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410146+0000","last_change":"2024-05-10T04:38:08.348738+0000","last_active":"2024-05-10T04:38:32.410146+0000","last_peered":"2024-05-10T04:38:32.410146+0000","last_clean":"2024-05-10T04:38:32.410146+0000","last_became_active":"2024-05-10T04:38:08.348382+0000","last_became_peered":"2024-05-10T04:38:08.348382+0000","last_unstale":"2024-05-10T04:38:32.410146+0000","last_undegraded":"2024-05-10T04:38:32.410146+0000","last_fullsized":"2024-05-10T04:38:32.410146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:02.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156551+0000","last_change":"2024-05-10T04:38:08.781483+0000","last_active":"2024-05-10T04:38:13.156551+0000","last_peered":"2024-05-10T04:38:13.156551+0000","last_clean":"2024-05-10T04:38:13.156551+0000","last_became_active":"2024-05-10T04:38:08.777713+0000","last_became_peered":"2024-05-10T04:38:08.777713+0000","last_unstale":"2024-05-10T04:38:13.156551+0000","last_undegraded":"2024-05-10T04:38:13.156551+0000","last_fullsized":"2024-05-10T04:38:13.156551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:41.050901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930627+0000","last_change":"2024-05-10T04:38:12.800656+0000","last_active":"2024-05-10T04:38:13.930627+0000","last_peered":"2024-05-10T04:38:13.930627+0000","last_clean":"2024-05-10T04:38:13.930627+0000","last_became_active":"2024-05-10T04:38:12.800191+0000","last_became_peered":"2024-05-10T04:38:12.800191+0000","last_unstale":"2024-05-10T04:38:13.930627+0000","last_undegraded":"2024-05-10T04:38:13.930627+0000","last_fullsized":"2024-05-10T04:38:13.930627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:47.017642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968707+0000","last_change":"2024-05-10T04:38:08.344017+0000","last_active":"2024-05-10T04:38:34.968707+0000","last_peered":"2024-05-10T04:38:34.968707+0000","last_clean":"2024-05-10T04:38:34.968707+0000","last_became_active":"2024-05-10T04:38:08.343646+0000","last_became_peered":"2024-05-10T04:38:08.343646+0000","last_unstale":"2024-05-10T04:38:34.968707+0000","last_undegraded":"2024-05-10T04:38:34.968707+0000","last_fullsized":"2024-05-10T04:38:34.968707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:02:30.455019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786221+0000","last_change":"2024-05-10T04:38:08.779065+0000","last_active":"2024-05-10T04:38:12.786221+0000","last_peered":"2024-05-10T04:38:12.786221+0000","last_clean":"2024-05-10T04:38:12.786221+0000","last_became_active":"2024-05-10T04:38:08.778685+0000","last_became_peered":"2024-05-10T04:38:08.778685+0000","last_unstale":"2024-05-10T04:38:12.786221+0000","last_undegraded":"2024-05-10T04:38:12.786221+0000","last_fullsized":"2024-05-10T04:38:12.786221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:25:58.780080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220710+0000","last_change":"2024-05-10T04:38:12.801524+0000","last_active":"2024-05-10T04:38:13.220710+0000","last_peered":"2024-05-10T04:38:13.220710+0000","last_clean":"2024-05-10T04:38:13.220710+0000","last_became_active":"2024-05-10T04:38:12.800777+0000","last_became_peered":"2024-05-10T04:38:12.800777+0000","last_unstale":"2024-05-10T04:38:13.220710+0000","last_undegraded":"2024-05-10T04:38:13.220710+0000","last_fullsized":"2024-05-10T04:38:13.220710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:14.139236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787348+0000","last_change":"2024-05-10T04:38:08.344785+0000","last_active":"2024-05-10T04:38:12.787348+0000","last_peered":"2024-05-10T04:38:12.787348+0000","last_clean":"2024-05-10T04:38:12.787348+0000","last_became_active":"2024-05-10T04:38:08.344337+0000","last_became_peered":"2024-05-10T04:38:08.344337+0000","last_unstale":"2024-05-10T04:38:12.787348+0000","last_undegraded":"2024-05-10T04:38:12.787348+0000","last_fullsized":"2024-05-10T04:38:12.787348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:47.682532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788214+0000","last_change":"2024-05-10T04:38:08.768686+0000","last_active":"2024-05-10T04:38:12.788214+0000","last_peered":"2024-05-10T04:38:12.788214+0000","last_clean":"2024-05-10T04:38:12.788214+0000","last_became_active":"2024-05-10T04:38:08.768317+0000","last_became_peered":"2024-05-10T04:38:08.768317+0000","last_unstale":"2024-05-10T04:38:12.788214+0000","last_undegraded":"2024-05-10T04:38:12.788214+0000","last_fullsized":"2024-05-10T04:38:12.788214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:42:33.670912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220479+0000","last_change":"2024-05-10T04:38:12.809825+0000","last_active":"2024-05-10T04:38:13.220479+0000","last_peered":"2024-05-10T04:38:13.220479+0000","last_clean":"2024-05-10T04:38:13.220479+0000","last_became_active":"2024-05-10T04:38:12.809378+0000","last_became_peered":"2024-05-10T04:38:12.809378+0000","last_unstale":"2024-05-10T04:38:13.220479+0000","last_undegraded":"2024-05-10T04:38:13.220479+0000","last_fullsized":"2024-05-10T04:38:13.220479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:28.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410164+0000","last_change":"2024-05-10T04:38:08.347871+0000","last_active":"2024-05-10T04:38:32.410164+0000","last_peered":"2024-05-10T04:38:32.410164+0000","last_clean":"2024-05-10T04:38:32.410164+0000","last_became_active":"2024-05-10T04:38:08.347618+0000","last_became_peered":"2024-05-10T04:38:08.347618+0000","last_unstale":"2024-05-10T04:38:32.410164+0000","last_undegraded":"2024-05-10T04:38:32.410164+0000","last_fullsized":"2024-05-10T04:38:32.410164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:09:42.060082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787115+0000","last_change":"2024-05-10T04:38:08.779191+0000","last_active":"2024-05-10T04:38:12.787115+0000","last_peered":"2024-05-10T04:38:12.787115+0000","last_clean":"2024-05-10T04:38:12.787115+0000","last_became_active":"2024-05-10T04:38:08.778829+0000","last_became_peered":"2024-05-10T04:38:08.778829+0000","last_unstale":"2024-05-10T04:38:12.787115+0000","last_undegraded":"2024-05-10T04:38:12.787115+0000","last_fullsized":"2024-05-10T04:38:12.787115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:07.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931626+0000","last_change":"2024-05-10T04:38:12.788693+0000","last_active":"2024-05-10T04:38:13.931626+0000","last_peered":"2024-05-10T04:38:13.931626+0000","last_clean":"2024-05-10T04:38:13.931626+0000","last_became_active":"2024-05-10T04:38:12.788156+0000","last_became_peered":"2024-05-10T04:38:12.788156+0000","last_unstale":"2024-05-10T04:38:13.931626+0000","last_undegraded":"2024-05-10T04:38:13.931626+0000","last_fullsized":"2024-05-10T04:38:13.931626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:30.119523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950714+0000","last_change":"2024-05-10T04:38:08.344211+0000","last_active":"2024-05-10T04:38:32.950714+0000","last_peered":"2024-05-10T04:38:32.950714+0000","last_clean":"2024-05-10T04:38:32.950714+0000","last_became_active":"2024-05-10T04:38:08.343443+0000","last_became_peered":"2024-05-10T04:38:08.343443+0000","last_unstale":"2024-05-10T04:38:32.950714+0000","last_undegraded":"2024-05-10T04:38:32.950714+0000","last_fullsized":"2024-05-10T04:38:32.950714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:33:06.960942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787186+0000","last_change":"2024-05-10T04:38:08.780352+0000","last_active":"2024-05-10T04:38:12.787186+0000","last_peered":"2024-05-10T04:38:12.787186+0000","last_clean":"2024-05-10T04:38:12.787186+0000","last_became_active":"2024-05-10T04:38:08.779779+0000","last_became_peered":"2024-05-10T04:38:08.779779+0000","last_unstale":"2024-05-10T04:38:12.787186+0000","last_undegraded":"2024-05-10T04:38:12.787186+0000","last_fullsized":"2024-05-10T04:38:12.787186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:54:31.973896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.340967+0000","last_change":"2024-05-10T04:38:12.805310+0000","last_active":"2024-05-10T04:38:13.340967+0000","last_peered":"2024-05-10T04:38:13.340967+0000","last_clean":"2024-05-10T04:38:13.340967+0000","last_became_active":"2024-05-10T04:38:12.804827+0000","last_became_peered":"2024-05-10T04:38:12.804827+0000","last_unstale":"2024-05-10T04:38:13.340967+0000","last_undegraded":"2024-05-10T04:38:13.340967+0000","last_fullsized":"2024-05-10T04:38:13.340967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:01.655788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963212+0000","last_change":"2024-05-10T04:38:08.343813+0000","last_active":"2024-05-10T04:38:33.963212+0000","last_peered":"2024-05-10T04:38:33.963212+0000","last_clean":"2024-05-10T04:38:33.963212+0000","last_became_active":"2024-05-10T04:38:08.343309+0000","last_became_peered":"2024-05-10T04:38:08.343309+0000","last_unstale":"2024-05-10T04:38:33.963212+0000","last_undegraded":"2024-05-10T04:38:33.963212+0000","last_fullsized":"2024-05-10T04:38:33.963212+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:07.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786851+0000","last_change":"2024-05-10T04:38:08.780598+0000","last_active":"2024-05-10T04:38:12.786851+0000","last_peered":"2024-05-10T04:38:12.786851+0000","last_clean":"2024-05-10T04:38:12.786851+0000","last_became_active":"2024-05-10T04:38:08.775367+0000","last_became_peered":"2024-05-10T04:38:08.775367+0000","last_unstale":"2024-05-10T04:38:12.786851+0000","last_undegraded":"2024-05-10T04:38:12.786851+0000","last_fullsized":"2024-05-10T04:38:12.786851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:33.826116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.929645+0000","last_change":"2024-05-10T04:38:12.795975+0000","last_active":"2024-05-10T04:38:13.929645+0000","last_peered":"2024-05-10T04:38:13.929645+0000","last_clean":"2024-05-10T04:38:13.929645+0000","last_became_active":"2024-05-10T04:38:12.795582+0000","last_became_peered":"2024-05-10T04:38:12.795582+0000","last_unstale":"2024-05-10T04:38:13.929645+0000","last_undegraded":"2024-05-10T04:38:13.929645+0000","last_fullsized":"2024-05-10T04:38:13.929645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:42:30.734833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.969664+0000","last_change":"2024-05-10T04:38:08.346338+0000","last_active":"2024-05-10T04:38:34.969664+0000","last_peered":"2024-05-10T04:38:34.969664+0000","last_clean":"2024-05-10T04:38:34.969664+0000","last_became_active":"2024-05-10T04:38:08.345994+0000","last_became_peered":"2024-05-10T04:38:08.345994+0000","last_unstale":"2024-05-10T04:38:34.969664+0000","last_undegraded":"2024-05-10T04:38:34.969664+0000","last_fullsized":"2024-05-10T04:38:34.969664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:22.962514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787140+0000","last_change":"2024-05-10T04:38:08.780925+0000","last_active":"2024-05-10T04:38:12.787140+0000","last_peered":"2024-05-10T04:38:12.787140+0000","last_clean":"2024-05-10T04:38:12.787140+0000","last_became_active":"2024-05-10T04:38:08.778777+0000","last_became_peered":"2024-05-10T04:38:08.778777+0000","last_unstale":"2024-05-10T04:38:12.787140+0000","last_undegraded":"2024-05-10T04:38:12.787140+0000","last_fullsized":"2024-05-10T04:38:12.787140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:57.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786327+0000","last_change":"2024-05-10T04:38:08.780281+0000","last_active":"2024-05-10T04:38:12.786327+0000","last_peered":"2024-05-10T04:38:12.786327+0000","last_clean":"2024-05-10T04:38:12.786327+0000","last_became_active":"2024-05-10T04:38:08.779927+0000","last_became_peered":"2024-05-10T04:38:08.779927+0000","last_unstale":"2024-05-10T04:38:12.786327+0000","last_undegraded":"2024-05-10T04:38:12.786327+0000","last_fullsized":"2024-05-10T04:38:12.786327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:33.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789688+0000","last_change":"2024-05-10T04:38:08.784916+0000","last_active":"2024-05-10T04:38:12.789688+0000","last_peered":"2024-05-10T04:38:12.789688+0000","last_clean":"2024-05-10T04:38:12.789688+0000","last_became_active":"2024-05-10T04:38:08.784544+0000","last_became_peered":"2024-05-10T04:38:08.784544+0000","last_unstale":"2024-05-10T04:38:12.789688+0000","last_undegraded":"2024-05-10T04:38:12.789688+0000","last_fullsized":"2024-05-10T04:38:12.789688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:03.669897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786689+0000","last_change":"2024-05-10T04:38:08.769149+0000","last_active":"2024-05-10T04:38:12.786689+0000","last_peered":"2024-05-10T04:38:12.786689+0000","last_clean":"2024-05-10T04:38:12.786689+0000","last_became_active":"2024-05-10T04:38:08.768808+0000","last_became_peered":"2024-05-10T04:38:08.768808+0000","last_unstale":"2024-05-10T04:38:12.786689+0000","last_undegraded":"2024-05-10T04:38:12.786689+0000","last_fullsized":"2024-05-10T04:38:12.786689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:42:46.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787125+0000","last_change":"2024-05-10T04:38:08.784411+0000","last_active":"2024-05-10T04:38:12.787125+0000","last_peered":"2024-05-10T04:38:12.787125+0000","last_clean":"2024-05-10T04:38:12.787125+0000","last_became_active":"2024-05-10T04:38:08.784064+0000","last_became_peered":"2024-05-10T04:38:08.784064+0000","last_unstale":"2024-05-10T04:38:12.787125+0000","last_undegraded":"2024-05-10T04:38:12.787125+0000","last_fullsized":"2024-05-10T04:38:12.787125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:32:46.493888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782409+0000","last_change":"2024-05-10T04:38:08.779643+0000","last_active":"2024-05-10T04:38:12.782409+0000","last_peered":"2024-05-10T04:38:12.782409+0000","last_clean":"2024-05-10T04:38:12.782409+0000","last_became_active":"2024-05-10T04:38:08.779291+0000","last_became_peered":"2024-05-10T04:38:08.779291+0000","last_unstale":"2024-05-10T04:38:12.782409+0000","last_undegraded":"2024-05-10T04:38:12.782409+0000","last_fullsized":"2024-05-10T04:38:12.782409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:24.535515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786192+0000","last_change":"2024-05-10T04:38:08.789382+0000","last_active":"2024-05-10T04:38:12.786192+0000","last_peered":"2024-05-10T04:38:12.786192+0000","last_clean":"2024-05-10T04:38:12.786192+0000","last_became_active":"2024-05-10T04:38:08.789003+0000","last_became_peered":"2024-05-10T04:38:08.789003+0000","last_unstale":"2024-05-10T04:38:12.786192+0000","last_undegraded":"2024-05-10T04:38:12.786192+0000","last_fullsized":"2024-05-10T04:38:12.786192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:03.031319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788582+0000","last_change":"2024-05-10T04:38:08.777290+0000","last_active":"2024-05-10T04:38:12.788582+0000","last_peered":"2024-05-10T04:38:12.788582+0000","last_clean":"2024-05-10T04:38:12.788582+0000","last_became_active":"2024-05-10T04:38:08.776331+0000","last_became_peered":"2024-05-10T04:38:08.776331+0000","last_unstale":"2024-05-10T04:38:12.788582+0000","last_undegraded":"2024-05-10T04:38:12.788582+0000","last_fullsized":"2024-05-10T04:38:12.788582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:51.074991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789825+0000","last_change":"2024-05-10T04:38:08.780937+0000","last_active":"2024-05-10T04:38:12.789825+0000","last_peered":"2024-05-10T04:38:12.789825+0000","last_clean":"2024-05-10T04:38:12.789825+0000","last_became_active":"2024-05-10T04:38:08.780620+0000","last_became_peered":"2024-05-10T04:38:08.780620+0000","last_unstale":"2024-05-10T04:38:12.789825+0000","last_undegraded":"2024-05-10T04:38:12.789825+0000","last_fullsized":"2024-05-10T04:38:12.789825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:27.697571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781208+0000","last_change":"2024-05-10T04:38:08.773314+0000","last_active":"2024-05-10T04:38:12.781208+0000","last_peered":"2024-05-10T04:38:12.781208+0000","last_clean":"2024-05-10T04:38:12.781208+0000","last_became_active":"2024-05-10T04:38:08.772973+0000","last_became_peered":"2024-05-10T04:38:08.772973+0000","last_unstale":"2024-05-10T04:38:12.781208+0000","last_undegraded":"2024-05-10T04:38:12.781208+0000","last_fullsized":"2024-05-10T04:38:12.781208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:53.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788890+0000","last_change":"2024-05-10T04:38:08.775632+0000","last_active":"2024-05-10T04:38:12.788890+0000","last_peered":"2024-05-10T04:38:12.788890+0000","last_clean":"2024-05-10T04:38:12.788890+0000","last_became_active":"2024-05-10T04:38:08.775308+0000","last_became_peered":"2024-05-10T04:38:08.775308+0000","last_unstale":"2024-05-10T04:38:12.788890+0000","last_undegraded":"2024-05-10T04:38:12.788890+0000","last_fullsized":"2024-05-10T04:38:12.788890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:58.758863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783887+0000","last_change":"2024-05-10T04:38:08.787354+0000","last_active":"2024-05-10T04:38:12.783887+0000","last_peered":"2024-05-10T04:38:12.783887+0000","last_clean":"2024-05-10T04:38:12.783887+0000","last_became_active":"2024-05-10T04:38:08.786645+0000","last_became_peered":"2024-05-10T04:38:08.786645+0000","last_unstale":"2024-05-10T04:38:12.783887+0000","last_undegraded":"2024-05-10T04:38:12.783887+0000","last_fullsized":"2024-05-10T04:38:12.783887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:35.060271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782372+0000","last_change":"2024-05-10T04:38:08.769233+0000","last_active":"2024-05-10T04:38:12.782372+0000","last_peered":"2024-05-10T04:38:12.782372+0000","last_clean":"2024-05-10T04:38:12.782372+0000","last_became_active":"2024-05-10T04:38:08.768936+0000","last_became_peered":"2024-05-10T04:38:08.768936+0000","last_unstale":"2024-05-10T04:38:12.782372+0000","last_undegraded":"2024-05-10T04:38:12.782372+0000","last_fullsized":"2024-05-10T04:38:12.782372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:08.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786960+0000","last_change":"2024-05-10T04:38:08.788189+0000","last_active":"2024-05-10T04:38:12.786960+0000","last_peered":"2024-05-10T04:38:12.786960+0000","last_clean":"2024-05-10T04:38:12.786960+0000","last_became_active":"2024-05-10T04:38:08.787806+0000","last_became_peered":"2024-05-10T04:38:08.787806+0000","last_unstale":"2024-05-10T04:38:12.786960+0000","last_undegraded":"2024-05-10T04:38:12.786960+0000","last_fullsized":"2024-05-10T04:38:12.786960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:00:51.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280050,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8004,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":144,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":45604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481807,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":942080,"data_stored":466157,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.77700000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.749}]},{"osd":7,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.90100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.86099999999999999}]},{"osd":10,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.92700000000000005}]}]},{"osd":10,"up_from":69,"seq":296352743441,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1054551,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.501},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.751},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55600000000000005}]},{"osd":5,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.73199999999999998}]},{"osd":9,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906966,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":461432,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.44}]},{"osd":3,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.372},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.41699999999999998}]},{"osd":4,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.65300000000000002}]},{"osd":8,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.501}]},{"osd":10,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.54700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135897,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":937984,"data_stored":461150,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.377}]},{"osd":3,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63900000000000001}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":999424,"data_stored":1053260,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.532,"5min":3.532,"15min":3.532},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69399999999999995,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.3759999999999999,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.60799999999999998,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.625,"15min":0.59899999999999998},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.9609999999999999,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6179999999999999,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.72899999999999998,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4860000000000002,"5min":3.4860000000000002,"15min":3.4860000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.70699999999999996,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":3.996,"5min":3.996,"15min":3.996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.47199999999999998}]},{"osd":8,"last update":"Fri May 10 04:37:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.499}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.2670000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":461198,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.624},{"interface":"front","average":{"1min":0.626,"5min":0.63,"15min":0.63},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri May 10 04:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri May 10 04:37:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65000000000000002}]},{"osd":9,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.51100000000000001}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.72199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":958464,"data_stored":472626,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.47899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.85999999999999999}]},{"osd":3,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri May 10 04:38:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46100000000000002,"15min":0.438},"min":{"1min":0.33200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.47999999999999998,"15min":0.44700000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri May 10 04:37:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.51600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.53500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.96399999999999997}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.008,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.879}]},{"osd":8,"last update":"Fri May 10 04:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.018}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.048},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.0049999999999999}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.755},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.073}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":461218,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.503,"15min":0.47299999999999998},"min":{"1min":0.379,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83299999999999996,"5min":0.996,"15min":0.996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.5,"15min":0.46800000000000003},"min":{"1min":0.40300000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71099999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri May 10 04:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.68500000000000005,"5min":1.123,"15min":1.123},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.65100000000000002}]},{"osd":3,"last update":"Fri May 10 04:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.623},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85899999999999999,"5min":1.093,"15min":1.093},"last":1.0089999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81000000000000005,"5min":1.151,"15min":1.151},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.86699999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:07 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.95699999999999996}]},{"osd":7,"last update":"Fri May 10 04:37:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.997}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":1.181},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.0389999999999999}]},{"osd":9,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.131}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":1.075},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.026}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":464928,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.502,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.47999999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.67800000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80000000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.85799999999999998}]},{"osd":4,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64200000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri May 10 04:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri May 10 04:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.90900000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.873}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":460932,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35799999999999998,"15min":0.34000000000000002},"min":{"1min":0.31900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri May 10 04:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.626},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.48599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri May 10 04:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78300000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri May 10 04:38:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69599999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96799999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri May 10 04:38:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46700000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":4,"up_from":33,"seq":141733920807,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":460546,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":1.143,"15min":1.143},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.35699999999999998},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.51300000000000001,"15min":0.5},"min":{"1min":0.41299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.53800000000000003,"15min":0.52100000000000002},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92600000000000005,"5min":1.03,"15min":1.03},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0620000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67200000000000004,"5min":1.069,"15min":1.069},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":1.079,"15min":1.079},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.71599999999999997}]},{"osd":9,"last update":"Fri May 10 04:38:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78700000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757284,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1053082,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86199999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.44700000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.28299999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86699999999999999,"5min":1.089,"15min":1.089},"last":0.626},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.749}]},{"osd":6,"last update":"Fri May 10 04:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.48399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66800000000000004}]},{"osd":8,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68600000000000005}]},{"osd":9,"last update":"Fri May 10 04:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:38:37.802 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:38:37.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:37 smithi161 ceph-mon[46912]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:37.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:37 smithi161 ceph-mon[46912]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:37.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:37 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1289182560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 -- 172.21.15.150:0/2408813564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 msgr2=0x7fdab4109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 --2- 172.21.15.150:0/2408813564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab4109200 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00b0a0 tx=0x7fdaac00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 -- 172.21.15.150:0/2408813564 shutdown_connections 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 --2- 172.21.15.150:0/2408813564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab4109200 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 --2- 172.21.15.150:0/2408813564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdab41010c0 0x7fdab41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 --2- 172.21.15.150:0/2408813564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab4101fb0 0x7fdab41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 -- 172.21.15.150:0/2408813564 >> 172.21.15.150:0/2408813564 conn(0x7fdab40fb880 msgr2=0x7fdab40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 -- 172.21.15.150:0/2408813564 shutdown_connections 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.964+0000 7fdabb680700 1 -- 172.21.15.150:0/2408813564 wait complete. 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 Processor -- start 2024-05-10T04:38:37.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 -- start start 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab41010c0 0x7fdab4100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdab4101fb0 0x7fdab419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab41a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab41a1f80 con 0x7fdab4102900 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab41a20f0 con 0x7fdab41010c0 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdabb680700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab41a22b0 con 0x7fdab4101fb0 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdab9c1d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab41a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab941c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab41010c0 0x7fdab4100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.965+0000 7fdab9c1d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab41a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57204/0 (socket says 172.21.15.150:57204) 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab9c1d700 1 -- 172.21.15.150:0/3484884166 learned_addr learned my addr 172.21.15.150:0/3484884166 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab8c1b700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdab4101fb0 0x7fdab419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab9c1d700 1 -- 172.21.15.150:0/3484884166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdab4101fb0 msgr2=0x7fdab419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab9c1d700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdab4101fb0 0x7fdab419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab9c1d700 1 -- 172.21.15.150:0/3484884166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab41010c0 msgr2=0x7fdab4100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab9c1d700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab41010c0 0x7fdab4100780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab9c1d700 1 -- 172.21.15.150:0/3484884166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac009d00 con 0x7fdab4102900 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdab9c1d700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab41a19b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdaac006070 tx=0x7fdaac004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdaaa7fc700 1 -- 172.21.15.150:0/3484884166 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00e040 con 0x7fdab4102900 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdaaa7fc700 1 -- 172.21.15.150:0/3484884166 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaac00bda0 con 0x7fdab4102900 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab41a2530 con 0x7fdab4102900 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdaaa7fc700 1 -- 172.21.15.150:0/3484884166 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac01ba00 con 0x7fdab4102900 2024-05-10T04:38:37.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.966+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab41a29b0 con 0x7fdab4102900 2024-05-10T04:38:37.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.967+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab41080a0 con 0x7fdab4102900 2024-05-10T04:38:37.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.968+0000 7fdaaa7fc700 1 -- 172.21.15.150:0/3484884166 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdaac01dc50 con 0x7fdab4102900 2024-05-10T04:38:37.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.969+0000 7fdaaa7fc700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdaa00a1230 0x7fdaa00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:37.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.969+0000 7fdab941c700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdaa00a1230 0x7fdaa00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:37.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.969+0000 7fdaaa7fc700 1 -- 172.21.15.150:0/3484884166 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdaac0ce950 con 0x7fdab4102900 2024-05-10T04:38:37.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.970+0000 7fdab941c700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdaa00a1230 0x7fdaa00a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdab4101f40 tx=0x7fdaa40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:37.974 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:37.973+0000 7fdaaa7fc700 1 -- 172.21.15.150:0/3484884166 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdaac097dc0 con 0x7fdab4102900 2024-05-10T04:38:38.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.169+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdab40008d0 con 0x7fdaa00a1230 2024-05-10T04:38:38.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.181+0000 7fdaaa7fc700 1 -- 172.21.15.150:0/3484884166 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422114 (secure 0 0 0) 0x7fdab40008d0 con 0x7fdaa00a1230 2024-05-10T04:38:38.191 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdaa00a1230 msgr2=0x7fdaa00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdaa00a1230 0x7fdaa00a3740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdab4101f40 tx=0x7fdaa40073d0 comp rx=0 tx=0).stop 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 msgr2=0x7fdab41a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab41a19b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdaac006070 tx=0x7fdaac004d00 comp rx=0 tx=0).stop 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 shutdown_connections 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102900 0x7fdab41a19b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab41010c0 0x7fdab4100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:38.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdaa00a1230 0x7fdaa00a3740 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:38.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 --2- 172.21.15.150:0/3484884166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdab4101fb0 0x7fdab419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:38.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 >> 172.21.15.150:0/3484884166 conn(0x7fdab40fb880 msgr2=0x7fdab41051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:38.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 shutdown_connections 2024-05-10T04:38:38.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:38.185+0000 7fdabb680700 1 -- 172.21.15.150:0/3484884166 wait complete. 2024-05-10T04:38:38.197 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:38:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-05-10T04:38:37.310333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598372,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":143,"num_read_kb":123,"num_write":325,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":266,"ondisk_log_size":266,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360008,"kb_used_data":11240,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581816,"statfs":{"total":1151940427776,"available":1151571779584,"internally_reserved":0,"allocated":11509760,"data_stored":7331080,"data_compressed":247101,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19127,"internal_metadata":357021001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3551,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":97,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003825"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787827+0000","last_change":"2024-05-10T04:38:08.784019+0000","last_active":"2024-05-10T04:38:12.787827+0000","last_peered":"2024-05-10T04:38:12.787827+0000","last_clean":"2024-05-10T04:38:12.787827+0000","last_became_active":"2024-05-10T04:38:08.783585+0000","last_became_peered":"2024-05-10T04:38:08.783585+0000","last_unstale":"2024-05-10T04:38:12.787827+0000","last_undegraded":"2024-05-10T04:38:12.787827+0000","last_fullsized":"2024-05-10T04:38:12.787827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:56.963068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788295+0000","last_change":"2024-05-10T04:38:08.777364+0000","last_active":"2024-05-10T04:38:12.788295+0000","last_peered":"2024-05-10T04:38:12.788295+0000","last_clean":"2024-05-10T04:38:12.788295+0000","last_became_active":"2024-05-10T04:38:08.776488+0000","last_became_peered":"2024-05-10T04:38:08.776488+0000","last_unstale":"2024-05-10T04:38:12.788295+0000","last_undegraded":"2024-05-10T04:38:12.788295+0000","last_fullsized":"2024-05-10T04:38:12.788295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:40.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787110+0000","last_change":"2024-05-10T04:38:08.777169+0000","last_active":"2024-05-10T04:38:12.787110+0000","last_peered":"2024-05-10T04:38:12.787110+0000","last_clean":"2024-05-10T04:38:12.787110+0000","last_became_active":"2024-05-10T04:38:08.776713+0000","last_became_peered":"2024-05-10T04:38:08.776713+0000","last_unstale":"2024-05-10T04:38:12.787110+0000","last_undegraded":"2024-05-10T04:38:12.787110+0000","last_fullsized":"2024-05-10T04:38:12.787110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:08:41.947577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791723+0000","last_change":"2024-05-10T04:38:08.788464+0000","last_active":"2024-05-10T04:38:12.791723+0000","last_peered":"2024-05-10T04:38:12.791723+0000","last_clean":"2024-05-10T04:38:12.791723+0000","last_became_active":"2024-05-10T04:38:08.787811+0000","last_became_peered":"2024-05-10T04:38:08.787811+0000","last_unstale":"2024-05-10T04:38:12.791723+0000","last_undegraded":"2024-05-10T04:38:12.791723+0000","last_fullsized":"2024-05-10T04:38:12.791723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:22.574882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156610+0000","last_change":"2024-05-10T04:38:08.772904+0000","last_active":"2024-05-10T04:38:13.156610+0000","last_peered":"2024-05-10T04:38:13.156610+0000","last_clean":"2024-05-10T04:38:13.156610+0000","last_became_active":"2024-05-10T04:38:08.772503+0000","last_became_peered":"2024-05-10T04:38:08.772503+0000","last_unstale":"2024-05-10T04:38:13.156610+0000","last_undegraded":"2024-05-10T04:38:13.156610+0000","last_fullsized":"2024-05-10T04:38:13.156610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:58.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809064+0000","last_change":"2024-05-10T04:38:08.779442+0000","last_active":"2024-05-10T04:38:12.809064+0000","last_peered":"2024-05-10T04:38:12.809064+0000","last_clean":"2024-05-10T04:38:12.809064+0000","last_became_active":"2024-05-10T04:38:08.777348+0000","last_became_peered":"2024-05-10T04:38:08.777348+0000","last_unstale":"2024-05-10T04:38:12.809064+0000","last_undegraded":"2024-05-10T04:38:12.809064+0000","last_fullsized":"2024-05-10T04:38:12.809064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:25.619031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786546+0000","last_change":"2024-05-10T04:38:08.775646+0000","last_active":"2024-05-10T04:38:12.786546+0000","last_peered":"2024-05-10T04:38:12.786546+0000","last_clean":"2024-05-10T04:38:12.786546+0000","last_became_active":"2024-05-10T04:38:08.768229+0000","last_became_peered":"2024-05-10T04:38:08.768229+0000","last_unstale":"2024-05-10T04:38:12.786546+0000","last_undegraded":"2024-05-10T04:38:12.786546+0000","last_fullsized":"2024-05-10T04:38:12.786546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:14.653065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787545+0000","last_change":"2024-05-10T04:38:08.771012+0000","last_active":"2024-05-10T04:38:12.787545+0000","last_peered":"2024-05-10T04:38:12.787545+0000","last_clean":"2024-05-10T04:38:12.787545+0000","last_became_active":"2024-05-10T04:38:08.770674+0000","last_became_peered":"2024-05-10T04:38:08.770674+0000","last_unstale":"2024-05-10T04:38:12.787545+0000","last_undegraded":"2024-05-10T04:38:12.787545+0000","last_fullsized":"2024-05-10T04:38:12.787545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:14.389263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784668+0000","last_change":"2024-05-10T04:38:08.782109+0000","last_active":"2024-05-10T04:38:12.784668+0000","last_peered":"2024-05-10T04:38:12.784668+0000","last_clean":"2024-05-10T04:38:12.784668+0000","last_became_active":"2024-05-10T04:38:08.781717+0000","last_became_peered":"2024-05-10T04:38:08.781717+0000","last_unstale":"2024-05-10T04:38:12.784668+0000","last_undegraded":"2024-05-10T04:38:12.784668+0000","last_fullsized":"2024-05-10T04:38:12.784668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:17.505448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809930+0000","last_change":"2024-05-10T04:38:08.778429+0000","last_active":"2024-05-10T04:38:12.809930+0000","last_peered":"2024-05-10T04:38:12.809930+0000","last_clean":"2024-05-10T04:38:12.809930+0000","last_became_active":"2024-05-10T04:38:08.777874+0000","last_became_peered":"2024-05-10T04:38:08.777874+0000","last_unstale":"2024-05-10T04:38:12.809930+0000","last_undegraded":"2024-05-10T04:38:12.809930+0000","last_fullsized":"2024-05-10T04:38:12.809930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:20:19.704678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786985+0000","last_change":"2024-05-10T04:38:08.780104+0000","last_active":"2024-05-10T04:38:12.786985+0000","last_peered":"2024-05-10T04:38:12.786985+0000","last_clean":"2024-05-10T04:38:12.786985+0000","last_became_active":"2024-05-10T04:38:08.779298+0000","last_became_peered":"2024-05-10T04:38:08.779298+0000","last_unstale":"2024-05-10T04:38:12.786985+0000","last_undegraded":"2024-05-10T04:38:12.786985+0000","last_fullsized":"2024-05-10T04:38:12.786985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:41.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791619+0000","last_change":"2024-05-10T04:38:08.788605+0000","last_active":"2024-05-10T04:38:12.791619+0000","last_peered":"2024-05-10T04:38:12.791619+0000","last_clean":"2024-05-10T04:38:12.791619+0000","last_became_active":"2024-05-10T04:38:08.788207+0000","last_became_peered":"2024-05-10T04:38:08.788207+0000","last_unstale":"2024-05-10T04:38:12.791619+0000","last_undegraded":"2024-05-10T04:38:12.791619+0000","last_fullsized":"2024-05-10T04:38:12.791619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:35.000980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782473+0000","last_change":"2024-05-10T04:38:08.761953+0000","last_active":"2024-05-10T04:38:12.782473+0000","last_peered":"2024-05-10T04:38:12.782473+0000","last_clean":"2024-05-10T04:38:12.782473+0000","last_became_active":"2024-05-10T04:38:08.761585+0000","last_became_peered":"2024-05-10T04:38:08.761585+0000","last_unstale":"2024-05-10T04:38:12.782473+0000","last_undegraded":"2024-05-10T04:38:12.782473+0000","last_fullsized":"2024-05-10T04:38:12.782473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:25.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786743+0000","last_change":"2024-05-10T04:38:08.777656+0000","last_active":"2024-05-10T04:38:12.786743+0000","last_peered":"2024-05-10T04:38:12.786743+0000","last_clean":"2024-05-10T04:38:12.786743+0000","last_became_active":"2024-05-10T04:38:08.777332+0000","last_became_peered":"2024-05-10T04:38:08.777332+0000","last_unstale":"2024-05-10T04:38:12.786743+0000","last_undegraded":"2024-05-10T04:38:12.786743+0000","last_fullsized":"2024-05-10T04:38:12.786743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:17:52.878943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787112+0000","last_change":"2024-05-10T04:38:08.784934+0000","last_active":"2024-05-10T04:38:12.787112+0000","last_peered":"2024-05-10T04:38:12.787112+0000","last_clean":"2024-05-10T04:38:12.787112+0000","last_became_active":"2024-05-10T04:38:08.784644+0000","last_became_peered":"2024-05-10T04:38:08.784644+0000","last_unstale":"2024-05-10T04:38:12.787112+0000","last_undegraded":"2024-05-10T04:38:12.787112+0000","last_fullsized":"2024-05-10T04:38:12.787112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:43.889305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783537+0000","last_change":"2024-05-10T04:38:08.784110+0000","last_active":"2024-05-10T04:38:12.783537+0000","last_peered":"2024-05-10T04:38:12.783537+0000","last_clean":"2024-05-10T04:38:12.783537+0000","last_became_active":"2024-05-10T04:38:08.783779+0000","last_became_peered":"2024-05-10T04:38:08.783779+0000","last_unstale":"2024-05-10T04:38:12.783537+0000","last_undegraded":"2024-05-10T04:38:12.783537+0000","last_fullsized":"2024-05-10T04:38:12.783537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:24.233024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787536+0000","last_change":"2024-05-10T04:38:08.788685+0000","last_active":"2024-05-10T04:38:12.787536+0000","last_peered":"2024-05-10T04:38:12.787536+0000","last_clean":"2024-05-10T04:38:12.787536+0000","last_became_active":"2024-05-10T04:38:08.788122+0000","last_became_peered":"2024-05-10T04:38:08.788122+0000","last_unstale":"2024-05-10T04:38:12.787536+0000","last_undegraded":"2024-05-10T04:38:12.787536+0000","last_fullsized":"2024-05-10T04:38:12.787536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:21:01.098784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802717+0000","last_change":"2024-05-10T04:38:08.776330+0000","last_active":"2024-05-10T04:38:12.802717+0000","last_peered":"2024-05-10T04:38:12.802717+0000","last_clean":"2024-05-10T04:38:12.802717+0000","last_became_active":"2024-05-10T04:38:08.776005+0000","last_became_peered":"2024-05-10T04:38:08.776005+0000","last_unstale":"2024-05-10T04:38:12.802717+0000","last_undegraded":"2024-05-10T04:38:12.802717+0000","last_fullsized":"2024-05-10T04:38:12.802717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:46:14.407766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787933+0000","last_change":"2024-05-10T04:38:08.772811+0000","last_active":"2024-05-10T04:38:12.787933+0000","last_peered":"2024-05-10T04:38:12.787933+0000","last_clean":"2024-05-10T04:38:12.787933+0000","last_became_active":"2024-05-10T04:38:08.772462+0000","last_became_peered":"2024-05-10T04:38:08.772462+0000","last_unstale":"2024-05-10T04:38:12.787933+0000","last_undegraded":"2024-05-10T04:38:12.787933+0000","last_fullsized":"2024-05-10T04:38:12.787933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:39.703137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410816+0000","last_change":"2024-05-10T04:38:12.799483+0000","last_active":"2024-05-10T04:38:13.410816+0000","last_peered":"2024-05-10T04:38:13.410816+0000","last_clean":"2024-05-10T04:38:13.410816+0000","last_became_active":"2024-05-10T04:38:12.799049+0000","last_became_peered":"2024-05-10T04:38:12.799049+0000","last_unstale":"2024-05-10T04:38:13.410816+0000","last_undegraded":"2024-05-10T04:38:13.410816+0000","last_fullsized":"2024-05-10T04:38:13.410816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:00.380645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968383+0000","last_change":"2024-05-10T04:38:08.345935+0000","last_active":"2024-05-10T04:38:34.968383+0000","last_peered":"2024-05-10T04:38:34.968383+0000","last_clean":"2024-05-10T04:38:34.968383+0000","last_became_active":"2024-05-10T04:38:08.345580+0000","last_became_peered":"2024-05-10T04:38:08.345580+0000","last_unstale":"2024-05-10T04:38:34.968383+0000","last_undegraded":"2024-05-10T04:38:34.968383+0000","last_fullsized":"2024-05-10T04:38:34.968383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:11.527710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787511+0000","last_change":"2024-05-10T04:38:08.771072+0000","last_active":"2024-05-10T04:38:12.787511+0000","last_peered":"2024-05-10T04:38:12.787511+0000","last_clean":"2024-05-10T04:38:12.787511+0000","last_became_active":"2024-05-10T04:38:08.770711+0000","last_became_peered":"2024-05-10T04:38:08.770711+0000","last_unstale":"2024-05-10T04:38:12.787511+0000","last_undegraded":"2024-05-10T04:38:12.787511+0000","last_fullsized":"2024-05-10T04:38:12.787511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:44.803219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220274+0000","last_change":"2024-05-10T04:38:12.801374+0000","last_active":"2024-05-10T04:38:13.220274+0000","last_peered":"2024-05-10T04:38:13.220274+0000","last_clean":"2024-05-10T04:38:13.220274+0000","last_became_active":"2024-05-10T04:38:12.800889+0000","last_became_peered":"2024-05-10T04:38:12.800889+0000","last_unstale":"2024-05-10T04:38:13.220274+0000","last_undegraded":"2024-05-10T04:38:13.220274+0000","last_fullsized":"2024-05-10T04:38:13.220274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:35:51.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217645+0000","last_change":"2024-05-10T04:38:08.348806+0000","last_active":"2024-05-10T04:38:13.217645+0000","last_peered":"2024-05-10T04:38:13.217645+0000","last_clean":"2024-05-10T04:38:13.217645+0000","last_became_active":"2024-05-10T04:38:08.348513+0000","last_became_peered":"2024-05-10T04:38:08.348513+0000","last_unstale":"2024-05-10T04:38:13.217645+0000","last_undegraded":"2024-05-10T04:38:13.217645+0000","last_fullsized":"2024-05-10T04:38:13.217645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:45.474403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787353+0000","last_change":"2024-05-10T04:38:08.782738+0000","last_active":"2024-05-10T04:38:12.787353+0000","last_peered":"2024-05-10T04:38:12.787353+0000","last_clean":"2024-05-10T04:38:12.787353+0000","last_became_active":"2024-05-10T04:38:08.782375+0000","last_became_peered":"2024-05-10T04:38:08.782375+0000","last_unstale":"2024-05-10T04:38:12.787353+0000","last_undegraded":"2024-05-10T04:38:12.787353+0000","last_fullsized":"2024-05-10T04:38:12.787353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:16.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220388+0000","last_change":"2024-05-10T04:38:12.801322+0000","last_active":"2024-05-10T04:38:13.220388+0000","last_peered":"2024-05-10T04:38:13.220388+0000","last_clean":"2024-05-10T04:38:13.220388+0000","last_became_active":"2024-05-10T04:38:12.800388+0000","last_became_peered":"2024-05-10T04:38:12.800388+0000","last_unstale":"2024-05-10T04:38:13.220388+0000","last_undegraded":"2024-05-10T04:38:13.220388+0000","last_fullsized":"2024-05-10T04:38:13.220388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:57.678691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950748+0000","last_change":"2024-05-10T04:38:08.347997+0000","last_active":"2024-05-10T04:38:32.950748+0000","last_peered":"2024-05-10T04:38:32.950748+0000","last_clean":"2024-05-10T04:38:32.950748+0000","last_became_active":"2024-05-10T04:38:08.347469+0000","last_became_peered":"2024-05-10T04:38:08.347469+0000","last_unstale":"2024-05-10T04:38:32.950748+0000","last_undegraded":"2024-05-10T04:38:32.950748+0000","last_fullsized":"2024-05-10T04:38:32.950748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:39.122148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783940+0000","last_change":"2024-05-10T04:38:08.781068+0000","last_active":"2024-05-10T04:38:12.783940+0000","last_peered":"2024-05-10T04:38:12.783940+0000","last_clean":"2024-05-10T04:38:12.783940+0000","last_became_active":"2024-05-10T04:38:08.780671+0000","last_became_peered":"2024-05-10T04:38:08.780671+0000","last_unstale":"2024-05-10T04:38:12.783940+0000","last_undegraded":"2024-05-10T04:38:12.783940+0000","last_fullsized":"2024-05-10T04:38:12.783940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220322+0000","last_change":"2024-05-10T04:38:12.801702+0000","last_active":"2024-05-10T04:38:13.220322+0000","last_peered":"2024-05-10T04:38:13.220322+0000","last_clean":"2024-05-10T04:38:13.220322+0000","last_became_active":"2024-05-10T04:38:12.801031+0000","last_became_peered":"2024-05-10T04:38:12.801031+0000","last_unstale":"2024-05-10T04:38:13.220322+0000","last_undegraded":"2024-05-10T04:38:13.220322+0000","last_fullsized":"2024-05-10T04:38:13.220322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:05:07.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786570+0000","last_change":"2024-05-10T04:38:08.347981+0000","last_active":"2024-05-10T04:38:12.786570+0000","last_peered":"2024-05-10T04:38:12.786570+0000","last_clean":"2024-05-10T04:38:12.786570+0000","last_became_active":"2024-05-10T04:38:08.346915+0000","last_became_peered":"2024-05-10T04:38:08.346915+0000","last_unstale":"2024-05-10T04:38:12.786570+0000","last_undegraded":"2024-05-10T04:38:12.786570+0000","last_fullsized":"2024-05-10T04:38:12.786570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:52.383734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780699+0000","last_change":"2024-05-10T04:38:08.779164+0000","last_active":"2024-05-10T04:38:12.780699+0000","last_peered":"2024-05-10T04:38:12.780699+0000","last_clean":"2024-05-10T04:38:12.780699+0000","last_became_active":"2024-05-10T04:38:08.778851+0000","last_became_peered":"2024-05-10T04:38:08.778851+0000","last_unstale":"2024-05-10T04:38:12.780699+0000","last_undegraded":"2024-05-10T04:38:12.780699+0000","last_fullsized":"2024-05-10T04:38:12.780699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:01.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411122+0000","last_change":"2024-05-10T04:38:12.796175+0000","last_active":"2024-05-10T04:38:13.411122+0000","last_peered":"2024-05-10T04:38:13.411122+0000","last_clean":"2024-05-10T04:38:13.411122+0000","last_became_active":"2024-05-10T04:38:12.795752+0000","last_became_peered":"2024-05-10T04:38:12.795752+0000","last_unstale":"2024-05-10T04:38:13.411122+0000","last_undegraded":"2024-05-10T04:38:13.411122+0000","last_fullsized":"2024-05-10T04:38:13.411122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:48.077369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.946490+0000","last_change":"2024-05-10T04:38:08.341633+0000","last_active":"2024-05-10T04:38:33.946490+0000","last_peered":"2024-05-10T04:38:33.946490+0000","last_clean":"2024-05-10T04:38:33.946490+0000","last_became_active":"2024-05-10T04:38:08.341220+0000","last_became_peered":"2024-05-10T04:38:08.341220+0000","last_unstale":"2024-05-10T04:38:33.946490+0000","last_undegraded":"2024-05-10T04:38:33.946490+0000","last_fullsized":"2024-05-10T04:38:33.946490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:36.861191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156596+0000","last_change":"2024-05-10T04:38:08.790969+0000","last_active":"2024-05-10T04:38:13.156596+0000","last_peered":"2024-05-10T04:38:13.156596+0000","last_clean":"2024-05-10T04:38:13.156596+0000","last_became_active":"2024-05-10T04:38:08.790539+0000","last_became_peered":"2024-05-10T04:38:08.790539+0000","last_unstale":"2024-05-10T04:38:13.156596+0000","last_undegraded":"2024-05-10T04:38:13.156596+0000","last_fullsized":"2024-05-10T04:38:13.156596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:50.174798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411137+0000","last_change":"2024-05-10T04:38:12.800901+0000","last_active":"2024-05-10T04:38:13.411137+0000","last_peered":"2024-05-10T04:38:13.411137+0000","last_clean":"2024-05-10T04:38:13.411137+0000","last_became_active":"2024-05-10T04:38:12.800287+0000","last_became_peered":"2024-05-10T04:38:12.800287+0000","last_unstale":"2024-05-10T04:38:13.411137+0000","last_undegraded":"2024-05-10T04:38:13.411137+0000","last_fullsized":"2024-05-10T04:38:13.411137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:23:25.463044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809056+0000","last_change":"2024-05-10T04:38:08.342449+0000","last_active":"2024-05-10T04:38:12.809056+0000","last_peered":"2024-05-10T04:38:12.809056+0000","last_clean":"2024-05-10T04:38:12.809056+0000","last_became_active":"2024-05-10T04:38:08.342043+0000","last_became_peered":"2024-05-10T04:38:08.342043+0000","last_unstale":"2024-05-10T04:38:12.809056+0000","last_undegraded":"2024-05-10T04:38:12.809056+0000","last_fullsized":"2024-05-10T04:38:12.809056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:31.161203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780542+0000","last_change":"2024-05-10T04:38:08.783759+0000","last_active":"2024-05-10T04:38:12.780542+0000","last_peered":"2024-05-10T04:38:12.780542+0000","last_clean":"2024-05-10T04:38:12.780542+0000","last_became_active":"2024-05-10T04:38:08.783419+0000","last_became_peered":"2024-05-10T04:38:08.783419+0000","last_unstale":"2024-05-10T04:38:12.780542+0000","last_undegraded":"2024-05-10T04:38:12.780542+0000","last_fullsized":"2024-05-10T04:38:12.780542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:16.114017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218482+0000","last_change":"2024-05-10T04:38:12.803018+0000","last_active":"2024-05-10T04:38:13.218482+0000","last_peered":"2024-05-10T04:38:13.218482+0000","last_clean":"2024-05-10T04:38:13.218482+0000","last_became_active":"2024-05-10T04:38:12.802546+0000","last_became_peered":"2024-05-10T04:38:12.802546+0000","last_unstale":"2024-05-10T04:38:13.218482+0000","last_undegraded":"2024-05-10T04:38:13.218482+0000","last_fullsized":"2024-05-10T04:38:13.218482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:40.676335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950654+0000","last_change":"2024-05-10T04:38:08.341404+0000","last_active":"2024-05-10T04:38:32.950654+0000","last_peered":"2024-05-10T04:38:32.950654+0000","last_clean":"2024-05-10T04:38:32.950654+0000","last_became_active":"2024-05-10T04:38:08.341013+0000","last_became_peered":"2024-05-10T04:38:08.341013+0000","last_unstale":"2024-05-10T04:38:32.950654+0000","last_undegraded":"2024-05-10T04:38:32.950654+0000","last_fullsized":"2024-05-10T04:38:32.950654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:43.513989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787639+0000","last_change":"2024-05-10T04:38:08.775650+0000","last_active":"2024-05-10T04:38:12.787639+0000","last_peered":"2024-05-10T04:38:12.787639+0000","last_clean":"2024-05-10T04:38:12.787639+0000","last_became_active":"2024-05-10T04:38:08.775320+0000","last_became_peered":"2024-05-10T04:38:08.775320+0000","last_unstale":"2024-05-10T04:38:12.787639+0000","last_undegraded":"2024-05-10T04:38:12.787639+0000","last_fullsized":"2024-05-10T04:38:12.787639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:49:07.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.341020+0000","last_change":"2024-05-10T04:38:12.797168+0000","last_active":"2024-05-10T04:38:13.341020+0000","last_peered":"2024-05-10T04:38:13.341020+0000","last_clean":"2024-05-10T04:38:13.341020+0000","last_became_active":"2024-05-10T04:38:12.796776+0000","last_became_peered":"2024-05-10T04:38:12.796776+0000","last_unstale":"2024-05-10T04:38:13.341020+0000","last_undegraded":"2024-05-10T04:38:13.341020+0000","last_fullsized":"2024-05-10T04:38:13.341020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:59:02.363398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963802+0000","last_change":"2024-05-10T04:38:08.344086+0000","last_active":"2024-05-10T04:38:33.963802+0000","last_peered":"2024-05-10T04:38:33.963802+0000","last_clean":"2024-05-10T04:38:33.963802+0000","last_became_active":"2024-05-10T04:38:08.343744+0000","last_became_peered":"2024-05-10T04:38:08.343744+0000","last_unstale":"2024-05-10T04:38:33.963802+0000","last_undegraded":"2024-05-10T04:38:33.963802+0000","last_fullsized":"2024-05-10T04:38:33.963802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:01.800590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802611+0000","last_change":"2024-05-10T04:38:08.773638+0000","last_active":"2024-05-10T04:38:12.802611+0000","last_peered":"2024-05-10T04:38:12.802611+0000","last_clean":"2024-05-10T04:38:12.802611+0000","last_became_active":"2024-05-10T04:38:08.773218+0000","last_became_peered":"2024-05-10T04:38:08.773218+0000","last_unstale":"2024-05-10T04:38:12.802611+0000","last_undegraded":"2024-05-10T04:38:12.802611+0000","last_fullsized":"2024-05-10T04:38:12.802611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:54:38.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217138+0000","last_change":"2024-05-10T04:38:12.796344+0000","last_active":"2024-05-10T04:38:13.217138+0000","last_peered":"2024-05-10T04:38:13.217138+0000","last_clean":"2024-05-10T04:38:13.217138+0000","last_became_active":"2024-05-10T04:38:12.795958+0000","last_became_peered":"2024-05-10T04:38:12.795958+0000","last_unstale":"2024-05-10T04:38:13.217138+0000","last_undegraded":"2024-05-10T04:38:13.217138+0000","last_fullsized":"2024-05-10T04:38:13.217138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:45.708078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809634+0000","last_change":"2024-05-10T04:38:08.341327+0000","last_active":"2024-05-10T04:38:12.809634+0000","last_peered":"2024-05-10T04:38:12.809634+0000","last_clean":"2024-05-10T04:38:12.809634+0000","last_became_active":"2024-05-10T04:38:08.339606+0000","last_became_peered":"2024-05-10T04:38:08.339606+0000","last_unstale":"2024-05-10T04:38:12.809634+0000","last_undegraded":"2024-05-10T04:38:12.809634+0000","last_fullsized":"2024-05-10T04:38:12.809634+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:22:40.212212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784430+0000","last_change":"2024-05-10T04:38:08.781195+0000","last_active":"2024-05-10T04:38:12.784430+0000","last_peered":"2024-05-10T04:38:12.784430+0000","last_clean":"2024-05-10T04:38:12.784430+0000","last_became_active":"2024-05-10T04:38:08.780779+0000","last_became_peered":"2024-05-10T04:38:08.780779+0000","last_unstale":"2024-05-10T04:38:12.784430+0000","last_undegraded":"2024-05-10T04:38:12.784430+0000","last_fullsized":"2024-05-10T04:38:12.784430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:24.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930221+0000","last_change":"2024-05-10T04:38:12.799494+0000","last_active":"2024-05-10T04:38:13.930221+0000","last_peered":"2024-05-10T04:38:13.930221+0000","last_clean":"2024-05-10T04:38:13.930221+0000","last_became_active":"2024-05-10T04:38:12.799071+0000","last_became_peered":"2024-05-10T04:38:12.799071+0000","last_unstale":"2024-05-10T04:38:13.930221+0000","last_undegraded":"2024-05-10T04:38:13.930221+0000","last_fullsized":"2024-05-10T04:38:13.930221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:00.814875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.408813+0000","last_change":"2024-05-10T04:38:08.348569+0000","last_active":"2024-05-10T04:38:32.408813+0000","last_peered":"2024-05-10T04:38:32.408813+0000","last_clean":"2024-05-10T04:38:32.408813+0000","last_became_active":"2024-05-10T04:38:08.347942+0000","last_became_peered":"2024-05-10T04:38:08.347942+0000","last_unstale":"2024-05-10T04:38:32.408813+0000","last_undegraded":"2024-05-10T04:38:32.408813+0000","last_fullsized":"2024-05-10T04:38:32.408813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.865369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787640+0000","last_change":"2024-05-10T04:38:08.784447+0000","last_active":"2024-05-10T04:38:12.787640+0000","last_peered":"2024-05-10T04:38:12.787640+0000","last_clean":"2024-05-10T04:38:12.787640+0000","last_became_active":"2024-05-10T04:38:08.783989+0000","last_became_peered":"2024-05-10T04:38:08.783989+0000","last_unstale":"2024-05-10T04:38:12.787640+0000","last_undegraded":"2024-05-10T04:38:12.787640+0000","last_fullsized":"2024-05-10T04:38:12.787640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:37.511400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220978+0000","last_change":"2024-05-10T04:38:12.807478+0000","last_active":"2024-05-10T04:38:13.220978+0000","last_peered":"2024-05-10T04:38:13.220978+0000","last_clean":"2024-05-10T04:38:13.220978+0000","last_became_active":"2024-05-10T04:38:12.806801+0000","last_became_peered":"2024-05-10T04:38:12.806801+0000","last_unstale":"2024-05-10T04:38:13.220978+0000","last_undegraded":"2024-05-10T04:38:13.220978+0000","last_fullsized":"2024-05-10T04:38:13.220978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:32.168836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.943959+0000","last_change":"2024-05-10T04:38:08.343610+0000","last_active":"2024-05-10T04:38:32.943959+0000","last_peered":"2024-05-10T04:38:32.943959+0000","last_clean":"2024-05-10T04:38:32.943959+0000","last_became_active":"2024-05-10T04:38:08.342880+0000","last_became_peered":"2024-05-10T04:38:08.342880+0000","last_unstale":"2024-05-10T04:38:32.943959+0000","last_undegraded":"2024-05-10T04:38:32.943959+0000","last_fullsized":"2024-05-10T04:38:32.943959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:45:11.502258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784342+0000","last_change":"2024-05-10T04:38:08.780354+0000","last_active":"2024-05-10T04:38:12.784342+0000","last_peered":"2024-05-10T04:38:12.784342+0000","last_clean":"2024-05-10T04:38:12.784342+0000","last_became_active":"2024-05-10T04:38:08.779777+0000","last_became_peered":"2024-05-10T04:38:08.779777+0000","last_unstale":"2024-05-10T04:38:12.784342+0000","last_undegraded":"2024-05-10T04:38:12.784342+0000","last_fullsized":"2024-05-10T04:38:12.784342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:28.243474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931246+0000","last_change":"2024-05-10T04:38:12.800759+0000","last_active":"2024-05-10T04:38:13.931246+0000","last_peered":"2024-05-10T04:38:13.931246+0000","last_clean":"2024-05-10T04:38:13.931246+0000","last_became_active":"2024-05-10T04:38:12.800417+0000","last_became_peered":"2024-05-10T04:38:12.800417+0000","last_unstale":"2024-05-10T04:38:13.931246+0000","last_undegraded":"2024-05-10T04:38:13.931246+0000","last_fullsized":"2024-05-10T04:38:13.931246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:04.734817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:23.906052+0000","last_change":"2024-05-10T04:38:08.343873+0000","last_active":"2024-05-10T04:38:23.906052+0000","last_peered":"2024-05-10T04:38:23.906052+0000","last_clean":"2024-05-10T04:38:23.906052+0000","last_became_active":"2024-05-10T04:38:08.343360+0000","last_became_peered":"2024-05-10T04:38:08.343360+0000","last_unstale":"2024-05-10T04:38:23.906052+0000","last_undegraded":"2024-05-10T04:38:23.906052+0000","last_fullsized":"2024-05-10T04:38:23.906052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:06:10.816343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156650+0000","last_change":"2024-05-10T04:38:08.783955+0000","last_active":"2024-05-10T04:38:13.156650+0000","last_peered":"2024-05-10T04:38:13.156650+0000","last_clean":"2024-05-10T04:38:13.156650+0000","last_became_active":"2024-05-10T04:38:08.779131+0000","last_became_peered":"2024-05-10T04:38:08.779131+0000","last_unstale":"2024-05-10T04:38:13.156650+0000","last_undegraded":"2024-05-10T04:38:13.156650+0000","last_fullsized":"2024-05-10T04:38:13.156650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:58.619571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930607+0000","last_change":"2024-05-10T04:38:12.795140+0000","last_active":"2024-05-10T04:38:13.930607+0000","last_peered":"2024-05-10T04:38:13.930607+0000","last_clean":"2024-05-10T04:38:13.930607+0000","last_became_active":"2024-05-10T04:38:12.794666+0000","last_became_peered":"2024-05-10T04:38:12.794666+0000","last_unstale":"2024-05-10T04:38:13.930607+0000","last_undegraded":"2024-05-10T04:38:13.930607+0000","last_fullsized":"2024-05-10T04:38:13.930607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:44.468321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950141+0000","last_change":"2024-05-10T04:38:08.338866+0000","last_active":"2024-05-10T04:38:32.950141+0000","last_peered":"2024-05-10T04:38:32.950141+0000","last_clean":"2024-05-10T04:38:32.950141+0000","last_became_active":"2024-05-10T04:38:08.338434+0000","last_became_peered":"2024-05-10T04:38:08.338434+0000","last_unstale":"2024-05-10T04:38:32.950141+0000","last_undegraded":"2024-05-10T04:38:32.950141+0000","last_fullsized":"2024-05-10T04:38:32.950141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:49.858034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781395+0000","last_change":"2024-05-10T04:38:08.768998+0000","last_active":"2024-05-10T04:38:12.781395+0000","last_peered":"2024-05-10T04:38:12.781395+0000","last_clean":"2024-05-10T04:38:12.781395+0000","last_became_active":"2024-05-10T04:38:08.768687+0000","last_became_peered":"2024-05-10T04:38:08.768687+0000","last_unstale":"2024-05-10T04:38:12.781395+0000","last_undegraded":"2024-05-10T04:38:12.781395+0000","last_fullsized":"2024-05-10T04:38:12.781395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:19.390746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218211+0000","last_change":"2024-05-10T04:38:12.801223+0000","last_active":"2024-05-10T04:38:13.218211+0000","last_peered":"2024-05-10T04:38:13.218211+0000","last_clean":"2024-05-10T04:38:13.218211+0000","last_became_active":"2024-05-10T04:38:12.800750+0000","last_became_peered":"2024-05-10T04:38:12.800750+0000","last_unstale":"2024-05-10T04:38:13.218211+0000","last_undegraded":"2024-05-10T04:38:13.218211+0000","last_fullsized":"2024-05-10T04:38:13.218211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:33.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963267+0000","last_change":"2024-05-10T04:38:08.344579+0000","last_active":"2024-05-10T04:38:33.963267+0000","last_peered":"2024-05-10T04:38:33.963267+0000","last_clean":"2024-05-10T04:38:33.963267+0000","last_became_active":"2024-05-10T04:38:08.344250+0000","last_became_peered":"2024-05-10T04:38:08.344250+0000","last_unstale":"2024-05-10T04:38:33.963267+0000","last_undegraded":"2024-05-10T04:38:33.963267+0000","last_fullsized":"2024-05-10T04:38:33.963267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:56.491732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786793+0000","last_change":"2024-05-10T04:38:08.784556+0000","last_active":"2024-05-10T04:38:12.786793+0000","last_peered":"2024-05-10T04:38:12.786793+0000","last_clean":"2024-05-10T04:38:12.786793+0000","last_became_active":"2024-05-10T04:38:08.784229+0000","last_became_peered":"2024-05-10T04:38:08.784229+0000","last_unstale":"2024-05-10T04:38:12.786793+0000","last_undegraded":"2024-05-10T04:38:12.786793+0000","last_fullsized":"2024-05-10T04:38:12.786793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:48:46.519837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930516+0000","last_change":"2024-05-10T04:38:12.805795+0000","last_active":"2024-05-10T04:38:13.930516+0000","last_peered":"2024-05-10T04:38:13.930516+0000","last_clean":"2024-05-10T04:38:13.930516+0000","last_became_active":"2024-05-10T04:38:12.805366+0000","last_became_peered":"2024-05-10T04:38:12.805366+0000","last_unstale":"2024-05-10T04:38:13.930516+0000","last_undegraded":"2024-05-10T04:38:13.930516+0000","last_fullsized":"2024-05-10T04:38:13.930516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:38:11.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950525+0000","last_change":"2024-05-10T04:38:08.345396+0000","last_active":"2024-05-10T04:38:32.950525+0000","last_peered":"2024-05-10T04:38:32.950525+0000","last_clean":"2024-05-10T04:38:32.950525+0000","last_became_active":"2024-05-10T04:38:08.345076+0000","last_became_peered":"2024-05-10T04:38:08.345076+0000","last_unstale":"2024-05-10T04:38:32.950525+0000","last_undegraded":"2024-05-10T04:38:32.950525+0000","last_fullsized":"2024-05-10T04:38:32.950525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:38.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787293+0000","last_change":"2024-05-10T04:38:08.791089+0000","last_active":"2024-05-10T04:38:12.787293+0000","last_peered":"2024-05-10T04:38:12.787293+0000","last_clean":"2024-05-10T04:38:12.787293+0000","last_became_active":"2024-05-10T04:38:08.790813+0000","last_became_peered":"2024-05-10T04:38:08.790813+0000","last_unstale":"2024-05-10T04:38:12.787293+0000","last_undegraded":"2024-05-10T04:38:12.787293+0000","last_fullsized":"2024-05-10T04:38:12.787293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:52:45.753196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809384+0000","last_change":"2024-05-10T04:38:12.803896+0000","last_active":"2024-05-10T04:38:12.809384+0000","last_peered":"2024-05-10T04:38:12.809384+0000","last_clean":"2024-05-10T04:38:12.809384+0000","last_became_active":"2024-05-10T04:38:12.803512+0000","last_became_peered":"2024-05-10T04:38:12.803512+0000","last_unstale":"2024-05-10T04:38:12.809384+0000","last_undegraded":"2024-05-10T04:38:12.809384+0000","last_fullsized":"2024-05-10T04:38:12.809384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:01:23.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.952446+0000","last_change":"2024-05-10T04:38:08.348187+0000","last_active":"2024-05-10T04:38:32.952446+0000","last_peered":"2024-05-10T04:38:32.952446+0000","last_clean":"2024-05-10T04:38:32.952446+0000","last_became_active":"2024-05-10T04:38:08.346986+0000","last_became_peered":"2024-05-10T04:38:08.346986+0000","last_unstale":"2024-05-10T04:38:32.952446+0000","last_undegraded":"2024-05-10T04:38:32.952446+0000","last_fullsized":"2024-05-10T04:38:32.952446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:46.245042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780738+0000","last_change":"2024-05-10T04:38:08.782905+0000","last_active":"2024-05-10T04:38:12.780738+0000","last_peered":"2024-05-10T04:38:12.780738+0000","last_clean":"2024-05-10T04:38:12.780738+0000","last_became_active":"2024-05-10T04:38:08.782508+0000","last_became_peered":"2024-05-10T04:38:08.782508+0000","last_unstale":"2024-05-10T04:38:12.780738+0000","last_undegraded":"2024-05-10T04:38:12.780738+0000","last_fullsized":"2024-05-10T04:38:12.780738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:29.026282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930246+0000","last_change":"2024-05-10T04:38:12.796046+0000","last_active":"2024-05-10T04:38:13.930246+0000","last_peered":"2024-05-10T04:38:13.930246+0000","last_clean":"2024-05-10T04:38:13.930246+0000","last_became_active":"2024-05-10T04:38:12.795635+0000","last_became_peered":"2024-05-10T04:38:12.795635+0000","last_unstale":"2024-05-10T04:38:13.930246+0000","last_undegraded":"2024-05-10T04:38:13.930246+0000","last_fullsized":"2024-05-10T04:38:13.930246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:50.822404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.869606+0000","last_change":"2024-05-10T04:38:08.348701+0000","last_active":"2024-05-10T04:38:13.869606+0000","last_peered":"2024-05-10T04:38:13.869606+0000","last_clean":"2024-05-10T04:38:13.869606+0000","last_became_active":"2024-05-10T04:38:08.348345+0000","last_became_peered":"2024-05-10T04:38:08.348345+0000","last_unstale":"2024-05-10T04:38:13.869606+0000","last_undegraded":"2024-05-10T04:38:13.869606+0000","last_fullsized":"2024-05-10T04:38:13.869606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:31.505996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787204+0000","last_change":"2024-05-10T04:38:08.793070+0000","last_active":"2024-05-10T04:38:12.787204+0000","last_peered":"2024-05-10T04:38:12.787204+0000","last_clean":"2024-05-10T04:38:12.787204+0000","last_became_active":"2024-05-10T04:38:08.792694+0000","last_became_peered":"2024-05-10T04:38:08.792694+0000","last_unstale":"2024-05-10T04:38:12.787204+0000","last_undegraded":"2024-05-10T04:38:12.787204+0000","last_fullsized":"2024-05-10T04:38:12.787204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:23.636222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156578+0000","last_change":"2024-05-10T04:38:12.809215+0000","last_active":"2024-05-10T04:38:13.156578+0000","last_peered":"2024-05-10T04:38:13.156578+0000","last_clean":"2024-05-10T04:38:13.156578+0000","last_became_active":"2024-05-10T04:38:12.808717+0000","last_became_peered":"2024-05-10T04:38:12.808717+0000","last_unstale":"2024-05-10T04:38:13.156578+0000","last_undegraded":"2024-05-10T04:38:13.156578+0000","last_fullsized":"2024-05-10T04:38:13.156578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:40.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931223+0000","last_change":"2024-05-10T04:38:08.347770+0000","last_active":"2024-05-10T04:38:13.931223+0000","last_peered":"2024-05-10T04:38:13.931223+0000","last_clean":"2024-05-10T04:38:13.931223+0000","last_became_active":"2024-05-10T04:38:08.347444+0000","last_became_peered":"2024-05-10T04:38:08.347444+0000","last_unstale":"2024-05-10T04:38:13.931223+0000","last_undegraded":"2024-05-10T04:38:13.931223+0000","last_fullsized":"2024-05-10T04:38:13.931223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:49.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809464+0000","last_change":"2024-05-10T04:38:08.765358+0000","last_active":"2024-05-10T04:38:12.809464+0000","last_peered":"2024-05-10T04:38:12.809464+0000","last_clean":"2024-05-10T04:38:12.809464+0000","last_became_active":"2024-05-10T04:38:08.765090+0000","last_became_peered":"2024-05-10T04:38:08.765090+0000","last_unstale":"2024-05-10T04:38:12.809464+0000","last_undegraded":"2024-05-10T04:38:12.809464+0000","last_fullsized":"2024-05-10T04:38:12.809464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:55:25.722331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.798587+0000","last_change":"2024-05-10T04:38:12.794765+0000","last_active":"2024-05-10T04:38:12.798587+0000","last_peered":"2024-05-10T04:38:12.798587+0000","last_clean":"2024-05-10T04:38:12.798587+0000","last_became_active":"2024-05-10T04:38:12.794400+0000","last_became_peered":"2024-05-10T04:38:12.794400+0000","last_unstale":"2024-05-10T04:38:12.798587+0000","last_undegraded":"2024-05-10T04:38:12.798587+0000","last_fullsized":"2024-05-10T04:38:12.798587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:15:16.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930593+0000","last_change":"2024-05-10T04:38:08.348812+0000","last_active":"2024-05-10T04:38:13.930593+0000","last_peered":"2024-05-10T04:38:13.930593+0000","last_clean":"2024-05-10T04:38:13.930593+0000","last_became_active":"2024-05-10T04:38:08.348207+0000","last_became_peered":"2024-05-10T04:38:08.348207+0000","last_unstale":"2024-05-10T04:38:13.930593+0000","last_undegraded":"2024-05-10T04:38:13.930593+0000","last_fullsized":"2024-05-10T04:38:13.930593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:17.350094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809431+0000","last_change":"2024-05-10T04:38:08.785502+0000","last_active":"2024-05-10T04:38:12.809431+0000","last_peered":"2024-05-10T04:38:12.809431+0000","last_clean":"2024-05-10T04:38:12.809431+0000","last_became_active":"2024-05-10T04:38:08.785073+0000","last_became_peered":"2024-05-10T04:38:08.785073+0000","last_unstale":"2024-05-10T04:38:12.809431+0000","last_undegraded":"2024-05-10T04:38:12.809431+0000","last_fullsized":"2024-05-10T04:38:12.809431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:08:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.810257+0000","last_change":"2024-05-10T04:38:12.809599+0000","last_active":"2024-05-10T04:38:12.810257+0000","last_peered":"2024-05-10T04:38:12.810257+0000","last_clean":"2024-05-10T04:38:12.810257+0000","last_became_active":"2024-05-10T04:38:12.809200+0000","last_became_peered":"2024-05-10T04:38:12.809200+0000","last_unstale":"2024-05-10T04:38:12.810257+0000","last_undegraded":"2024-05-10T04:38:12.810257+0000","last_fullsized":"2024-05-10T04:38:12.810257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:54.940517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781249+0000","last_change":"2024-05-10T04:38:08.779837+0000","last_active":"2024-05-10T04:38:12.781249+0000","last_peered":"2024-05-10T04:38:12.781249+0000","last_clean":"2024-05-10T04:38:12.781249+0000","last_became_active":"2024-05-10T04:38:08.779501+0000","last_became_peered":"2024-05-10T04:38:08.779501+0000","last_unstale":"2024-05-10T04:38:12.781249+0000","last_undegraded":"2024-05-10T04:38:12.781249+0000","last_fullsized":"2024-05-10T04:38:12.781249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:00.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.985942+0000","last_change":"2024-05-10T04:38:08.343649+0000","last_active":"2024-05-10T04:38:35.985942+0000","last_peered":"2024-05-10T04:38:35.985942+0000","last_clean":"2024-05-10T04:38:35.985942+0000","last_became_active":"2024-05-10T04:38:08.343292+0000","last_became_peered":"2024-05-10T04:38:08.343292+0000","last_unstale":"2024-05-10T04:38:35.985942+0000","last_undegraded":"2024-05-10T04:38:35.985942+0000","last_fullsized":"2024-05-10T04:38:35.985942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:53.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930867+0000","last_change":"2024-05-10T04:38:12.796552+0000","last_active":"2024-05-10T04:38:13.930867+0000","last_peered":"2024-05-10T04:38:13.930867+0000","last_clean":"2024-05-10T04:38:13.930867+0000","last_became_active":"2024-05-10T04:38:12.796151+0000","last_became_peered":"2024-05-10T04:38:12.796151+0000","last_unstale":"2024-05-10T04:38:13.930867+0000","last_undegraded":"2024-05-10T04:38:13.930867+0000","last_fullsized":"2024-05-10T04:38:13.930867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:05.520859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787088+0000","last_change":"2024-05-10T04:38:08.770957+0000","last_active":"2024-05-10T04:38:12.787088+0000","last_peered":"2024-05-10T04:38:12.787088+0000","last_clean":"2024-05-10T04:38:12.787088+0000","last_became_active":"2024-05-10T04:38:08.770572+0000","last_became_peered":"2024-05-10T04:38:08.770572+0000","last_unstale":"2024-05-10T04:38:12.787088+0000","last_undegraded":"2024-05-10T04:38:12.787088+0000","last_fullsized":"2024-05-10T04:38:12.787088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:21.421033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.940410+0000","last_change":"2024-05-10T04:38:08.340788+0000","last_active":"2024-05-10T04:38:32.940410+0000","last_peered":"2024-05-10T04:38:32.940410+0000","last_clean":"2024-05-10T04:38:32.940410+0000","last_became_active":"2024-05-10T04:38:08.340407+0000","last_became_peered":"2024-05-10T04:38:08.340407+0000","last_unstale":"2024-05-10T04:38:32.940410+0000","last_undegraded":"2024-05-10T04:38:32.940410+0000","last_fullsized":"2024-05-10T04:38:32.940410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:30:05.276951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930055+0000","last_change":"2024-05-10T04:37:20.961282+0000","last_active":"2024-05-10T04:38:13.930055+0000","last_peered":"2024-05-10T04:38:13.930055+0000","last_clean":"2024-05-10T04:38:13.930055+0000","last_became_active":"2024-05-10T04:37:20.960828+0000","last_became_peered":"2024-05-10T04:37:20.960828+0000","last_unstale":"2024-05-10T04:38:13.930055+0000","last_undegraded":"2024-05-10T04:38:13.930055+0000","last_fullsized":"2024-05-10T04:38:13.930055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_clean_scrub_stamp":"2024-05-10T04:35:00.248380+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:08:04.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930338+0000","last_change":"2024-05-10T04:38:12.809899+0000","last_active":"2024-05-10T04:38:13.930338+0000","last_peered":"2024-05-10T04:38:13.930338+0000","last_clean":"2024-05-10T04:38:13.930338+0000","last_became_active":"2024-05-10T04:38:12.809441+0000","last_became_peered":"2024-05-10T04:38:12.809441+0000","last_unstale":"2024-05-10T04:38:13.930338+0000","last_undegraded":"2024-05-10T04:38:13.930338+0000","last_fullsized":"2024-05-10T04:38:13.930338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:48.378890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781228+0000","last_change":"2024-05-10T04:38:08.784838+0000","last_active":"2024-05-10T04:38:12.781228+0000","last_peered":"2024-05-10T04:38:12.781228+0000","last_clean":"2024-05-10T04:38:12.781228+0000","last_became_active":"2024-05-10T04:38:08.784464+0000","last_became_peered":"2024-05-10T04:38:08.784464+0000","last_unstale":"2024-05-10T04:38:12.781228+0000","last_undegraded":"2024-05-10T04:38:12.781228+0000","last_fullsized":"2024-05-10T04:38:12.781228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:00.335350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.952966+0000","last_change":"2024-05-10T04:38:08.348639+0000","last_active":"2024-05-10T04:38:32.952966+0000","last_peered":"2024-05-10T04:38:32.952966+0000","last_clean":"2024-05-10T04:38:32.952966+0000","last_became_active":"2024-05-10T04:38:08.348271+0000","last_became_peered":"2024-05-10T04:38:08.348271+0000","last_unstale":"2024-05-10T04:38:32.952966+0000","last_undegraded":"2024-05-10T04:38:32.952966+0000","last_fullsized":"2024-05-10T04:38:32.952966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:18.486150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218189+0000","last_change":"2024-05-10T04:38:12.801079+0000","last_active":"2024-05-10T04:38:13.218189+0000","last_peered":"2024-05-10T04:38:13.218189+0000","last_clean":"2024-05-10T04:38:13.218189+0000","last_became_active":"2024-05-10T04:38:12.800630+0000","last_became_peered":"2024-05-10T04:38:12.800630+0000","last_unstale":"2024-05-10T04:38:13.218189+0000","last_undegraded":"2024-05-10T04:38:13.218189+0000","last_fullsized":"2024-05-10T04:38:13.218189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:05.213620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.408355+0000","last_change":"2024-05-10T04:38:08.348764+0000","last_active":"2024-05-10T04:38:32.408355+0000","last_peered":"2024-05-10T04:38:32.408355+0000","last_clean":"2024-05-10T04:38:32.408355+0000","last_became_active":"2024-05-10T04:38:08.347939+0000","last_became_peered":"2024-05-10T04:38:08.347939+0000","last_unstale":"2024-05-10T04:38:32.408355+0000","last_undegraded":"2024-05-10T04:38:32.408355+0000","last_fullsized":"2024-05-10T04:38:32.408355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:01.308255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782312+0000","last_change":"2024-05-10T04:38:08.771382+0000","last_active":"2024-05-10T04:38:12.782312+0000","last_peered":"2024-05-10T04:38:12.782312+0000","last_clean":"2024-05-10T04:38:12.782312+0000","last_became_active":"2024-05-10T04:38:08.771108+0000","last_became_peered":"2024-05-10T04:38:08.771108+0000","last_unstale":"2024-05-10T04:38:12.782312+0000","last_undegraded":"2024-05-10T04:38:12.782312+0000","last_fullsized":"2024-05-10T04:38:12.782312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:32.664640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217051+0000","last_change":"2024-05-10T04:38:12.809582+0000","last_active":"2024-05-10T04:38:13.217051+0000","last_peered":"2024-05-10T04:38:13.217051+0000","last_clean":"2024-05-10T04:38:13.217051+0000","last_became_active":"2024-05-10T04:38:12.809134+0000","last_became_peered":"2024-05-10T04:38:12.809134+0000","last_unstale":"2024-05-10T04:38:13.217051+0000","last_undegraded":"2024-05-10T04:38:13.217051+0000","last_fullsized":"2024-05-10T04:38:13.217051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:31:07.694368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.965023+0000","last_change":"2024-05-10T04:38:08.345903+0000","last_active":"2024-05-10T04:38:33.965023+0000","last_peered":"2024-05-10T04:38:33.965023+0000","last_clean":"2024-05-10T04:38:33.965023+0000","last_became_active":"2024-05-10T04:38:08.345371+0000","last_became_peered":"2024-05-10T04:38:08.345371+0000","last_unstale":"2024-05-10T04:38:33.965023+0000","last_undegraded":"2024-05-10T04:38:33.965023+0000","last_fullsized":"2024-05-10T04:38:33.965023+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:03.581112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789703+0000","last_change":"2024-05-10T04:38:08.779853+0000","last_active":"2024-05-10T04:38:12.789703+0000","last_peered":"2024-05-10T04:38:12.789703+0000","last_clean":"2024-05-10T04:38:12.789703+0000","last_became_active":"2024-05-10T04:38:08.779531+0000","last_became_peered":"2024-05-10T04:38:08.779531+0000","last_unstale":"2024-05-10T04:38:12.789703+0000","last_undegraded":"2024-05-10T04:38:12.789703+0000","last_fullsized":"2024-05-10T04:38:12.789703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:05.079540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410643+0000","last_change":"2024-05-10T04:38:12.795970+0000","last_active":"2024-05-10T04:38:13.410643+0000","last_peered":"2024-05-10T04:38:13.410643+0000","last_clean":"2024-05-10T04:38:13.410643+0000","last_became_active":"2024-05-10T04:38:12.795497+0000","last_became_peered":"2024-05-10T04:38:12.795497+0000","last_unstale":"2024-05-10T04:38:13.410643+0000","last_undegraded":"2024-05-10T04:38:13.410643+0000","last_fullsized":"2024-05-10T04:38:13.410643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:11.174672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.409250+0000","last_change":"2024-05-10T04:38:08.343268+0000","last_active":"2024-05-10T04:38:32.409250+0000","last_peered":"2024-05-10T04:38:32.409250+0000","last_clean":"2024-05-10T04:38:32.409250+0000","last_became_active":"2024-05-10T04:38:08.342952+0000","last_became_peered":"2024-05-10T04:38:08.342952+0000","last_unstale":"2024-05-10T04:38:32.409250+0000","last_undegraded":"2024-05-10T04:38:32.409250+0000","last_fullsized":"2024-05-10T04:38:32.409250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:49.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788390+0000","last_change":"2024-05-10T04:38:08.784276+0000","last_active":"2024-05-10T04:38:12.788390+0000","last_peered":"2024-05-10T04:38:12.788390+0000","last_clean":"2024-05-10T04:38:12.788390+0000","last_became_active":"2024-05-10T04:38:08.783951+0000","last_became_peered":"2024-05-10T04:38:08.783951+0000","last_unstale":"2024-05-10T04:38:12.788390+0000","last_undegraded":"2024-05-10T04:38:12.788390+0000","last_fullsized":"2024-05-10T04:38:12.788390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:45:43.992567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220551+0000","last_change":"2024-05-10T04:38:12.811689+0000","last_active":"2024-05-10T04:38:13.220551+0000","last_peered":"2024-05-10T04:38:13.220551+0000","last_clean":"2024-05-10T04:38:13.220551+0000","last_became_active":"2024-05-10T04:38:12.811138+0000","last_became_peered":"2024-05-10T04:38:12.811138+0000","last_unstale":"2024-05-10T04:38:13.220551+0000","last_undegraded":"2024-05-10T04:38:13.220551+0000","last_fullsized":"2024-05-10T04:38:13.220551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:09.291137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410146+0000","last_change":"2024-05-10T04:38:08.348738+0000","last_active":"2024-05-10T04:38:32.410146+0000","last_peered":"2024-05-10T04:38:32.410146+0000","last_clean":"2024-05-10T04:38:32.410146+0000","last_became_active":"2024-05-10T04:38:08.348382+0000","last_became_peered":"2024-05-10T04:38:08.348382+0000","last_unstale":"2024-05-10T04:38:32.410146+0000","last_undegraded":"2024-05-10T04:38:32.410146+0000","last_fullsized":"2024-05-10T04:38:32.410146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:02.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156551+0000","last_change":"2024-05-10T04:38:08.781483+0000","last_active":"2024-05-10T04:38:13.156551+0000","last_peered":"2024-05-10T04:38:13.156551+0000","last_clean":"2024-05-10T04:38:13.156551+0000","last_became_active":"2024-05-10T04:38:08.777713+0000","last_became_peered":"2024-05-10T04:38:08.777713+0000","last_unstale":"2024-05-10T04:38:13.156551+0000","last_undegraded":"2024-05-10T04:38:13.156551+0000","last_fullsized":"2024-05-10T04:38:13.156551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:41.050901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930627+0000","last_change":"2024-05-10T04:38:12.800656+0000","last_active":"2024-05-10T04:38:13.930627+0000","last_peered":"2024-05-10T04:38:13.930627+0000","last_clean":"2024-05-10T04:38:13.930627+0000","last_became_active":"2024-05-10T04:38:12.800191+0000","last_became_peered":"2024-05-10T04:38:12.800191+0000","last_unstale":"2024-05-10T04:38:13.930627+0000","last_undegraded":"2024-05-10T04:38:13.930627+0000","last_fullsized":"2024-05-10T04:38:13.930627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:47.017642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968707+0000","last_change":"2024-05-10T04:38:08.344017+0000","last_active":"2024-05-10T04:38:34.968707+0000","last_peered":"2024-05-10T04:38:34.968707+0000","last_clean":"2024-05-10T04:38:34.968707+0000","last_became_active":"2024-05-10T04:38:08.343646+0000","last_became_peered":"2024-05-10T04:38:08.343646+0000","last_unstale":"2024-05-10T04:38:34.968707+0000","last_undegraded":"2024-05-10T04:38:34.968707+0000","last_fullsized":"2024-05-10T04:38:34.968707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:02:30.455019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786221+0000","last_change":"2024-05-10T04:38:08.779065+0000","last_active":"2024-05-10T04:38:12.786221+0000","last_peered":"2024-05-10T04:38:12.786221+0000","last_clean":"2024-05-10T04:38:12.786221+0000","last_became_active":"2024-05-10T04:38:08.778685+0000","last_became_peered":"2024-05-10T04:38:08.778685+0000","last_unstale":"2024-05-10T04:38:12.786221+0000","last_undegraded":"2024-05-10T04:38:12.786221+0000","last_fullsized":"2024-05-10T04:38:12.786221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:25:58.780080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220710+0000","last_change":"2024-05-10T04:38:12.801524+0000","last_active":"2024-05-10T04:38:13.220710+0000","last_peered":"2024-05-10T04:38:13.220710+0000","last_clean":"2024-05-10T04:38:13.220710+0000","last_became_active":"2024-05-10T04:38:12.800777+0000","last_became_peered":"2024-05-10T04:38:12.800777+0000","last_unstale":"2024-05-10T04:38:13.220710+0000","last_undegraded":"2024-05-10T04:38:13.220710+0000","last_fullsized":"2024-05-10T04:38:13.220710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:14.139236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787348+0000","last_change":"2024-05-10T04:38:08.344785+0000","last_active":"2024-05-10T04:38:12.787348+0000","last_peered":"2024-05-10T04:38:12.787348+0000","last_clean":"2024-05-10T04:38:12.787348+0000","last_became_active":"2024-05-10T04:38:08.344337+0000","last_became_peered":"2024-05-10T04:38:08.344337+0000","last_unstale":"2024-05-10T04:38:12.787348+0000","last_undegraded":"2024-05-10T04:38:12.787348+0000","last_fullsized":"2024-05-10T04:38:12.787348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:47.682532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788214+0000","last_change":"2024-05-10T04:38:08.768686+0000","last_active":"2024-05-10T04:38:12.788214+0000","last_peered":"2024-05-10T04:38:12.788214+0000","last_clean":"2024-05-10T04:38:12.788214+0000","last_became_active":"2024-05-10T04:38:08.768317+0000","last_became_peered":"2024-05-10T04:38:08.768317+0000","last_unstale":"2024-05-10T04:38:12.788214+0000","last_undegraded":"2024-05-10T04:38:12.788214+0000","last_fullsized":"2024-05-10T04:38:12.788214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:42:33.670912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220479+0000","last_change":"2024-05-10T04:38:12.809825+0000","last_active":"2024-05-10T04:38:13.220479+0000","last_peered":"2024-05-10T04:38:13.220479+0000","last_clean":"2024-05-10T04:38:13.220479+0000","last_became_active":"2024-05-10T04:38:12.809378+0000","last_became_peered":"2024-05-10T04:38:12.809378+0000","last_unstale":"2024-05-10T04:38:13.220479+0000","last_undegraded":"2024-05-10T04:38:13.220479+0000","last_fullsized":"2024-05-10T04:38:13.220479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:28.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410164+0000","last_change":"2024-05-10T04:38:08.347871+0000","last_active":"2024-05-10T04:38:32.410164+0000","last_peered":"2024-05-10T04:38:32.410164+0000","last_clean":"2024-05-10T04:38:32.410164+0000","last_became_active":"2024-05-10T04:38:08.347618+0000","last_became_peered":"2024-05-10T04:38:08.347618+0000","last_unstale":"2024-05-10T04:38:32.410164+0000","last_undegraded":"2024-05-10T04:38:32.410164+0000","last_fullsized":"2024-05-10T04:38:32.410164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:09:42.060082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787115+0000","last_change":"2024-05-10T04:38:08.779191+0000","last_active":"2024-05-10T04:38:12.787115+0000","last_peered":"2024-05-10T04:38:12.787115+0000","last_clean":"2024-05-10T04:38:12.787115+0000","last_became_active":"2024-05-10T04:38:08.778829+0000","last_became_peered":"2024-05-10T04:38:08.778829+0000","last_unstale":"2024-05-10T04:38:12.787115+0000","last_undegraded":"2024-05-10T04:38:12.787115+0000","last_fullsized":"2024-05-10T04:38:12.787115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:07.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931626+0000","last_change":"2024-05-10T04:38:12.788693+0000","last_active":"2024-05-10T04:38:13.931626+0000","last_peered":"2024-05-10T04:38:13.931626+0000","last_clean":"2024-05-10T04:38:13.931626+0000","last_became_active":"2024-05-10T04:38:12.788156+0000","last_became_peered":"2024-05-10T04:38:12.788156+0000","last_unstale":"2024-05-10T04:38:13.931626+0000","last_undegraded":"2024-05-10T04:38:13.931626+0000","last_fullsized":"2024-05-10T04:38:13.931626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:30.119523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950714+0000","last_change":"2024-05-10T04:38:08.344211+0000","last_active":"2024-05-10T04:38:32.950714+0000","last_peered":"2024-05-10T04:38:32.950714+0000","last_clean":"2024-05-10T04:38:32.950714+0000","last_became_active":"2024-05-10T04:38:08.343443+0000","last_became_peered":"2024-05-10T04:38:08.343443+0000","last_unstale":"2024-05-10T04:38:32.950714+0000","last_undegraded":"2024-05-10T04:38:32.950714+0000","last_fullsized":"2024-05-10T04:38:32.950714+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:33:06.960942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787186+0000","last_change":"2024-05-10T04:38:08.780352+0000","last_active":"2024-05-10T04:38:12.787186+0000","last_peered":"2024-05-10T04:38:12.787186+0000","last_clean":"2024-05-10T04:38:12.787186+0000","last_became_active":"2024-05-10T04:38:08.779779+0000","last_became_peered":"2024-05-10T04:38:08.779779+0000","last_unstale":"2024-05-10T04:38:12.787186+0000","last_undegraded":"2024-05-10T04:38:12.787186+0000","last_fullsized":"2024-05-10T04:38:12.787186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:54:31.973896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.340967+0000","last_change":"2024-05-10T04:38:12.805310+0000","last_active":"2024-05-10T04:38:13.340967+0000","last_peered":"2024-05-10T04:38:13.340967+0000","last_clean":"2024-05-10T04:38:13.340967+0000","last_became_active":"2024-05-10T04:38:12.804827+0000","last_became_peered":"2024-05-10T04:38:12.804827+0000","last_unstale":"2024-05-10T04:38:13.340967+0000","last_undegraded":"2024-05-10T04:38:13.340967+0000","last_fullsized":"2024-05-10T04:38:13.340967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:01.655788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963212+0000","last_change":"2024-05-10T04:38:08.343813+0000","last_active":"2024-05-10T04:38:33.963212+0000","last_peered":"2024-05-10T04:38:33.963212+0000","last_clean":"2024-05-10T04:38:33.963212+0000","last_became_active":"2024-05-10T04:38:08.343309+0000","last_became_peered":"2024-05-10T04:38:08.343309+0000","last_unstale":"2024-05-10T04:38:33.963212+0000","last_undegraded":"2024-05-10T04:38:33.963212+0000","last_fullsized":"2024-05-10T04:38:33.963212+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:07.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786851+0000","last_change":"2024-05-10T04:38:08.780598+0000","last_active":"2024-05-10T04:38:12.786851+0000","last_peered":"2024-05-10T04:38:12.786851+0000","last_clean":"2024-05-10T04:38:12.786851+0000","last_became_active":"2024-05-10T04:38:08.775367+0000","last_became_peered":"2024-05-10T04:38:08.775367+0000","last_unstale":"2024-05-10T04:38:12.786851+0000","last_undegraded":"2024-05-10T04:38:12.786851+0000","last_fullsized":"2024-05-10T04:38:12.786851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:33.826116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.929645+0000","last_change":"2024-05-10T04:38:12.795975+0000","last_active":"2024-05-10T04:38:13.929645+0000","last_peered":"2024-05-10T04:38:13.929645+0000","last_clean":"2024-05-10T04:38:13.929645+0000","last_became_active":"2024-05-10T04:38:12.795582+0000","last_became_peered":"2024-05-10T04:38:12.795582+0000","last_unstale":"2024-05-10T04:38:13.929645+0000","last_undegraded":"2024-05-10T04:38:13.929645+0000","last_fullsized":"2024-05-10T04:38:13.929645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:42:30.734833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.969664+0000","last_change":"2024-05-10T04:38:08.346338+0000","last_active":"2024-05-10T04:38:34.969664+0000","last_peered":"2024-05-10T04:38:34.969664+0000","last_clean":"2024-05-10T04:38:34.969664+0000","last_became_active":"2024-05-10T04:38:08.345994+0000","last_became_peered":"2024-05-10T04:38:08.345994+0000","last_unstale":"2024-05-10T04:38:34.969664+0000","last_undegraded":"2024-05-10T04:38:34.969664+0000","last_fullsized":"2024-05-10T04:38:34.969664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:22.962514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787140+0000","last_change":"2024-05-10T04:38:08.780925+0000","last_active":"2024-05-10T04:38:12.787140+0000","last_peered":"2024-05-10T04:38:12.787140+0000","last_clean":"2024-05-10T04:38:12.787140+0000","last_became_active":"2024-05-10T04:38:08.778777+0000","last_became_peered":"2024-05-10T04:38:08.778777+0000","last_unstale":"2024-05-10T04:38:12.787140+0000","last_undegraded":"2024-05-10T04:38:12.787140+0000","last_fullsized":"2024-05-10T04:38:12.787140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:57.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786327+0000","last_change":"2024-05-10T04:38:08.780281+0000","last_active":"2024-05-10T04:38:12.786327+0000","last_peered":"2024-05-10T04:38:12.786327+0000","last_clean":"2024-05-10T04:38:12.786327+0000","last_became_active":"2024-05-10T04:38:08.779927+0000","last_became_peered":"2024-05-10T04:38:08.779927+0000","last_unstale":"2024-05-10T04:38:12.786327+0000","last_undegraded":"2024-05-10T04:38:12.786327+0000","last_fullsized":"2024-05-10T04:38:12.786327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:33.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789688+0000","last_change":"2024-05-10T04:38:08.784916+0000","last_active":"2024-05-10T04:38:12.789688+0000","last_peered":"2024-05-10T04:38:12.789688+0000","last_clean":"2024-05-10T04:38:12.789688+0000","last_became_active":"2024-05-10T04:38:08.784544+0000","last_became_peered":"2024-05-10T04:38:08.784544+0000","last_unstale":"2024-05-10T04:38:12.789688+0000","last_undegraded":"2024-05-10T04:38:12.789688+0000","last_fullsized":"2024-05-10T04:38:12.789688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:03.669897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786689+0000","last_change":"2024-05-10T04:38:08.769149+0000","last_active":"2024-05-10T04:38:12.786689+0000","last_peered":"2024-05-10T04:38:12.786689+0000","last_clean":"2024-05-10T04:38:12.786689+0000","last_became_active":"2024-05-10T04:38:08.768808+0000","last_became_peered":"2024-05-10T04:38:08.768808+0000","last_unstale":"2024-05-10T04:38:12.786689+0000","last_undegraded":"2024-05-10T04:38:12.786689+0000","last_fullsized":"2024-05-10T04:38:12.786689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:42:46.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787125+0000","last_change":"2024-05-10T04:38:08.784411+0000","last_active":"2024-05-10T04:38:12.787125+0000","last_peered":"2024-05-10T04:38:12.787125+0000","last_clean":"2024-05-10T04:38:12.787125+0000","last_became_active":"2024-05-10T04:38:08.784064+0000","last_became_peered":"2024-05-10T04:38:08.784064+0000","last_unstale":"2024-05-10T04:38:12.787125+0000","last_undegraded":"2024-05-10T04:38:12.787125+0000","last_fullsized":"2024-05-10T04:38:12.787125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:32:46.493888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782409+0000","last_change":"2024-05-10T04:38:08.779643+0000","last_active":"2024-05-10T04:38:12.782409+0000","last_peered":"2024-05-10T04:38:12.782409+0000","last_clean":"2024-05-10T04:38:12.782409+0000","last_became_active":"2024-05-10T04:38:08.779291+0000","last_became_peered":"2024-05-10T04:38:08.779291+0000","last_unstale":"2024-05-10T04:38:12.782409+0000","last_undegraded":"2024-05-10T04:38:12.782409+0000","last_fullsized":"2024-05-10T04:38:12.782409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:24.535515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786192+0000","last_change":"2024-05-10T04:38:08.789382+0000","last_active":"2024-05-10T04:38:12.786192+0000","last_peered":"2024-05-10T04:38:12.786192+0000","last_clean":"2024-05-10T04:38:12.786192+0000","last_became_active":"2024-05-10T04:38:08.789003+0000","last_became_peered":"2024-05-10T04:38:08.789003+0000","last_unstale":"2024-05-10T04:38:12.786192+0000","last_undegraded":"2024-05-10T04:38:12.786192+0000","last_fullsized":"2024-05-10T04:38:12.786192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:03.031319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788582+0000","last_change":"2024-05-10T04:38:08.777290+0000","last_active":"2024-05-10T04:38:12.788582+0000","last_peered":"2024-05-10T04:38:12.788582+0000","last_clean":"2024-05-10T04:38:12.788582+0000","last_became_active":"2024-05-10T04:38:08.776331+0000","last_became_peered":"2024-05-10T04:38:08.776331+0000","last_unstale":"2024-05-10T04:38:12.788582+0000","last_undegraded":"2024-05-10T04:38:12.788582+0000","last_fullsized":"2024-05-10T04:38:12.788582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:51.074991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789825+0000","last_change":"2024-05-10T04:38:08.780937+0000","last_active":"2024-05-10T04:38:12.789825+0000","last_peered":"2024-05-10T04:38:12.789825+0000","last_clean":"2024-05-10T04:38:12.789825+0000","last_became_active":"2024-05-10T04:38:08.780620+0000","last_became_peered":"2024-05-10T04:38:08.780620+0000","last_unstale":"2024-05-10T04:38:12.789825+0000","last_undegraded":"2024-05-10T04:38:12.789825+0000","last_fullsized":"2024-05-10T04:38:12.789825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:27.697571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781208+0000","last_change":"2024-05-10T04:38:08.773314+0000","last_active":"2024-05-10T04:38:12.781208+0000","last_peered":"2024-05-10T04:38:12.781208+0000","last_clean":"2024-05-10T04:38:12.781208+0000","last_became_active":"2024-05-10T04:38:08.772973+0000","last_became_peered":"2024-05-10T04:38:08.772973+0000","last_unstale":"2024-05-10T04:38:12.781208+0000","last_undegraded":"2024-05-10T04:38:12.781208+0000","last_fullsized":"2024-05-10T04:38:12.781208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:53.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788890+0000","last_change":"2024-05-10T04:38:08.775632+0000","last_active":"2024-05-10T04:38:12.788890+0000","last_peered":"2024-05-10T04:38:12.788890+0000","last_clean":"2024-05-10T04:38:12.788890+0000","last_became_active":"2024-05-10T04:38:08.775308+0000","last_became_peered":"2024-05-10T04:38:08.775308+0000","last_unstale":"2024-05-10T04:38:12.788890+0000","last_undegraded":"2024-05-10T04:38:12.788890+0000","last_fullsized":"2024-05-10T04:38:12.788890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:58.758863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783887+0000","last_change":"2024-05-10T04:38:08.787354+0000","last_active":"2024-05-10T04:38:12.783887+0000","last_peered":"2024-05-10T04:38:12.783887+0000","last_clean":"2024-05-10T04:38:12.783887+0000","last_became_active":"2024-05-10T04:38:08.786645+0000","last_became_peered":"2024-05-10T04:38:08.786645+0000","last_unstale":"2024-05-10T04:38:12.783887+0000","last_undegraded":"2024-05-10T04:38:12.783887+0000","last_fullsized":"2024-05-10T04:38:12.783887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:35.060271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782372+0000","last_change":"2024-05-10T04:38:08.769233+0000","last_active":"2024-05-10T04:38:12.782372+0000","last_peered":"2024-05-10T04:38:12.782372+0000","last_clean":"2024-05-10T04:38:12.782372+0000","last_became_active":"2024-05-10T04:38:08.768936+0000","last_became_peered":"2024-05-10T04:38:08.768936+0000","last_unstale":"2024-05-10T04:38:12.782372+0000","last_undegraded":"2024-05-10T04:38:12.782372+0000","last_fullsized":"2024-05-10T04:38:12.782372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:08.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786960+0000","last_change":"2024-05-10T04:38:08.788189+0000","last_active":"2024-05-10T04:38:12.786960+0000","last_peered":"2024-05-10T04:38:12.786960+0000","last_clean":"2024-05-10T04:38:12.786960+0000","last_became_active":"2024-05-10T04:38:08.787806+0000","last_became_peered":"2024-05-10T04:38:08.787806+0000","last_unstale":"2024-05-10T04:38:12.786960+0000","last_undegraded":"2024-05-10T04:38:12.786960+0000","last_fullsized":"2024-05-10T04:38:12.786960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:00:51.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280050,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8004,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":144,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":45604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481807,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":942080,"data_stored":466157,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.77700000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.749}]},{"osd":7,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.90100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.86099999999999999}]},{"osd":10,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.92700000000000005}]}]},{"osd":10,"up_from":69,"seq":296352743441,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1054551,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.501},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.751},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55600000000000005}]},{"osd":5,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.73199999999999998}]},{"osd":9,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906966,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":461432,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.30299999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.44}]},{"osd":3,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.372},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.41699999999999998}]},{"osd":4,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.65300000000000002}]},{"osd":8,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.501}]},{"osd":10,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.54700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135897,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":937984,"data_stored":461150,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.377}]},{"osd":3,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63900000000000001}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":0.71699999999999997}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":999424,"data_stored":1053260,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.532,"5min":3.532,"15min":3.532},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69399999999999995,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.3759999999999999,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.60799999999999998,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.625,"15min":0.59899999999999998},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.9609999999999999,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6179999999999999,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.72899999999999998,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4860000000000002,"5min":3.4860000000000002,"15min":3.4860000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.70699999999999996,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":3.996,"5min":3.996,"15min":3.996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.47199999999999998}]},{"osd":8,"last update":"Fri May 10 04:37:42 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.499}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.5630000000000002,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.2670000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":0.72799999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":461198,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.624},{"interface":"front","average":{"1min":0.626,"5min":0.63,"15min":0.63},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri May 10 04:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri May 10 04:37:42 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65000000000000002}]},{"osd":9,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.51100000000000001}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.72199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":958464,"data_stored":472626,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52600000000000002,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.47899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.85999999999999999}]},{"osd":3,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.94599999999999995}]},{"osd":4,"last update":"Fri May 10 04:38:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46100000000000002,"15min":0.438},"min":{"1min":0.33200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.47999999999999998,"15min":0.44700000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri May 10 04:37:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.51600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.53500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.96399999999999997}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.008,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.879}]},{"osd":8,"last update":"Fri May 10 04:37:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.018}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.048},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.0049999999999999}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.755},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.073}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":461218,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.503,"15min":0.47299999999999998},"min":{"1min":0.379,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83299999999999996,"5min":0.996,"15min":0.996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.5,"15min":0.46800000000000003},"min":{"1min":0.40300000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71099999999999997,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri May 10 04:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.68500000000000005,"5min":1.123,"15min":1.123},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.65100000000000002}]},{"osd":3,"last update":"Fri May 10 04:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.623},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85899999999999999,"5min":1.093,"15min":1.093},"last":1.0089999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81000000000000005,"5min":1.151,"15min":1.151},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.86699999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:07 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.95699999999999996}]},{"osd":7,"last update":"Fri May 10 04:37:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.997}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":1.181},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.0389999999999999}]},{"osd":9,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.131}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":1.075},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.026}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":464928,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.502,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.47999999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.67800000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80000000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.85799999999999998}]},{"osd":4,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64200000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68000000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri May 10 04:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69299999999999995}]},{"osd":8,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri May 10 04:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.90900000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.873}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":460932,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35799999999999998,"15min":0.34000000000000002},"min":{"1min":0.31900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri May 10 04:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.626},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.48599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri May 10 04:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.502,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78300000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri May 10 04:38:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69599999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96799999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri May 10 04:38:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46700000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":4,"up_from":33,"seq":141733920807,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":460546,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":1.143,"15min":1.143},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.35699999999999998},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.36599999999999999,"15min":0.35599999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.51300000000000001,"15min":0.5},"min":{"1min":0.41299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.53800000000000003,"15min":0.52100000000000002},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92600000000000005,"5min":1.03,"15min":1.03},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri May 10 04:37:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0620000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67200000000000004,"5min":1.069,"15min":1.069},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":1.079,"15min":1.079},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.71599999999999997}]},{"osd":9,"last update":"Fri May 10 04:38:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78700000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757284,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1053082,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.54000000000000004,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86199999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.44700000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.28299999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri May 10 04:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86699999999999999,"5min":1.089,"15min":1.089},"last":0.626},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.749}]},{"osd":6,"last update":"Fri May 10 04:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.48399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66800000000000004}]},{"osd":8,"last update":"Fri May 10 04:37:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68600000000000005}]},{"osd":9,"last update":"Fri May 10 04:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:38:38.207 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-10T04:38:38.208 INFO:tasks.cephadm:Setting up client nodes... 2024-05-10T04:38:38.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-10T04:38:38.466 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:38.585 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: 2024-05-10T04:38:38.287+0000 7fa2dad01700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-10T04:38:38.585 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:38:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: 2024-05-10T04:38:38.287+0000 7fa2dad01700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-10T04:38:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:38 smithi161 ceph-mon[46912]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:38 smithi161 ceph-mon[46912]: pgmap v239: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-10T04:38:38.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:38 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/276510158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:38.603 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:38 smithi150 ceph-mon[42404]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:38.603 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:38 smithi150 ceph-mon[42404]: pgmap v239: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-10T04:38:38.603 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:38 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/276510158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:38.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:38 smithi181 ceph-mon[47314]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:38:38.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:38 smithi181 ceph-mon[47314]: pgmap v239: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-10T04:38:38.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:38 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/276510158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:39.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.381+0000 7fca4930b700 1 -- 172.21.15.150:0/3146076728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 msgr2=0x7fca44101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:39.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.381+0000 7fca4930b700 1 --2- 172.21.15.150:0/3146076728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44101f90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009990 tx=0x7fca2c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.381+0000 7fca4930b700 1 -- 172.21.15.150:0/3146076728 shutdown_connections 2024-05-10T04:38:39.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.381+0000 7fca4930b700 1 --2- 172.21.15.150:0/3146076728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca44102620 0x7fca441090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.381+0000 7fca4930b700 1 --2- 172.21.15.150:0/3146076728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca441010e0 0x7fca441014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.381+0000 7fca4930b700 1 --2- 172.21.15.150:0/3146076728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44101f90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.381+0000 7fca4930b700 1 -- 172.21.15.150:0/3146076728 >> 172.21.15.150:0/3146076728 conn(0x7fca440fb950 msgr2=0x7fca440fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:39.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.382+0000 7fca4930b700 1 -- 172.21.15.150:0/3146076728 shutdown_connections 2024-05-10T04:38:39.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.382+0000 7fca4930b700 1 -- 172.21.15.150:0/3146076728 wait complete. 2024-05-10T04:38:39.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.382+0000 7fca4930b700 1 Processor -- start 2024-05-10T04:38:39.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.382+0000 7fca4930b700 1 -- start start 2024-05-10T04:38:39.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca441010e0 0x7fca44197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca44102620 0x7fca44198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44198970 con 0x7fca44101ac0 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44198ab0 con 0x7fca44102620 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44198c70 con 0x7fca441010e0 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57230/0 (socket says 172.21.15.150:57230) 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 -- 172.21.15.150:0/3004495777 learned_addr learned my addr 172.21.15.150:0/3004495777 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca42ffd700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca441010e0 0x7fca44197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca437fe700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca44102620 0x7fca44198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 -- 172.21.15.150:0/3004495777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca441010e0 msgr2=0x7fca44197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca441010e0 0x7fca44197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 -- 172.21.15.150:0/3004495777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca44102620 msgr2=0x7fca44198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca44102620 0x7fca44198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 -- 172.21.15.150:0/3004495777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c0095f0 con 0x7fca44101ac0 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca427fc700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44197560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0098e0 tx=0x7fca2c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca3bfff700 1 -- 172.21.15.150:0/3004495777 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c01d070 con 0x7fca44101ac0 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca44198ef0 con 0x7fca44101ac0 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca3bfff700 1 -- 172.21.15.150:0/3004495777 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca2c021410 con 0x7fca44101ac0 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca3bfff700 1 -- 172.21.15.150:0/3004495777 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c017650 con 0x7fca44101ac0 2024-05-10T04:38:39.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.383+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4419f730 con 0x7fca44101ac0 2024-05-10T04:38:39.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.385+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca44107fc0 con 0x7fca44101ac0 2024-05-10T04:38:39.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.386+0000 7fca3bfff700 1 -- 172.21.15.150:0/3004495777 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fca2c021580 con 0x7fca44101ac0 2024-05-10T04:38:39.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.386+0000 7fca3bfff700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca300a11b0 0x7fca300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:39.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.386+0000 7fca3bfff700 1 -- 172.21.15.150:0/3004495777 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fca2c0ce080 con 0x7fca44101ac0 2024-05-10T04:38:39.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.386+0000 7fca42ffd700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca300a11b0 0x7fca300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:39.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.387+0000 7fca42ffd700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca300a11b0 0x7fca300a36c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca34005950 tx=0x7fca3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:39.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.391+0000 7fca3bfff700 1 -- 172.21.15.150:0/3004495777 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fca2c097550 con 0x7fca44101ac0 2024-05-10T04:38:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:39 smithi150 ceph-mon[42404]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:38:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:39 smithi150 ceph-mon[42404]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:38:39.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.668+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fca440ff610 con 0x7fca44101ac0 2024-05-10T04:38:39.676 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.675+0000 7fca3bfff700 1 -- 172.21.15.150:0/3004495777 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fca2c09ab70 con 0x7fca44101ac0 2024-05-10T04:38:39.677 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-05-10T04:38:39.677 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDPpD1mVr/uJxAANHynf/3fhptLB8db63ELWA== 2024-05-10T04:38:39.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca300a11b0 msgr2=0x7fca300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:39.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca300a11b0 0x7fca300a36c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca34005950 tx=0x7fca3400b410 comp rx=0 tx=0).stop 2024-05-10T04:38:39.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 msgr2=0x7fca44197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:39.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44197560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0098e0 tx=0x7fca2c009580 comp rx=0 tx=0).stop 2024-05-10T04:38:39.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 shutdown_connections 2024-05-10T04:38:39.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca44102620 0x7fca44198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca300a11b0 0x7fca300a36c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca441010e0 0x7fca44197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 --2- 172.21.15.150:0/3004495777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca44101ac0 0x7fca44197560 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:39.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 >> 172.21.15.150:0/3004495777 conn(0x7fca440fb950 msgr2=0x7fca44105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:39.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 shutdown_connections 2024-05-10T04:38:39.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:39.678+0000 7fca4930b700 1 -- 172.21.15.150:0/3004495777 wait complete. 2024-05-10T04:38:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:39 smithi161 ceph-mon[46912]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:38:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:39 smithi161 ceph-mon[46912]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:38:39.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:39 smithi181 ceph-mon[47314]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:38:39.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:39 smithi181 ceph-mon[47314]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:38:40.075 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:38:40.075 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-10T04:38:40.075 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-10T04:38:40.117 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-10T04:38:40.117 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-10T04:38:40.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-10T04:38:40.380 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:40.761 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:40 smithi150 ceph-mon[42404]: pgmap v240: 129 pgs: 129 active+clean; 622 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.6 KiB/s wr, 18 op/s 2024-05-10T04:38:40.762 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:40 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3004495777' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T04:38:40.762 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:40 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3004495777' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T04:38:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:40 smithi161 ceph-mon[46912]: pgmap v240: 129 pgs: 129 active+clean; 622 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.6 KiB/s wr, 18 op/s 2024-05-10T04:38:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:40 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3004495777' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T04:38:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:40 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3004495777' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T04:38:40.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:40 smithi181 ceph-mon[47314]: pgmap v240: 129 pgs: 129 active+clean; 622 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 7.6 KiB/s wr, 18 op/s 2024-05-10T04:38:40.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:40 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3004495777' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-10T04:38:40.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:40 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3004495777' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 -- 172.21.15.150:0/2232530355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4100ec0 msgr2=0x7f7cf41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/2232530355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4100ec0 0x7f7cf41012d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009990 tx=0x7f7ce8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 -- 172.21.15.150:0/2232530355 shutdown_connections 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/2232530355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf4106140 0x7f7cf410daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/2232530355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4100ec0 0x7f7cf41012d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/2232530355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf41057f0 0x7f7cf4105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 -- 172.21.15.150:0/2232530355 >> 172.21.15.150:0/2232530355 conn(0x7f7cf4076720 msgr2=0x7f7cf4076b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 -- 172.21.15.150:0/2232530355 shutdown_connections 2024-05-10T04:38:41.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.272+0000 7f7cfaf95700 1 -- 172.21.15.150:0/2232530355 wait complete. 2024-05-10T04:38:41.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 Processor -- start 2024-05-10T04:38:41.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 -- start start 2024-05-10T04:38:41.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf4100ec0 0x7f7cf4199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:41.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf41057f0 0x7f7cf4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:41.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4106140 0x7f7cf41a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf419a260 con 0x7f7cf4106140 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf41a4980 con 0x7f7cf41057f0 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cfaf95700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf41a4b40 con 0x7f7cf4100ec0 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.273+0000 7f7cf9532700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4106140 0x7f7cf41a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4106140 0x7f7cf41a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57246/0 (socket says 172.21.15.150:57246) 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 -- 172.21.15.150:0/3168732398 learned_addr learned my addr 172.21.15.150:0/3168732398 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf3fff700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf41057f0 0x7f7cf4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf8d31700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf4100ec0 0x7f7cf4199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 -- 172.21.15.150:0/3168732398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf4100ec0 msgr2=0x7f7cf4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf4100ec0 0x7f7cf4199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 -- 172.21.15.150:0/3168732398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf41057f0 msgr2=0x7f7cf4199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf41057f0 0x7f7cf4199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 -- 172.21.15.150:0/3168732398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce80095f0 con 0x7f7cf4106140 2024-05-10T04:38:41.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf9532700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4106140 0x7f7cf41a4440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00ada0 tx=0x7f7cec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:41.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf1ffb700 1 -- 172.21.15.150:0/3168732398 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec00f070 con 0x7f7cf4106140 2024-05-10T04:38:41.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf1ffb700 1 -- 172.21.15.150:0/3168732398 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cec007b30 con 0x7f7cf4106140 2024-05-10T04:38:41.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cf1ffb700 1 -- 172.21.15.150:0/3168732398 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0053e0 con 0x7f7cf4106140 2024-05-10T04:38:41.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf41a4e20 con 0x7f7cf4106140 2024-05-10T04:38:41.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.274+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf41a5210 con 0x7f7cf4106140 2024-05-10T04:38:41.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.275+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf4109190 con 0x7f7cf4106140 2024-05-10T04:38:41.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.277+0000 7f7cf1ffb700 1 -- 172.21.15.150:0/3168732398 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7cec00c980 con 0x7f7cf4106140 2024-05-10T04:38:41.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.277+0000 7f7cf1ffb700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7ce40a11b0 0x7f7ce40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:41.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.277+0000 7f7cf1ffb700 1 -- 172.21.15.150:0/3168732398 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7cec0cc7c0 con 0x7f7cf4106140 2024-05-10T04:38:41.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.277+0000 7f7cf8d31700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7ce40a11b0 0x7f7ce40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:41.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.278+0000 7f7cf8d31700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7ce40a11b0 0x7f7ce40a36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006010 tx=0x7f7ce800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:41.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.281+0000 7f7cf1ffb700 1 -- 172.21.15.150:0/3168732398 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7cec095c90 con 0x7f7cf4106140 2024-05-10T04:38:41.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.538+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7cf4102370 con 0x7f7cf4106140 2024-05-10T04:38:41.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.543+0000 7f7cf1ffb700 1 -- 172.21.15.150:0/3168732398 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260295 (secure 0 0 0) 0x7f7cec0992b0 con 0x7f7cf4106140 2024-05-10T04:38:41.548 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7ce40a11b0 msgr2=0x7f7ce40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7ce40a11b0 0x7f7ce40a36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006010 tx=0x7f7ce800b540 comp rx=0 tx=0).stop 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4106140 msgr2=0x7f7cf41a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4106140 0x7f7cf41a4440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00ada0 tx=0x7f7cec00c330 comp rx=0 tx=0).stop 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 shutdown_connections 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cf4106140 0x7f7cf41a4440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7ce40a11b0 0x7f7ce40a36c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf4100ec0 0x7f7cf4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 --2- 172.21.15.150:0/3168732398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf41057f0 0x7f7cf4199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:41.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 >> 172.21.15.150:0/3168732398 conn(0x7f7cf4076720 msgr2=0x7f7cf40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:41.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 shutdown_connections 2024-05-10T04:38:41.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:41.546+0000 7f7cfaf95700 1 -- 172.21.15.150:0/3168732398 wait complete. 2024-05-10T04:38:41.728 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:41 smithi150 ceph-mon[42404]: pgmap v241: 129 pgs: 129 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 KiB/s wr, 20 op/s 2024-05-10T04:38:41.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:41 smithi161 ceph-mon[46912]: pgmap v241: 129 pgs: 129 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 KiB/s wr, 20 op/s 2024-05-10T04:38:41.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:41 smithi181 ceph-mon[47314]: pgmap v241: 129 pgs: 129 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 KiB/s wr, 20 op/s 2024-05-10T04:38:41.933 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3064659046},{"type":"v1","addr":"172.21.15.150:6801","nonce":3064659046}]},"active_addr":"172.21.15.150:6801/3064659046","active_change":"2024-05-10T04:33:11.238519+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.150:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2510422392}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2863686883}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2036243910}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":3461201232}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2167568633}]}]} 2024-05-10T04:38:41.941 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-10T04:38:41.941 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-10T04:38:41.941 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T04:38:42.166 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:42.603 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:42 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3168732398' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T04:38:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:42 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3168732398' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T04:38:42.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:42 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3168732398' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-10T04:38:43.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 -- 172.21.15.150:0/3111384862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128100e00 msgr2=0x7fe128101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:43.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 --2- 172.21.15.150:0/3111384862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128100e00 0x7fe128101210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009990 tx=0x7fe11c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 -- 172.21.15.150:0/3111384862 shutdown_connections 2024-05-10T04:38:43.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 --2- 172.21.15.150:0/3111384862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe128106080 0x7fe12810d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 --2- 172.21.15.150:0/3111384862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128100e00 0x7fe128101210 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 --2- 172.21.15.150:0/3111384862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe128105730 0x7fe128105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 -- 172.21.15.150:0/3111384862 >> 172.21.15.150:0/3111384862 conn(0x7fe1280fc7a0 msgr2=0x7fe1280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:43.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 -- 172.21.15.150:0/3111384862 shutdown_connections 2024-05-10T04:38:43.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.114+0000 7fe12fa41700 1 -- 172.21.15.150:0/3111384862 wait complete. 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 Processor -- start 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 -- start start 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe128100e00 0x7fe1281984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe128105730 0x7fe128198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128106080 0x7fe12819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12819d4c0 con 0x7fe128106080 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12819d630 con 0x7fe128105730 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12fa41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12819d7f0 con 0x7fe128100e00 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12dfde700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128106080 0x7fe12819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12dfde700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128106080 0x7fe12819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57272/0 (socket says 172.21.15.150:57272) 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12dfde700 1 -- 172.21.15.150:0/1172063265 learned_addr learned my addr 172.21.15.150:0/1172063265 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.115+0000 7fe12d7dd700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe128100e00 0x7fe1281984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:43.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12cfdc700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe128105730 0x7fe128198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12dfde700 1 -- 172.21.15.150:0/1172063265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe128100e00 msgr2=0x7fe1281984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12dfde700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe128100e00 0x7fe1281984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12dfde700 1 -- 172.21.15.150:0/1172063265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe128105730 msgr2=0x7fe128198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12dfde700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe128105730 0x7fe128198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12dfde700 1 -- 172.21.15.150:0/1172063265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c0095f0 con 0x7fe128106080 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12d7dd700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe128100e00 0x7fe1281984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12cfdc700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe128105730 0x7fe128198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12dfde700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128106080 0x7fe12819cf50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe120007ee0 tx=0x7fe12000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe11a7fc700 1 -- 172.21.15.150:0/1172063265 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120017070 con 0x7fe128106080 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe11a7fc700 1 -- 172.21.15.150:0/1172063265 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe12000f040 con 0x7fe128106080 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12819dad0 con 0x7fe128106080 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe11a7fc700 1 -- 172.21.15.150:0/1172063265 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120005740 con 0x7fe128106080 2024-05-10T04:38:43.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.116+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12810fe40 con 0x7fe128106080 2024-05-10T04:38:43.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.117+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1281090d0 con 0x7fe128106080 2024-05-10T04:38:43.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.118+0000 7fe11a7fc700 1 -- 172.21.15.150:0/1172063265 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe1200058a0 con 0x7fe128106080 2024-05-10T04:38:43.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.119+0000 7fe11a7fc700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1140a11b0 0x7fe1140a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:43.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.119+0000 7fe12d7dd700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1140a11b0 0x7fe1140a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:43.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.119+0000 7fe11a7fc700 1 -- 172.21.15.150:0/1172063265 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe1200cc4a0 con 0x7fe128106080 2024-05-10T04:38:43.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.120+0000 7fe12d7dd700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1140a11b0 0x7fe1140a36c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00b2e0 tx=0x7fe11c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:43.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.124+0000 7fe11a7fc700 1 -- 172.21.15.150:0/1172063265 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe1200958e0 con 0x7fe128106080 2024-05-10T04:38:43.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.322+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1281022b0 con 0x7fe128106080 2024-05-10T04:38:43.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.324+0000 7fe11a7fc700 1 -- 172.21.15.150:0/1172063265 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7fe120098f00 con 0x7fe128106080 2024-05-10T04:38:43.325 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:43.325 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:43.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1140a11b0 msgr2=0x7fe1140a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:43.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1140a11b0 0x7fe1140a36c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00b2e0 tx=0x7fe11c01a040 comp rx=0 tx=0).stop 2024-05-10T04:38:43.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128106080 msgr2=0x7fe12819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:43.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128106080 0x7fe12819cf50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe120007ee0 tx=0x7fe12000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 shutdown_connections 2024-05-10T04:38:43.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe128106080 0x7fe12819cf50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe1140a11b0 0x7fe1140a36c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe128100e00 0x7fe1281984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 --2- 172.21.15.150:0/1172063265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe128105730 0x7fe128198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:43.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 >> 172.21.15.150:0/1172063265 conn(0x7fe1280fc7a0 msgr2=0x7fe128104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:43.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 shutdown_connections 2024-05-10T04:38:43.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:43.326+0000 7fe12fa41700 1 -- 172.21.15.150:0/1172063265 wait complete. 2024-05-10T04:38:43.606 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:43 smithi150 ceph-mon[42404]: pgmap v242: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-05-10T04:38:43.606 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:43 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1172063265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:43.726 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-10T04:38:43.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-10T04:38:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:43 smithi161 ceph-mon[46912]: pgmap v242: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-05-10T04:38:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:43 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1172063265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:43.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:43 smithi181 ceph-mon[47314]: pgmap v242: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-05-10T04:38:43.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:43 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1172063265' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:43.949 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:44.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 -- 172.21.15.150:0/360855047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c1001d0 msgr2=0x7f940c1005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:44.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 --2- 172.21.15.150:0/360855047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c1001d0 0x7f940c1005e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009990 tx=0x7f93fc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:44.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 -- 172.21.15.150:0/360855047 shutdown_connections 2024-05-10T04:38:44.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 --2- 172.21.15.150:0/360855047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f940c100b20 0x7f940c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:44.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 --2- 172.21.15.150:0/360855047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f940c106c30 0x7f940c107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:44.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 --2- 172.21.15.150:0/360855047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c1001d0 0x7f940c1005e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:44.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 -- 172.21.15.150:0/360855047 >> 172.21.15.150:0/360855047 conn(0x7f940c0fbe80 msgr2=0x7f940c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:44.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 -- 172.21.15.150:0/360855047 shutdown_connections 2024-05-10T04:38:44.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.890+0000 7f94115fd700 1 -- 172.21.15.150:0/360855047 wait complete. 2024-05-10T04:38:44.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 Processor -- start 2024-05-10T04:38:44.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 -- start start 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f940c1001d0 0x7f940c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c100b20 0x7f940c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f940c106c30 0x7f940c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c19d570 con 0x7f940c100b20 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c19d6e0 con 0x7f940c106c30 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.891+0000 7f94115fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c19d8a0 con 0x7f940c1001d0 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c100b20 0x7f940c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:44.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c100b20 0x7f940c198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:57280/0 (socket says 172.21.15.150:57280) 2024-05-10T04:38:44.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 -- 172.21.15.150:0/1069630774 learned_addr learned my addr 172.21.15.150:0/1069630774 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940affd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f940c1001d0 0x7f940c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940b7fe700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f940c106c30 0x7f940c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 -- 172.21.15.150:0/1069630774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f940c1001d0 msgr2=0x7f940c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f940c1001d0 0x7f940c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 -- 172.21.15.150:0/1069630774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f940c106c30 msgr2=0x7f940c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f940c106c30 0x7f940c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 -- 172.21.15.150:0/1069630774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0095f0 con 0x7f940c100b20 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940b7fe700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f940c106c30 0x7f940c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:44.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f940a7fc700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c100b20 0x7f940c198b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0098e0 tx=0x7f93fc00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:44.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f93f3fff700 1 -- 172.21.15.150:0/1069630774 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc01d070 con 0x7f940c100b20 2024-05-10T04:38:44.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c19db20 con 0x7f940c100b20 2024-05-10T04:38:44.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f93f3fff700 1 -- 172.21.15.150:0/1069630774 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93fc022470 con 0x7f940c100b20 2024-05-10T04:38:44.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f93f3fff700 1 -- 172.21.15.150:0/1069630774 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc017650 con 0x7f940c100b20 2024-05-10T04:38:44.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.892+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c1a52b0 con 0x7f940c100b20 2024-05-10T04:38:44.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.893+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c10a140 con 0x7f940c100b20 2024-05-10T04:38:44.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.895+0000 7f93f3fff700 1 -- 172.21.15.150:0/1069630774 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f93fc0225e0 con 0x7f940c100b20 2024-05-10T04:38:44.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.896+0000 7f93f3fff700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93f40a11b0 0x7f93f40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:44.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.896+0000 7f940affd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93f40a11b0 0x7f93f40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:44.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.896+0000 7f93f3fff700 1 -- 172.21.15.150:0/1069630774 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f93fc0ce0a0 con 0x7f940c100b20 2024-05-10T04:38:44.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.896+0000 7f940affd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93f40a11b0 0x7f93f40a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9400005950 tx=0x7f940000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:44.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:44.900+0000 7f93f3fff700 1 -- 172.21.15.150:0/1069630774 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f93fc09b120 con 0x7f940c100b20 2024-05-10T04:38:45.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.097+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f940c1034f0 con 0x7f940c100b20 2024-05-10T04:38:45.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.098+0000 7f93f3fff700 1 -- 172.21.15.150:0/1069630774 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7f93fc09acb0 con 0x7f940c100b20 2024-05-10T04:38:45.100 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:38:45.100 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:38:45.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93f40a11b0 msgr2=0x7f93f40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:45.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93f40a11b0 0x7f93f40a36c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9400005950 tx=0x7f940000b410 comp rx=0 tx=0).stop 2024-05-10T04:38:45.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c100b20 msgr2=0x7f940c198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:45.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c100b20 0x7f940c198b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0098e0 tx=0x7f93fc00f670 comp rx=0 tx=0).stop 2024-05-10T04:38:45.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 shutdown_connections 2024-05-10T04:38:45.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f940c106c30 0x7f940c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:45.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93f40a11b0 0x7f93f40a36c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:45.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f940c1001d0 0x7f940c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:45.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 --2- 172.21.15.150:0/1069630774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f940c100b20 0x7f940c198b40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:45.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 >> 172.21.15.150:0/1069630774 conn(0x7f940c0fbe80 msgr2=0x7f940c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:45.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 shutdown_connections 2024-05-10T04:38:45.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:45.101+0000 7f94115fd700 1 -- 172.21.15.150:0/1069630774 wait complete. 2024-05-10T04:38:45.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:45 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1069630774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:45.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:45 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1069630774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:45.458 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-10T04:38:45.458 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-10T04:38:45.458 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-10T04:38:45.459 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-10T04:38:45.459 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-10T04:38:45.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-10T04:38:45.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-10T04:38:45.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-10T04:38:45.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-10T04:38:45.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-10T04:38:45.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-10T04:38:45.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-10T04:38:45.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:45 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1069630774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:38:45.983 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:45.999 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:46.159 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:46.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:46 smithi150 ceph-mon[42404]: pgmap v243: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 23 op/s 2024-05-10T04:38:46.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:46 smithi161 ceph-mon[46912]: pgmap v243: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 23 op/s 2024-05-10T04:38:46.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:46 smithi181 ceph-mon[47314]: pgmap v243: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 23 op/s 2024-05-10T04:38:47.720 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:47.720 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:47.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 -- 172.21.15.150:0/2465810313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 msgr2=0x7f19cc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 --2- 172.21.15.150:0/2465810313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 0x7f19cc1031b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009990 tx=0x7f19c8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 -- 172.21.15.150:0/2465810313 shutdown_connections 2024-05-10T04:38:47.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 --2- 172.21.15.150:0/2465810313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19cc10c210 0x7f19cc10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 --2- 172.21.15.150:0/2465810313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 0x7f19cc1031b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 --2- 172.21.15.150:0/2465810313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19cc1036f0 0x7f19cc10bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 -- 172.21.15.150:0/2465810313 >> 172.21.15.150:0/2465810313 conn(0x7f19cc0fc7a0 msgr2=0x7f19cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:47.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 -- 172.21.15.150:0/2465810313 shutdown_connections 2024-05-10T04:38:47.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.768+0000 7f19d3ce9700 1 -- 172.21.15.150:0/2465810313 wait complete. 2024-05-10T04:38:47.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 Processor -- start 2024-05-10T04:38:47.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 -- start start 2024-05-10T04:38:47.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 0x7f19cc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19cc1036f0 0x7f19cc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19cc10c210 0x7f19cc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc19d520 con 0x7f19cc102da0 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc19d690 con 0x7f19cc10c210 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d3ce9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc19d850 con 0x7f19cc1036f0 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 0x7f19cc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 0x7f19cc198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51570/0 (socket says 172.21.15.150:51570) 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 -- 172.21.15.150:0/3953827897 learned_addr learned my addr 172.21.15.150:0/3953827897 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1284700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19cc1036f0 0x7f19cc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d2286700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19cc10c210 0x7f19cc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 -- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19cc1036f0 msgr2=0x7f19cc198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19cc1036f0 0x7f19cc198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 -- 172.21.15.150:0/3953827897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19cc10c210 msgr2=0x7f19cc19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19cc10c210 0x7f19cc19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.769+0000 7f19d1a85700 1 -- 172.21.15.150:0/3953827897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c80095f0 con 0x7f19cc102da0 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.770+0000 7f19d1a85700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 0x7f19cc198500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f19c80098e0 tx=0x7f19c800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.770+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c801d070 con 0x7f19cc102da0 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.770+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19c8022470 con 0x7f19cc102da0 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.770+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19cc19dad0 con 0x7f19cc102da0 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.770+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8017650 con 0x7f19cc102da0 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.770+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc1a5260 con 0x7f19cc102da0 2024-05-10T04:38:47.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.771+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19cc111e50 con 0x7f19cc102da0 2024-05-10T04:38:47.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.772+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f19c80225e0 con 0x7f19cc102da0 2024-05-10T04:38:47.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19c2ffd700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f19b80a1230 0x7f19b80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19d1284700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f19b80a1230 0x7f19b80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f19c80ce400 con 0x7f19cc102da0 2024-05-10T04:38:47.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19c2ffd700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] conn(0x7f19b80abe00 0x7f19b80ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 --> [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19b80ae960 con 0x7f19b80abe00 2024-05-10T04:38:47.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19c80ce810 con 0x7f19cc102da0 2024-05-10T04:38:47.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19d1284700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f19b80a1230 0x7f19b80a3740 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0ffca0 tx=0x7f19bc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:47.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.773+0000 7f19d2286700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] conn(0x7f19b80abe00 0x7f19b80ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.775 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.774+0000 7f19d2286700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] conn(0x7f19b80abe00 0x7f19b80ae290 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:47.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.775+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== osd.2 v2:172.21.15.181:6800/2822398363 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f19b80ae960 con 0x7f19b80abe00 2024-05-10T04:38:47.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.791+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 --> [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19cc061890 con 0x7f19b80abe00 2024-05-10T04:38:47.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.791+0000 7f19c2ffd700 1 -- 172.21.15.150:0/3953827897 <== osd.2 v2:172.21.15.181:6800/2822398363 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f19cc061890 con 0x7f19b80abe00 2024-05-10T04:38:47.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] conn(0x7f19b80abe00 msgr2=0x7f19b80ae290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] conn(0x7f19b80abe00 0x7f19b80ae290 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f19b80a1230 msgr2=0x7f19b80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f19b80a1230 0x7f19b80a3740 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0ffca0 tx=0x7f19bc00a300 comp rx=0 tx=0).stop 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 msgr2=0x7f19cc198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 --2- 172.21.15.150:0/3953827897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19cc102da0 0x7f19cc198500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f19c80098e0 tx=0x7f19c800f670 comp rx=0 tx=0).stop 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d2286700 1 -- 172.21.15.150:0/3953827897 reap_dead start 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 shutdown_connections 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 >> 172.21.15.150:0/3953827897 conn(0x7f19cc0fc7a0 msgr2=0x7f19cc0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 shutdown_connections 2024-05-10T04:38:47.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.792+0000 7f19d3ce9700 1 -- 172.21.15.150:0/3953827897 wait complete. 2024-05-10T04:38:47.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 -- 172.21.15.150:0/3206656641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8103710 msgr2=0x7f2ef810bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 --2- 172.21.15.150:0/3206656641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8103710 0x7f2ef810bc60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009990 tx=0x7f2eec009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 -- 172.21.15.150:0/3206656641 shutdown_connections 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 --2- 172.21.15.150:0/3206656641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ef810c230 0x7f2ef810e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 --2- 172.21.15.150:0/3206656641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef8102dc0 0x7f2ef81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 --2- 172.21.15.150:0/3206656641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8103710 0x7f2ef810bc60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 -- 172.21.15.150:0/3206656641 >> 172.21.15.150:0/3206656641 conn(0x7f2ef80fc7a0 msgr2=0x7f2ef80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 -- 172.21.15.150:0/3206656641 shutdown_connections 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 -- 172.21.15.150:0/3206656641 wait complete. 2024-05-10T04:38:47.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.878+0000 7f2efd715700 1 Processor -- start 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2efd715700 1 -- start start 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2efd715700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8102dc0 0x7f2ef81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2efd715700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ef8103710 0x7f2ef8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2efd715700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef810c230 0x7f2ef819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2efd715700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef819d4f0 con 0x7f2ef8102dc0 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2efd715700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef819d660 con 0x7f2ef810c230 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2efd715700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef819d820 con 0x7f2ef8103710 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8102dc0 0x7f2ef81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8102dc0 0x7f2ef81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51588/0 (socket says 172.21.15.150:51588) 2024-05-10T04:38:47.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 -- 172.21.15.150:0/123626693 learned_addr learned my addr 172.21.15.150:0/123626693 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef77fe700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef810c230 0x7f2ef819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef67fc700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ef8103710 0x7f2ef8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 -- 172.21.15.150:0/123626693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ef8103710 msgr2=0x7f2ef8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ef8103710 0x7f2ef8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 -- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef810c230 msgr2=0x7f2ef819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef810c230 0x7f2ef819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.879+0000 7f2ef6ffd700 1 -- 172.21.15.150:0/123626693 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0095f0 con 0x7f2ef8102dc0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.880+0000 7f2ef6ffd700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8102dc0 0x7f2ef81984d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800d980 tx=0x7f2ee800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.880+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800b950 con 0x7f2ef8102dc0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.880+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef819db00 con 0x7f2ef8102dc0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.880+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee8007620 con 0x7f2ef8102dc0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.880+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800f3c0 con 0x7f2ef8102dc0 2024-05-10T04:38:47.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.880+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef81a5400 con 0x7f2ef8102dc0 2024-05-10T04:38:47.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.881+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ef8111e00 con 0x7f2ef8102dc0 2024-05-10T04:38:47.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.882+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2ee8010640 con 0x7f2ef8102dc0 2024-05-10T04:38:47.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2edffff700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ee00a1230 0x7f2ee00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2ef67fc700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ee00a1230 0x7f2ee00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2ee80cca40 con 0x7f2ef8102dc0 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2edffff700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] conn(0x7f2ee00abe00 0x7f2ee00ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 --> [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ee00ae960 con 0x7f2ee00abe00 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ee80d12c0 con 0x7f2ef8102dc0 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2ef67fc700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ee00a1230 0x7f2ee00a3740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2eec006010 tx=0x7f2eec00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.883+0000 7f2ef77fe700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] conn(0x7f2ee00abe00 0x7f2ee00ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:47.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.884+0000 7f2ef77fe700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] conn(0x7f2ee00abe00 0x7f2ee00ae290 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:47.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.885+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== osd.7 v2:172.21.15.161:6816/724331973 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2ee00ae960 con 0x7f2ee00abe00 2024-05-10T04:38:47.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.900+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 --> [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ef8061910 con 0x7f2ee00abe00 2024-05-10T04:38:47.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2edffff700 1 -- 172.21.15.150:0/123626693 <== osd.7 v2:172.21.15.161:6816/724331973 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2ef8061910 con 0x7f2ee00abe00 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] conn(0x7f2ee00abe00 msgr2=0x7f2ee00ae290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] conn(0x7f2ee00abe00 0x7f2ee00ae290 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ee00a1230 msgr2=0x7f2ee00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ee00a1230 0x7f2ee00a3740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2eec006010 tx=0x7f2eec00b540 comp rx=0 tx=0).stop 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8102dc0 msgr2=0x7f2ef81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 --2- 172.21.15.150:0/123626693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ef8102dc0 0x7f2ef81984d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800d980 tx=0x7f2ee800dc90 comp rx=0 tx=0).stop 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2ef77fe700 1 -- 172.21.15.150:0/123626693 reap_dead start 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 shutdown_connections 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 >> 172.21.15.150:0/123626693 conn(0x7f2ef80fc7a0 msgr2=0x7f2ef80fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 shutdown_connections 2024-05-10T04:38:47.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:47.901+0000 7f2efd715700 1 -- 172.21.15.150:0/123626693 wait complete. 2024-05-10T04:38:48.423 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:48 smithi150 ceph-mon[42404]: pgmap v244: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 KiB/s wr, 19 op/s 2024-05-10T04:38:48.597 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:48.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:48 smithi181 ceph-mon[47314]: pgmap v244: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 KiB/s wr, 19 op/s 2024-05-10T04:38:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:48 smithi161 ceph-mon[46912]: pgmap v244: 129 pgs: 129 active+clean; 628 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 KiB/s wr, 19 op/s 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 -- 172.21.15.150:0/660003848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c106140 msgr2=0x7f1d5c10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 --2- 172.21.15.150:0/660003848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c106140 0x7f1d5c10daa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400b0a0 tx=0x7f1d5400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 -- 172.21.15.150:0/660003848 shutdown_connections 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 --2- 172.21.15.150:0/660003848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c106140 0x7f1d5c10daa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 --2- 172.21.15.150:0/660003848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d5c100ec0 0x7f1d5c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 --2- 172.21.15.150:0/660003848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c1057f0 0x7f1d5c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 -- 172.21.15.150:0/660003848 >> 172.21.15.150:0/660003848 conn(0x7f1d5c076720 msgr2=0x7f1d5c076b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 -- 172.21.15.150:0/660003848 shutdown_connections 2024-05-10T04:38:48.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.918+0000 7f1d611e5700 1 -- 172.21.15.150:0/660003848 wait complete. 2024-05-10T04:38:48.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 Processor -- start 2024-05-10T04:38:48.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 -- start start 2024-05-10T04:38:48.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c100ec0 0x7f1d5c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:48.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d5c1057f0 0x7f1d5c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:48.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c106140 0x7f1d5c1a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:48.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1a4a80 con 0x7f1d5c100ec0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1a4bc0 con 0x7f1d5c106140 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.919+0000 7f1d611e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1a4d00 con 0x7f1d5c1057f0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c100ec0 0x7f1d5c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c100ec0 0x7f1d5c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51606/0 (socket says 172.21.15.150:51606) 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 -- 172.21.15.150:0/3665305974 learned_addr learned my addr 172.21.15.150:0/3665305974 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5b59e700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c106140 0x7f1d5c1a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5a59c700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d5c1057f0 0x7f1d5c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 -- 172.21.15.150:0/3665305974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d5c1057f0 msgr2=0x7f1d5c199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d5c1057f0 0x7f1d5c199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 -- 172.21.15.150:0/3665305974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c106140 msgr2=0x7f1d5c1a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d5c106140 0x7f1d5c1a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 -- 172.21.15.150:0/3665305974 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54009d00 con 0x7f1d5c100ec0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d5ad9d700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c100ec0 0x7f1d5c199690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e9a0 tx=0x7f1d5000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50009790 con 0x7f1d5c100ec0 2024-05-10T04:38:48.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c1a4f60 con 0x7f1d5c100ec0 2024-05-10T04:38:48.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5000cbd0 con 0x7f1d5c100ec0 2024-05-10T04:38:48.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50018680 con 0x7f1d5c100ec0 2024-05-10T04:38:48.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.920+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c1a5370 con 0x7f1d5c100ec0 2024-05-10T04:38:48.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.921+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d5c110e00 con 0x7f1d5c100ec0 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.922+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1d5000cd40 con 0x7f1d5c100ec0 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.923+0000 7f1d43fff700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d440a11b0 0x7f1d440a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.923+0000 7f1d5a59c700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d440a11b0 0x7f1d440a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.923+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d50014070 con 0x7f1d5c100ec0 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.923+0000 7f1d43fff700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] conn(0x7f1d440abd80 0x7f1d440ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.923+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 --> [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d440ae8e0 con 0x7f1d440abd80 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.923+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d500cdd50 con 0x7f1d5c100ec0 2024-05-10T04:38:48.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.923+0000 7f1d5b59e700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] conn(0x7f1d440abd80 0x7f1d440ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:48.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.924+0000 7f1d5a59c700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d440a11b0 0x7f1d440a36c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c069270 tx=0x7f1d4c00d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:48.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.924+0000 7f1d5b59e700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] conn(0x7f1d440abd80 0x7f1d440ae210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:48.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.925+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== osd.3 v2:172.21.15.150:6810/217079495 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1d440ae8e0 con 0x7f1d440abd80 2024-05-10T04:38:48.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.940+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 --> [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d5c061890 con 0x7f1d440abd80 2024-05-10T04:38:48.942 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d43fff700 1 -- 172.21.15.150:0/3665305974 <== osd.3 v2:172.21.15.150:6810/217079495 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1d5c061890 con 0x7f1d440abd80 2024-05-10T04:38:48.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] conn(0x7f1d440abd80 msgr2=0x7f1d440ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:48.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] conn(0x7f1d440abd80 0x7f1d440ae210 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d440a11b0 msgr2=0x7f1d440a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:48.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1d440a11b0 0x7f1d440a36c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c069270 tx=0x7f1d4c00d4b0 comp rx=0 tx=0).stop 2024-05-10T04:38:48.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c100ec0 msgr2=0x7f1d5c199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:48.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 --2- 172.21.15.150:0/3665305974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d5c100ec0 0x7f1d5c199690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e9a0 tx=0x7f1d5000ed60 comp rx=0 tx=0).stop 2024-05-10T04:38:48.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d5b59e700 1 -- 172.21.15.150:0/3665305974 reap_dead start 2024-05-10T04:38:48.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 shutdown_connections 2024-05-10T04:38:48.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.941+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 >> 172.21.15.150:0/3665305974 conn(0x7f1d5c076720 msgr2=0x7f1d5c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:48.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.942+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 shutdown_connections 2024-05-10T04:38:48.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:48.942+0000 7f1d611e5700 1 -- 172.21.15.150:0/3665305974 wait complete. 2024-05-10T04:38:49.213 INFO:teuthology.orchestra.run.smithi150.stdout:77309411377 2024-05-10T04:38:49.214 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-10T04:38:49.226 INFO:teuthology.orchestra.run.smithi150.stdout:214748364831 2024-05-10T04:38:49.226 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-10T04:38:49.491 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:49.639 INFO:teuthology.orchestra.run.smithi150.stdout:111669149741 2024-05-10T04:38:49.639 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-10T04:38:49.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 -- 172.21.15.150:0/2938265656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed81071d0 msgr2=0x7f8ed810eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 --2- 172.21.15.150:0/2938265656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed81071d0 0x7f8ed810eb30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000b0a0 tx=0x7f8ed000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 -- 172.21.15.150:0/2938265656 shutdown_connections 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 --2- 172.21.15.150:0/2938265656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed81071d0 0x7f8ed810eb30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 --2- 172.21.15.150:0/2938265656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed8101f50 0x7f8ed8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 --2- 172.21.15.150:0/2938265656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed8106880 0x7f8ed8106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 -- 172.21.15.150:0/2938265656 >> 172.21.15.150:0/2938265656 conn(0x7f8ed80fd820 msgr2=0x7f8ed80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 -- 172.21.15.150:0/2938265656 shutdown_connections 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.681+0000 7f8edf822700 1 -- 172.21.15.150:0/2938265656 wait complete. 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.682+0000 7f8edf822700 1 Processor -- start 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.682+0000 7f8edf822700 1 -- start start 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edf822700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed8101f50 0x7f8ed819e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edf822700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed8106880 0x7f8ed819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edf822700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed81071d0 0x7f8ed8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edf822700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81989e0 con 0x7f8ed8101f50 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edf822700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8198b50 con 0x7f8ed81071d0 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edf822700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8198d10 con 0x7f8ed8106880 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed8101f50 0x7f8ed819e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed8101f50 0x7f8ed819e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51626/0 (socket says 172.21.15.150:51626) 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 -- 172.21.15.150:0/1303083449 learned_addr learned my addr 172.21.15.150:0/1303083449 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edcdbd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed8106880 0x7f8ed819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edddbf700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed81071d0 0x7f8ed8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 -- 172.21.15.150:0/1303083449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed8106880 msgr2=0x7f8ed819e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ed8106880 0x7f8ed819e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 -- 172.21.15.150:0/1303083449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed81071d0 msgr2=0x7f8ed8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed81071d0 0x7f8ed8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 -- 172.21.15.150:0/1303083449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed0009d00 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edddbf700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed81071d0 0x7f8ed8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.683+0000 7f8edd5be700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed8101f50 0x7f8ed819e4b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400e9a0 tx=0x7f8ed400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.684+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4009790 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.684+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed8198ff0 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.684+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed400cbd0 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.684+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4018680 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.684+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed806ae00 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.685+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ed8103400 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.687+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8ed400cd40 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.688+0000 7f8ece7fc700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ec40a1380 0x7f8ec40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.688+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8ed4014070 con 0x7f8ed8101f50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.688+0000 7f8edcdbd700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ec40a1380 0x7f8ec40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.688+0000 7f8ece7fc700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] conn(0x7f8ec40abf50 0x7f8ec40ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.688+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 --> [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ec40aeab0 con 0x7f8ec40abf50 2024-05-10T04:38:49.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.688+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed40cddc0 con 0x7f8ed8101f50 2024-05-10T04:38:49.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.689+0000 7f8edddbf700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] conn(0x7f8ec40abf50 0x7f8ec40ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.689+0000 7f8edcdbd700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ec40a1380 0x7f8ec40a3890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8199d80 tx=0x7f8ec80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:49.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.689+0000 7f8edddbf700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] conn(0x7f8ec40abf50 0x7f8ec40ae3e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:49.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.690+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== osd.9 v2:172.21.15.150:6826/2987376476 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8ec40aeab0 con 0x7f8ec40abf50 2024-05-10T04:38:49.700 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:49.704 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:49.704 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:49.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.710+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 --> [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ed8061890 con 0x7f8ec40abf50 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.711+0000 7f8ece7fc700 1 -- 172.21.15.150:0/1303083449 <== osd.9 v2:172.21.15.150:6826/2987376476 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8ed8061890 con 0x7f8ec40abf50 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] conn(0x7f8ec40abf50 msgr2=0x7f8ec40ae3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] conn(0x7f8ec40abf50 0x7f8ec40ae3e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ec40a1380 msgr2=0x7f8ec40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ec40a1380 0x7f8ec40a3890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8199d80 tx=0x7f8ec80073d0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed8101f50 msgr2=0x7f8ed819e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 --2- 172.21.15.150:0/1303083449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ed8101f50 0x7f8ed819e4b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400e9a0 tx=0x7f8ed400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edddbf700 1 -- 172.21.15.150:0/1303083449 reap_dead start 2024-05-10T04:38:49.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 shutdown_connections 2024-05-10T04:38:49.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.713+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 >> 172.21.15.150:0/1303083449 conn(0x7f8ed80fd820 msgr2=0x7f8ed81049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:49.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.716+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 shutdown_connections 2024-05-10T04:38:49.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.716+0000 7f8edf822700 1 -- 172.21.15.150:0/1303083449 wait complete. 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.872+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3198961786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 msgr2=0x7f6a0c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.872+0000 7f6a13c2c700 1 --2- 172.21.15.150:0/3198961786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 0x7f6a0c1031d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00b0a0 tx=0x7f69fc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.872+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3198961786 shutdown_connections 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.872+0000 7f6a13c2c700 1 --2- 172.21.15.150:0/3198961786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a0c10c230 0x7f6a0c10e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.872+0000 7f6a13c2c700 1 --2- 172.21.15.150:0/3198961786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 0x7f6a0c1031d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.872+0000 7f6a13c2c700 1 --2- 172.21.15.150:0/3198961786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a0c103710 0x7f6a0c10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.872+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3198961786 >> 172.21.15.150:0/3198961786 conn(0x7f6a0c0fc7a0 msgr2=0x7f6a0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.873+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3198961786 shutdown_connections 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.873+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3198961786 wait complete. 2024-05-10T04:38:49.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.873+0000 7f6a13c2c700 1 Processor -- start 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.873+0000 7f6a13c2c700 1 -- start start 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.873+0000 7f6a13c2c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 0x7f6a0c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.873+0000 7f6a13c2c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a0c103710 0x7f6a0c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.873+0000 7f6a13c2c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a0c10c230 0x7f6a0c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a13c2c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c198a00 con 0x7f6a0c102dc0 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a13c2c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c198b70 con 0x7f6a0c10c230 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a13c2c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c198d30 con 0x7f6a0c103710 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 0x7f6a0c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 0x7f6a0c19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51648/0 (socket says 172.21.15.150:51648) 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 -- 172.21.15.150:0/3674111911 learned_addr learned my addr 172.21.15.150:0/3674111911 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a121c9700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a0c10c230 0x7f6a0c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a111c7700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a0c103710 0x7f6a0c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 -- 172.21.15.150:0/3674111911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a0c103710 msgr2=0x7f6a0c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a0c103710 0x7f6a0c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 -- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a0c10c230 msgr2=0x7f6a0c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a0c10c230 0x7f6a0c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 -- 172.21.15.150:0/3674111911 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc009d00 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a121c9700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a0c10c230 0x7f6a0c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a111c7700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a0c103710 0x7f6a0c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a119c8700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 0x7f6a0c19e560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00b070 tx=0x7f69fc007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc021070 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69fc009440 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00e040 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0c198fb0 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.874+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c1009e0 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.875+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a0c06a9d0 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.877+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f69fc01d480 con 0x7f6a0c102dc0 2024-05-10T04:38:49.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.877+0000 7f6a0affd700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f69f40a1230 0x7f69f40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.877+0000 7f6a111c7700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f69f40a1230 0x7f69f40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.877+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f69fc0ce0c0 con 0x7f6a0c102dc0 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.878+0000 7f6a0affd700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] conn(0x7f69f40abe00 0x7f69f40ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.878+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 --> [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69f40ae960 con 0x7f69f40abe00 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.878+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69fc0ce4d0 con 0x7f6a0c102dc0 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.878+0000 7f6a111c7700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f69f40a1230 0x7f69f40a3740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6a00005d90 tx=0x7f6a00005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.878+0000 7f6a121c9700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] conn(0x7f69f40abe00 0x7f69f40ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:49.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.878+0000 7f6a121c9700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] conn(0x7f69f40abe00 0x7f69f40ae290 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.879+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== osd.1 v2:172.21.15.161:6800/2361791264 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f69f40ae960 con 0x7f69f40abe00 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.895+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 --> [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a0c061890 con 0x7f69f40abe00 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a0affd700 1 -- 172.21.15.150:0/3674111911 <== osd.1 v2:172.21.15.161:6800/2361791264 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a0c061890 con 0x7f69f40abe00 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] conn(0x7f69f40abe00 msgr2=0x7f69f40ae290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] conn(0x7f69f40abe00 0x7f69f40ae290 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f69f40a1230 msgr2=0x7f69f40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f69f40a1230 0x7f69f40a3740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6a00005d90 tx=0x7f6a00005d00 comp rx=0 tx=0).stop 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 msgr2=0x7f6a0c19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 --2- 172.21.15.150:0/3674111911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a0c102dc0 0x7f6a0c19e560 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00b070 tx=0x7f69fc007920 comp rx=0 tx=0).stop 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a121c9700 1 -- 172.21.15.150:0/3674111911 reap_dead start 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 shutdown_connections 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 >> 172.21.15.150:0/3674111911 conn(0x7f6a0c0fc7a0 msgr2=0x7f6a0c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 shutdown_connections 2024-05-10T04:38:49.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:49.896+0000 7f6a13c2c700 1 -- 172.21.15.150:0/3674111911 wait complete. 2024-05-10T04:38:50.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:50 smithi181 ceph-mon[47314]: pgmap v245: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-05-10T04:38:50.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:50 smithi150 ceph-mon[42404]: pgmap v245: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-05-10T04:38:50.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:50 smithi161 ceph-mon[46912]: pgmap v245: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-05-10T04:38:50.853 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 -- 172.21.15.150:0/2930335126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 msgr2=0x7fa914106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 --2- 172.21.15.150:0/2930335126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 0x7fa914106c90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa900009990 tx=0x7fa900009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 -- 172.21.15.150:0/2930335126 shutdown_connections 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 --2- 172.21.15.150:0/2930335126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141071d0 0x7fa91410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 --2- 172.21.15.150:0/2930335126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa914101fa0 0x7fa9141023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 --2- 172.21.15.150:0/2930335126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 0x7fa914106c90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 -- 172.21.15.150:0/2930335126 >> 172.21.15.150:0/2930335126 conn(0x7fa9140778b0 msgr2=0x7fa914077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 -- 172.21.15.150:0/2930335126 shutdown_connections 2024-05-10T04:38:51.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.693+0000 7fa91ac5e700 1 -- 172.21.15.150:0/2930335126 wait complete. 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 Processor -- start 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 -- start start 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa914101fa0 0x7fa9141984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 0x7fa914198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141071d0 0x7fa91419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91419d500 con 0x7fa914106880 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91419d670 con 0x7fa914101fa0 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.694+0000 7fa91ac5e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91419d830 con 0x7fa9141071d0 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 0x7fa914198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 0x7fa914198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51674/0 (socket says 172.21.15.150:51674) 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 -- 172.21.15.150:0/949571708 learned_addr learned my addr 172.21.15.150:0/949571708 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa9189fa700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa914101fa0 0x7fa9141984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa9191fb700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141071d0 0x7fa91419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:51.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 -- 172.21.15.150:0/949571708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141071d0 msgr2=0x7fa91419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141071d0 0x7fa91419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 -- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa914101fa0 msgr2=0x7fa9141984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa914101fa0 0x7fa9141984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 -- 172.21.15.150:0/949571708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9000095f0 con 0x7fa914106880 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa9191fb700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9141071d0 0x7fa91419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa913fff700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 0x7fa914198a80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa9000098e0 tx=0x7fa90000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90001d070 con 0x7fa914106880 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91419dab0 con 0x7fa914106880 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa900022470 con 0x7fa914106880 2024-05-10T04:38:51.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.695+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900017650 con 0x7fa914106880 2024-05-10T04:38:51.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.696+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa914073440 con 0x7fa914106880 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.697+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa914103400 con 0x7fa914106880 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.698+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa90000fae0 con 0x7fa914106880 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.698+0000 7fa911ffb700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9040a11b0 0x7fa9040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.699+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa9000ce080 con 0x7fa914106880 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.699+0000 7fa911ffb700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] conn(0x7fa9040abd80 0x7fa9040ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.699+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 --> [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9040ae8e0 con 0x7fa9040abd80 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.699+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9000ce490 con 0x7fa914106880 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.699+0000 7fa9189fa700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9040a11b0 0x7fa9040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.699+0000 7fa9191fb700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] conn(0x7fa9040abd80 0x7fa9040ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.699+0000 7fa9191fb700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] conn(0x7fa9040abd80 0x7fa9040ae210 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:51.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.700+0000 7fa9189fa700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9040a11b0 0x7fa9040a36c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa908007be0 tx=0x7fa9080073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:51.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.700+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== osd.10 v2:172.21.15.161:6824/2621232559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa9040ae8e0 con 0x7fa9040abd80 2024-05-10T04:38:51.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.717+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 --> [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa914061910 con 0x7fa9040abd80 2024-05-10T04:38:51.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.717+0000 7fa911ffb700 1 -- 172.21.15.150:0/949571708 <== osd.10 v2:172.21.15.161:6824/2621232559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa914061910 con 0x7fa9040abd80 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] conn(0x7fa9040abd80 msgr2=0x7fa9040ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] conn(0x7fa9040abd80 0x7fa9040ae210 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9040a11b0 msgr2=0x7fa9040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9040a11b0 0x7fa9040a36c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa908007be0 tx=0x7fa9080073d0 comp rx=0 tx=0).stop 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 msgr2=0x7fa914198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 --2- 172.21.15.150:0/949571708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa914106880 0x7fa914198a80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa9000098e0 tx=0x7fa90000f670 comp rx=0 tx=0).stop 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa9191fb700 1 -- 172.21.15.150:0/949571708 reap_dead start 2024-05-10T04:38:51.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 shutdown_connections 2024-05-10T04:38:51.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 >> 172.21.15.150:0/949571708 conn(0x7fa9140778b0 msgr2=0x7fa9140ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:51.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 shutdown_connections 2024-05-10T04:38:51.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:51.718+0000 7fa91ac5e700 1 -- 172.21.15.150:0/949571708 wait complete. 2024-05-10T04:38:51.756 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:52.007 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:52.040 INFO:teuthology.orchestra.run.smithi150.stdout:274877906969 2024-05-10T04:38:52.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-10T04:38:52.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 -- 172.21.15.150:0/3137732093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 msgr2=0x7f181c10ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 --2- 172.21.15.150:0/3137732093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 0x7f181c10ead0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f181400b0a0 tx=0x7f181400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 -- 172.21.15.150:0/3137732093 shutdown_connections 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 --2- 172.21.15.150:0/3137732093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 0x7f181c10ead0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 --2- 172.21.15.150:0/3137732093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181c106c60 0x7f181c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 --2- 172.21.15.150:0/3137732093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c1000f0 0x7f181c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 -- 172.21.15.150:0/3137732093 >> 172.21.15.150:0/3137732093 conn(0x7f181c0fbed0 msgr2=0x7f181c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 -- 172.21.15.150:0/3137732093 shutdown_connections 2024-05-10T04:38:52.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.108+0000 7f18233d1700 1 -- 172.21.15.150:0/3137732093 wait complete. 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.109+0000 7f18233d1700 1 Processor -- start 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.109+0000 7f18233d1700 1 -- start start 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.109+0000 7f18233d1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c1000f0 0x7f181c1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.109+0000 7f18233d1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 0x7f181c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.109+0000 7f18233d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181c106c60 0x7f181c101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.109+0000 7f18233d1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c1ab050 con 0x7f181c100a40 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.109+0000 7f18233d1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c1ab190 con 0x7f181c1000f0 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f18233d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c1ab300 con 0x7f181c106c60 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 0x7f181c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 0x7f181c101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51704/0 (socket says 172.21.15.150:51704) 2024-05-10T04:38:52.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 -- 172.21.15.150:0/3777168478 learned_addr learned my addr 172.21.15.150:0/3777168478 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182116d700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c1000f0 0x7f181c1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182196e700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181c106c60 0x7f181c101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 -- 172.21.15.150:0/3777168478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181c106c60 msgr2=0x7f181c101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f181c106c60 0x7f181c101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 -- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c1000f0 msgr2=0x7f181c1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f181c1000f0 0x7f181c1012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 -- 172.21.15.150:0/3777168478 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814009d00 con 0x7f181c100a40 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f182096c700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 0x7f181c101810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f180c00d980 tx=0x7f180c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.110+0000 7f18127fc700 1 -- 172.21.15.150:0/3777168478 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00f890 con 0x7f181c100a40 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.111+0000 7f18127fc700 1 -- 172.21.15.150:0/3777168478 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180c010460 con 0x7f181c100a40 2024-05-10T04:38:52.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.111+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c1ab5e0 con 0x7f181c100a40 2024-05-10T04:38:52.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.111+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c1abb30 con 0x7f181c100a40 2024-05-10T04:38:52.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.111+0000 7f18127fc700 1 -- 172.21.15.150:0/3777168478 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00e610 con 0x7f181c100a40 2024-05-10T04:38:52.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.114+0000 7f18127fc700 1 -- 172.21.15.150:0/3777168478 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f180c00f9f0 con 0x7f181c100a40 2024-05-10T04:38:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.114+0000 7f18127fc700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f18080a1230 0x7f18080a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.115+0000 7f18127fc700 1 -- 172.21.15.150:0/3777168478 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f180c0cc880 con 0x7f181c100a40 2024-05-10T04:38:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.116+0000 7f182116d700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f18080a1230 0x7f18080a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.117+0000 7f182116d700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f18080a1230 0x7f18080a3740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ff8d0 tx=0x7f1818009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.117+0000 7f18233d1700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] conn(0x7f18000015e0 0x7f1800003af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.117+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 --> [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1800006c20 con 0x7f18000015e0 2024-05-10T04:38:52.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.117+0000 7f182196e700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] conn(0x7f18000015e0 0x7f1800003af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.118+0000 7f182196e700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] conn(0x7f18000015e0 0x7f1800003af0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:52.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.118+0000 7f18127fc700 1 -- 172.21.15.150:0/3777168478 <== osd.4 v2:172.21.15.161:6808/402152898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f1800006c20 con 0x7f18000015e0 2024-05-10T04:38:52.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.136+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 --> [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1800005d00 con 0x7f18000015e0 2024-05-10T04:38:52.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.136+0000 7f18127fc700 1 -- 172.21.15.150:0/3777168478 <== osd.4 v2:172.21.15.161:6808/402152898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1800005d00 con 0x7f18000015e0 2024-05-10T04:38:52.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] conn(0x7f18000015e0 msgr2=0x7f1800003af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] conn(0x7f18000015e0 0x7f1800003af0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f18080a1230 msgr2=0x7f18080a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f18080a1230 0x7f18080a3740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f181c0ff8d0 tx=0x7f1818009260 comp rx=0 tx=0).stop 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 msgr2=0x7f181c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 --2- 172.21.15.150:0/3777168478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f181c100a40 0x7f181c101810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f180c00d980 tx=0x7f180c00dc90 comp rx=0 tx=0).stop 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f182196e700 1 -- 172.21.15.150:0/3777168478 reap_dead start 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 shutdown_connections 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 >> 172.21.15.150:0/3777168478 conn(0x7f181c0fbed0 msgr2=0x7f181c1080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 shutdown_connections 2024-05-10T04:38:52.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.137+0000 7f18233d1700 1 -- 172.21.15.150:0/3777168478 wait complete. 2024-05-10T04:38:52.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 -- 172.21.15.150:0/984742014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 msgr2=0x7faed410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 --2- 172.21.15.150:0/984742014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 0x7faed410d9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faecc009f90 tx=0x7faecc00b330 comp rx=0 tx=0).stop 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 -- 172.21.15.150:0/984742014 shutdown_connections 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 --2- 172.21.15.150:0/984742014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 0x7faed410d9e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 --2- 172.21.15.150:0/984742014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed4100e00 0x7faed4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 --2- 172.21.15.150:0/984742014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed4105730 0x7faed4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 -- 172.21.15.150:0/984742014 >> 172.21.15.150:0/984742014 conn(0x7faed40fc7a0 msgr2=0x7faed40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 -- 172.21.15.150:0/984742014 shutdown_connections 2024-05-10T04:38:52.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.170+0000 7faed91ab700 1 -- 172.21.15.150:0/984742014 wait complete. 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.171+0000 7faed91ab700 1 Processor -- start 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.171+0000 7faed91ab700 1 -- start start 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.171+0000 7faed91ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed4100e00 0x7faed419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.171+0000 7faed91ab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed4105730 0x7faed419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.171+0000 7faed91ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 0x7faed4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.171+0000 7faed91ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed41989e0 con 0x7faed4100e00 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed91ab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed4198b50 con 0x7faed4105730 2024-05-10T04:38:52.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed91ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed4198d10 con 0x7faed4106080 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed4100e00 0x7faed419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed4100e00 0x7faed419e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51724/0 (socket says 172.21.15.150:51724) 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 -- 172.21.15.150:0/3126301062 learned_addr learned my addr 172.21.15.150:0/3126301062 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed259c700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed4105730 0x7faed419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed359e700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 0x7faed4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 -- 172.21.15.150:0/3126301062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 msgr2=0x7faed4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 0x7faed4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 -- 172.21.15.150:0/3126301062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed4105730 msgr2=0x7faed419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed4105730 0x7faed419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 -- 172.21.15.150:0/3126301062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc009c40 con 0x7faed4100e00 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed359e700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faed4106080 0x7faed4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faed2d9d700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed4100e00 0x7faed419e4b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faec400e9c0 tx=0x7faec400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:52.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.172+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400cbd0 con 0x7faed4100e00 2024-05-10T04:38:52.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.173+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faec400eeb0 con 0x7faed4100e00 2024-05-10T04:38:52.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.173+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed4198ff0 con 0x7faed4100e00 2024-05-10T04:38:52.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.173+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4005580 con 0x7faed4100e00 2024-05-10T04:38:52.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.173+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed40fcd80 con 0x7faed4100e00 2024-05-10T04:38:52.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.173+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faed4110d40 con 0x7faed4100e00 2024-05-10T04:38:52.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.175+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7faec4010430 con 0x7faed4100e00 2024-05-10T04:38:52.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.175+0000 7faebbfff700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faebc0a11b0 0x7faebc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.175+0000 7faed259c700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faebc0a11b0 0x7faebc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.175+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faec4014070 con 0x7faed4100e00 2024-05-10T04:38:52.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.176+0000 7faebbfff700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] conn(0x7faebc0abd80 0x7faebc0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:52.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.176+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 --> [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faebc0ae8e0 con 0x7faebc0abd80 2024-05-10T04:38:52.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.176+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7faec40cdb00 con 0x7faed4100e00 2024-05-10T04:38:52.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.176+0000 7faed359e700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] conn(0x7faebc0abd80 0x7faebc0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:52.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.176+0000 7faed259c700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faebc0a11b0 0x7faebc0a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faed4199d80 tx=0x7faec8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:52.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.176+0000 7faed359e700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] conn(0x7faebc0abd80 0x7faebc0ae210 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:52.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.177+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== osd.0 v2:172.21.15.150:6802/4183993127 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7faebc0ae8e0 con 0x7faebc0abd80 2024-05-10T04:38:52.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.209+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 --> [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faed4061890 con 0x7faebc0abd80 2024-05-10T04:38:52.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.209+0000 7faebbfff700 1 -- 172.21.15.150:0/3126301062 <== osd.0 v2:172.21.15.150:6802/4183993127 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faed4061890 con 0x7faebc0abd80 2024-05-10T04:38:52.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.209+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] conn(0x7faebc0abd80 msgr2=0x7faebc0ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.209+0000 7faed91ab700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] conn(0x7faebc0abd80 0x7faebc0ae210 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faebc0a11b0 msgr2=0x7faebc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7faebc0a11b0 0x7faebc0a36c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faed4199d80 tx=0x7faec8009260 comp rx=0 tx=0).stop 2024-05-10T04:38:52.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed4100e00 msgr2=0x7faed419e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:52.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 --2- 172.21.15.150:0/3126301062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faed4100e00 0x7faed419e4b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faec400e9c0 tx=0x7faec400ecd0 comp rx=0 tx=0).stop 2024-05-10T04:38:52.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed359e700 1 -- 172.21.15.150:0/3126301062 reap_dead start 2024-05-10T04:38:52.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 shutdown_connections 2024-05-10T04:38:52.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 >> 172.21.15.150:0/3126301062 conn(0x7faed40fc7a0 msgr2=0x7faed4069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:52.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 shutdown_connections 2024-05-10T04:38:52.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:52.210+0000 7faed91ab700 1 -- 172.21.15.150:0/3126301062 wait complete. 2024-05-10T04:38:52.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:52 smithi181 ceph-mon[47314]: pgmap v246: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-10T04:38:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:52 smithi150 ceph-mon[42404]: pgmap v246: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-10T04:38:52.727 INFO:teuthology.orchestra.run.smithi150.stdout:34359738425 2024-05-10T04:38:52.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-10T04:38:52.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:52 smithi161 ceph-mon[46912]: pgmap v246: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-10T04:38:53.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 -- 172.21.15.150:0/1097356985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248100e30 msgr2=0x7fc248101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 --2- 172.21.15.150:0/1097356985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248100e30 0x7fc248101240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc244009990 tx=0x7fc244009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 -- 172.21.15.150:0/1097356985 shutdown_connections 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 --2- 172.21.15.150:0/1097356985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2481060b0 0x7fc248073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 --2- 172.21.15.150:0/1097356985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248100e30 0x7fc248101240 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 --2- 172.21.15.150:0/1097356985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc248105760 0x7fc248105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 -- 172.21.15.150:0/1097356985 >> 172.21.15.150:0/1097356985 conn(0x7fc2480fc760 msgr2=0x7fc2480feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 -- 172.21.15.150:0/1097356985 shutdown_connections 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.115+0000 7fc25015c700 1 -- 172.21.15.150:0/1097356985 wait complete. 2024-05-10T04:38:53.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 Processor -- start 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 -- start start 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248100e30 0x7fc24810fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248105760 0x7fc248110230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2481060b0 0x7fc248110770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2481aadd0 con 0x7fc248105760 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24819a4d0 con 0x7fc248100e30 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.116+0000 7fc25015c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24819a690 con 0x7fc2481060b0 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248105760 0x7fc248110230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248105760 0x7fc248110230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51746/0 (socket says 172.21.15.150:51746) 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 -- 172.21.15.150:0/1364555647 learned_addr learned my addr 172.21.15.150:0/1364555647 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24def8700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248100e30 0x7fc24810fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24e6f9700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2481060b0 0x7fc248110770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 -- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2481060b0 msgr2=0x7fc248110770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2481060b0 0x7fc248110770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 -- 172.21.15.150:0/1364555647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248100e30 msgr2=0x7fc24810fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248100e30 0x7fc24810fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 -- 172.21.15.150:0/1364555647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2440095f0 con 0x7fc248105760 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24def8700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc248100e30 0x7fc24810fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc24d6f7700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248105760 0x7fc248110230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e970 tx=0x7fc23800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238009790 con 0x7fc248105760 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc238004cf0 con 0x7fc248105760 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24819a970 con 0x7fc248105760 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.117+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238005680 con 0x7fc248105760 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.118+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24819ae70 con 0x7fc248105760 2024-05-10T04:38:53.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.118+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc24810b9c0 con 0x7fc248105760 2024-05-10T04:38:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.120+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc238004830 con 0x7fc248105760 2024-05-10T04:38:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc23effd700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2340a11b0 0x7fc2340a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc24def8700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2340a11b0 0x7fc2340a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc238014070 con 0x7fc248105760 2024-05-10T04:38:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc23effd700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] conn(0x7fc2340abd80 0x7fc2340ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 --> [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2340ae8e0 con 0x7fc2340abd80 2024-05-10T04:38:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2380cd7f0 con 0x7fc248105760 2024-05-10T04:38:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc24def8700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2340a11b0 0x7fc2340a36c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc244006010 tx=0x7fc24401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.121+0000 7fc24e6f9700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] conn(0x7fc2340abd80 0x7fc2340ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.122+0000 7fc24e6f9700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] conn(0x7fc2340abd80 0x7fc2340ae210 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:53.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.123+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== osd.8 v2:172.21.15.181:6816/646590427 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc2340ae8e0 con 0x7fc2340abd80 2024-05-10T04:38:53.140 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.140+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 --> [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc248104c50 con 0x7fc2340abd80 2024-05-10T04:38:53.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.140+0000 7fc23effd700 1 -- 172.21.15.150:0/1364555647 <== osd.8 v2:172.21.15.181:6816/646590427 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc248104c50 con 0x7fc2340abd80 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] conn(0x7fc2340abd80 msgr2=0x7fc2340ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] conn(0x7fc2340abd80 0x7fc2340ae210 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2340a11b0 msgr2=0x7fc2340a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2340a11b0 0x7fc2340a36c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc244006010 tx=0x7fc24401a040 comp rx=0 tx=0).stop 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248105760 msgr2=0x7fc248110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 --2- 172.21.15.150:0/1364555647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc248105760 0x7fc248110230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e970 tx=0x7fc23800ed30 comp rx=0 tx=0).stop 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc24e6f9700 1 -- 172.21.15.150:0/1364555647 reap_dead start 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 shutdown_connections 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 >> 172.21.15.150:0/1364555647 conn(0x7fc2480fc760 msgr2=0x7fc248104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 shutdown_connections 2024-05-10T04:38:53.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.141+0000 7fc25015c700 1 -- 172.21.15.150:0/1364555647 wait complete. 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 -- 172.21.15.150:0/3985315439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854100e00 msgr2=0x7f4854101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 --2- 172.21.15.150:0/3985315439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854100e00 0x7f4854101210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4844009990 tx=0x7f4844009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 -- 172.21.15.150:0/3985315439 shutdown_connections 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 --2- 172.21.15.150:0/3985315439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4854106080 0x7f485410d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 --2- 172.21.15.150:0/3985315439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854100e00 0x7f4854101210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 --2- 172.21.15.150:0/3985315439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4854105730 0x7f4854105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 -- 172.21.15.150:0/3985315439 >> 172.21.15.150:0/3985315439 conn(0x7f48540fc7a0 msgr2=0x7f48540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 -- 172.21.15.150:0/3985315439 shutdown_connections 2024-05-10T04:38:53.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.203+0000 7f4858b94700 1 -- 172.21.15.150:0/3985315439 wait complete. 2024-05-10T04:38:53.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.204+0000 7f4858b94700 1 Processor -- start 2024-05-10T04:38:53.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.204+0000 7f4858b94700 1 -- start start 2024-05-10T04:38:53.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4858b94700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4854100e00 0x7f4854197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4858b94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4854105730 0x7f4854197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4858b94700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854106080 0x7f485419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4858b94700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485419c430 con 0x7f4854106080 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4858b94700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485419c5a0 con 0x7f4854100e00 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4858b94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485419c760 con 0x7f4854105730 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854106080 0x7f485419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854106080 0x7f485419bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51760/0 (socket says 172.21.15.150:51760) 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 -- 172.21.15.150:0/2292695043 learned_addr learned my addr 172.21.15.150:0/2292695043 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4851d9b700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4854105730 0x7f4854197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f485259c700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4854100e00 0x7f4854197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 -- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4854105730 msgr2=0x7f4854197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4854105730 0x7f4854197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 -- 172.21.15.150:0/2292695043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4854100e00 msgr2=0x7f4854197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4854100e00 0x7f4854197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4852d9d700 1 -- 172.21.15.150:0/2292695043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440095f0 con 0x7f4854106080 2024-05-10T04:38:53.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.205+0000 7f4851d9b700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4854105730 0x7f4854197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:38:53.207 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.206+0000 7f4852d9d700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854106080 0x7f485419bec0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f484c00ef30 tx=0x7f484c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.206+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c018070 con 0x7f4854106080 2024-05-10T04:38:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.206+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485419c9e0 con 0x7f4854106080 2024-05-10T04:38:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.206+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484c00f040 con 0x7f4854106080 2024-05-10T04:38:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.206+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00cd00 con 0x7f4854106080 2024-05-10T04:38:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.206+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48541a4170 con 0x7f4854106080 2024-05-10T04:38:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.207+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4854110cb0 con 0x7f4854106080 2024-05-10T04:38:53.208 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.208+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f484c004000 con 0x7f4854106080 2024-05-10T04:38:53.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.209+0000 7f48437fe700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f483c0a11b0 0x7f483c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.209+0000 7f485259c700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f483c0a11b0 0x7f483c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.209+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f484c0cd490 con 0x7f4854106080 2024-05-10T04:38:53.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.209+0000 7f48437fe700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] conn(0x7f483c0abd80 0x7f483c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:53.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.209+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 --> [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f483c0ae8e0 con 0x7f483c0abd80 2024-05-10T04:38:53.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.209+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f484c0cd8a0 con 0x7f4854106080 2024-05-10T04:38:53.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.209+0000 7f485259c700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f483c0a11b0 0x7f483c0a36c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48440098e0 tx=0x7f484401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:53.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.210+0000 7f4851d9b700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] conn(0x7f483c0abd80 0x7f483c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:53.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.210+0000 7f4851d9b700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] conn(0x7f483c0abd80 0x7f483c0ae210 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:53.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.211+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== osd.11 v2:172.21.15.181:6824/2057565898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f483c0ae8e0 con 0x7f483c0abd80 2024-05-10T04:38:53.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.227+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 --> [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4854068fe0 con 0x7f483c0abd80 2024-05-10T04:38:53.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f48437fe700 1 -- 172.21.15.150:0/2292695043 <== osd.11 v2:172.21.15.181:6824/2057565898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4854068fe0 con 0x7f483c0abd80 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] conn(0x7f483c0abd80 msgr2=0x7f483c0ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] conn(0x7f483c0abd80 0x7f483c0ae210 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f483c0a11b0 msgr2=0x7f483c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f483c0a11b0 0x7f483c0a36c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48440098e0 tx=0x7f484401a040 comp rx=0 tx=0).stop 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854106080 msgr2=0x7f485419bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 --2- 172.21.15.150:0/2292695043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4854106080 0x7f485419bec0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f484c00ef30 tx=0x7f484c00c3a0 comp rx=0 tx=0).stop 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4852d9d700 1 -- 172.21.15.150:0/2292695043 reap_dead start 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 shutdown_connections 2024-05-10T04:38:53.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.228+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 >> 172.21.15.150:0/2292695043 conn(0x7f48540fc7a0 msgr2=0x7f48540ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:53.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.229+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 shutdown_connections 2024-05-10T04:38:53.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:53.229+0000 7f4858b94700 1 -- 172.21.15.150:0/2292695043 wait complete. 2024-05-10T04:38:54.090 INFO:teuthology.orchestra.run.smithi150.stdout:55834574901 2024-05-10T04:38:54.090 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-10T04:38:54.217 INFO:teuthology.orchestra.run.smithi150.stdout:296352743445 2024-05-10T04:38:54.218 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-10T04:38:54.396 INFO:teuthology.orchestra.run.smithi150.stdout:141733920811 2024-05-10T04:38:54.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-10T04:38:54.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.519+0000 7f975e3b9700 1 -- 172.21.15.150:0/729480898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f975810c210 msgr2=0x7f975810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:54.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.519+0000 7f975e3b9700 1 --2- 172.21.15.150:0/729480898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f975810c210 0x7f975810e720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f975000b0a0 tx=0x7f975000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:38:54.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.519+0000 7f975e3b9700 1 -- 172.21.15.150:0/729480898 shutdown_connections 2024-05-10T04:38:54.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.519+0000 7f975e3b9700 1 --2- 172.21.15.150:0/729480898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f975810c210 0x7f975810e720 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:54.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.519+0000 7f975e3b9700 1 --2- 172.21.15.150:0/729480898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9758102da0 0x7f97581031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:54.522 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.519+0000 7f975e3b9700 1 --2- 172.21.15.150:0/729480898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97581036f0 0x7f975810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.519+0000 7f975e3b9700 1 -- 172.21.15.150:0/729480898 >> 172.21.15.150:0/729480898 conn(0x7f97580fc7a0 msgr2=0x7f97580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.520+0000 7f975e3b9700 1 -- 172.21.15.150:0/729480898 shutdown_connections 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.520+0000 7f975e3b9700 1 -- 172.21.15.150:0/729480898 wait complete. 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.520+0000 7f975e3b9700 1 Processor -- start 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.520+0000 7f975e3b9700 1 -- start start 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f975e3b9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9758102da0 0x7f975819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f975e3b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97581036f0 0x7f975819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f975e3b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f975810c210 0x7f9758198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f975e3b9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758198a00 con 0x7f97581036f0 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f975e3b9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758198b70 con 0x7f9758102da0 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f975e3b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758198d30 con 0x7f975810c210 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9758102da0 0x7f975819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f97577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97581036f0 0x7f975819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:54.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9758102da0 0x7f975819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:40980/0 (socket says 172.21.15.150:40980) 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 -- 172.21.15.150:0/981832812 learned_addr learned my addr 172.21.15.150:0/981832812 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f975c956700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f975810c210 0x7f9758198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 -- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f975810c210 msgr2=0x7f9758198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f975810c210 0x7f9758198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 -- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97581036f0 msgr2=0x7f975819ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97581036f0 0x7f975819ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f9757fff700 1 -- 172.21.15.150:0/981832812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750009d00 con 0x7f9758102da0 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.521+0000 7f97577fe700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97581036f0 0x7f975819ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.522+0000 7f9757fff700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9758102da0 0x7f975819e4d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f974800d980 tx=0x7f974800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.522+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748004020 con 0x7f9758102da0 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.522+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9758199010 con 0x7f9758102da0 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.522+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f974800f690 con 0x7f9758102da0 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.522+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748003a60 con 0x7f9758102da0 2024-05-10T04:38:54.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.522+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97581a54a0 con 0x7f9758102da0 2024-05-10T04:38:54.525 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.522+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9758069600 con 0x7f9758102da0 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.524+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9748003bc0 con 0x7f9758102da0 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.524+0000 7f97557fa700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f97400a11b0 0x7f97400a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.525+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f97480cd000 con 0x7f9758102da0 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.525+0000 7f97577fe700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f97400a11b0 0x7f97400a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.525+0000 7f97557fa700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] conn(0x7f97400abd80 0x7f97400ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.525+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 --> [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97400ae8e0 con 0x7f97400abd80 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.525+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97480cd410 con 0x7f9758102da0 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.525+0000 7f97577fe700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f97400a11b0 0x7f97400a36c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9758199da0 tx=0x7f974c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.525+0000 7f975c956700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] conn(0x7f97400abd80 0x7f97400ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:54.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.526+0000 7f975c956700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] conn(0x7f97400abd80 0x7f97400ae210 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:54.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.527+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== osd.5 v2:172.21.15.181:6808/189565731 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f97400ae8e0 con 0x7f97400abd80 2024-05-10T04:38:54.543 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.543+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 --> [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9758061890 con 0x7f97400abd80 2024-05-10T04:38:54.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.544+0000 7f97557fa700 1 -- 172.21.15.150:0/981832812 <== osd.5 v2:172.21.15.181:6808/189565731 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9758061890 con 0x7f97400abd80 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.544+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] conn(0x7f97400abd80 msgr2=0x7f97400ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.544+0000 7f975e3b9700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] conn(0x7f97400abd80 0x7f97400ae210 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.544+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f97400a11b0 msgr2=0x7f97400a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.544+0000 7f975e3b9700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f97400a11b0 0x7f97400a36c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9758199da0 tx=0x7f974c00a300 comp rx=0 tx=0).stop 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.544+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9758102da0 msgr2=0x7f975819e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.544+0000 7f975e3b9700 1 --2- 172.21.15.150:0/981832812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9758102da0 0x7f975819e4d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f974800d980 tx=0x7f974800dc90 comp rx=0 tx=0).stop 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.545+0000 7f975c956700 1 -- 172.21.15.150:0/981832812 reap_dead start 2024-05-10T04:38:54.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.545+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 shutdown_connections 2024-05-10T04:38:54.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.545+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 >> 172.21.15.150:0/981832812 conn(0x7f97580fc7a0 msgr2=0x7f97580fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:54.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.545+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 shutdown_connections 2024-05-10T04:38:54.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:54.545+0000 7f975e3b9700 1 -- 172.21.15.150:0/981832812 wait complete. 2024-05-10T04:38:54.580 INFO:teuthology.orchestra.run.smithi150.stdout:244813135901 2024-05-10T04:38:54.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-10T04:38:54.651 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:54 smithi150 ceph-mon[42404]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-10T04:38:54.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:54 smithi181 ceph-mon[47314]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-10T04:38:54.746 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:54 smithi161 ceph-mon[46912]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-05-10T04:38:55.362 INFO:teuthology.orchestra.run.smithi150.stdout:163208757288 2024-05-10T04:38:55.363 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-10T04:38:55.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 -- 172.21.15.150:0/1062060780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc100e00 msgr2=0x7ff0cc101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 --2- 172.21.15.150:0/1062060780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc100e00 0x7ff0cc101210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009990 tx=0x7ff0bc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 -- 172.21.15.150:0/1062060780 shutdown_connections 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 --2- 172.21.15.150:0/1062060780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc106080 0x7ff0cc10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 --2- 172.21.15.150:0/1062060780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc100e00 0x7ff0cc101210 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 --2- 172.21.15.150:0/1062060780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc105730 0x7ff0cc105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 -- 172.21.15.150:0/1062060780 >> 172.21.15.150:0/1062060780 conn(0x7ff0cc0fc7a0 msgr2=0x7ff0cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 -- 172.21.15.150:0/1062060780 shutdown_connections 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.593+0000 7ff0d249c700 1 -- 172.21.15.150:0/1062060780 wait complete. 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.594+0000 7ff0d249c700 1 Processor -- start 2024-05-10T04:38:55.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.594+0000 7ff0d249c700 1 -- start start 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d249c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc100e00 0x7ff0cc1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d249c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc105730 0x7ff0cc197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d249c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc106080 0x7ff0cc19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d249c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc19c3a0 con 0x7ff0cc106080 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d249c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc19c510 con 0x7ff0cc105730 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d249c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc19c6d0 con 0x7ff0cc100e00 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d0a39700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc106080 0x7ff0cc19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d0a39700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc106080 0x7ff0cc19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51800/0 (socket says 172.21.15.150:51800) 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0cbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc100e00 0x7ff0cc1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d0a39700 1 -- 172.21.15.150:0/2280972099 learned_addr learned my addr 172.21.15.150:0/2280972099 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0cb7fe700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc105730 0x7ff0cc197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:55.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d0a39700 1 -- 172.21.15.150:0/2280972099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc100e00 msgr2=0x7ff0cc1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d0a39700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc100e00 0x7ff0cc1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d0a39700 1 -- 172.21.15.150:0/2280972099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc105730 msgr2=0x7ff0cc197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.595+0000 7ff0d0a39700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc105730 0x7ff0cc197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.596+0000 7ff0d0a39700 1 -- 172.21.15.150:0/2280972099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0095f0 con 0x7ff0cc106080 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.596+0000 7ff0d0a39700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc106080 0x7ff0cc19be60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4007d70 tx=0x7ff0c400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.596+0000 7ff0c97fa700 1 -- 172.21.15.150:0/2280972099 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400ac90 con 0x7ff0cc106080 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.596+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc19c9b0 con 0x7ff0cc106080 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.596+0000 7ff0c97fa700 1 -- 172.21.15.150:0/2280972099 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c400adf0 con 0x7ff0cc106080 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.596+0000 7ff0c97fa700 1 -- 172.21.15.150:0/2280972099 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40115e0 con 0x7ff0cc106080 2024-05-10T04:38:55.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.596+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc19cce0 con 0x7ff0cc106080 2024-05-10T04:38:55.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.597+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc1090d0 con 0x7ff0cc106080 2024-05-10T04:38:55.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.598+0000 7ff0c97fa700 1 -- 172.21.15.150:0/2280972099 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff0c4012420 con 0x7ff0cc106080 2024-05-10T04:38:55.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.599+0000 7ff0c97fa700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0b40a1230 0x7ff0b40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:55.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.599+0000 7ff0cbfff700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0b40a1230 0x7ff0b40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:55.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.599+0000 7ff0c97fa700 1 -- 172.21.15.150:0/2280972099 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff0c40ccc10 con 0x7ff0cc106080 2024-05-10T04:38:55.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.600+0000 7ff0cbfff700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0b40a1230 0x7ff0b40a3740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0098e0 tx=0x7ff0bc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:55.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.604+0000 7ff0c97fa700 1 -- 172.21.15.150:0/2280972099 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff0c4096160 con 0x7ff0cc106080 2024-05-10T04:38:55.698 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:55.721 INFO:teuthology.orchestra.run.smithi150.stdout:330712481811 2024-05-10T04:38:55.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-10T04:38:55.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.846+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0cc1022b0 con 0x7ff0cc106080 2024-05-10T04:38:55.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.847+0000 7ff0c97fa700 1 -- 172.21.15.150:0/2280972099 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0c4099780 con 0x7ff0cc106080 2024-05-10T04:38:55.847 INFO:teuthology.orchestra.run.smithi150.stdout:77309411379 2024-05-10T04:38:55.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0b40a1230 msgr2=0x7ff0b40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0b40a1230 0x7ff0b40a3740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0098e0 tx=0x7ff0bc005960 comp rx=0 tx=0).stop 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc106080 msgr2=0x7ff0cc19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc106080 0x7ff0cc19be60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4007d70 tx=0x7ff0c400a340 comp rx=0 tx=0).stop 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 shutdown_connections 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc106080 0x7ff0cc19be60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0b40a1230 0x7ff0b40a3740 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0098e0 tx=0x7ff0bc005960 comp rx=0 tx=0).stop 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0cc100e00 0x7ff0cc1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 --2- 172.21.15.150:0/2280972099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0cc105730 0x7ff0cc197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:55.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.850+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 >> 172.21.15.150:0/2280972099 conn(0x7ff0cc0fc7a0 msgr2=0x7ff0cc106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:55.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.852+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 shutdown_connections 2024-05-10T04:38:55.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:55.852+0000 7ff0d249c700 1 -- 172.21.15.150:0/2280972099 wait complete. 2024-05-10T04:38:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:55.975 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:56.578 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:56 smithi150 ceph-mon[42404]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-10T04:38:56.578 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:56 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2280972099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 -- 172.21.15.150:0/345087874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 msgr2=0x7f9478105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 --2- 172.21.15.150:0/345087874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 0x7f9478105b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9460009990 tx=0x7f9460009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 -- 172.21.15.150:0/345087874 shutdown_connections 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 --2- 172.21.15.150:0/345087874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94781060a0 0x7f947810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 --2- 172.21.15.150:0/345087874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9478100e20 0x7f9478101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 --2- 172.21.15.150:0/345087874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 0x7f9478105b60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 -- 172.21.15.150:0/345087874 >> 172.21.15.150:0/345087874 conn(0x7f94780fc7a0 msgr2=0x7f94780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 -- 172.21.15.150:0/345087874 shutdown_connections 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.587+0000 7f947cb45700 1 -- 172.21.15.150:0/345087874 wait complete. 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 Processor -- start 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 -- start start 2024-05-10T04:38:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478100e20 0x7f94781994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 0x7f9478199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94781060a0 0x7f94781a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947819a1b0 con 0x7f9478105750 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a48d0 con 0x7f9478100e20 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947cb45700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a4a90 con 0x7f94781060a0 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 0x7f9478199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 0x7f9478199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51822/0 (socket says 172.21.15.150:51822) 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 -- 172.21.15.150:0/3922811132 learned_addr learned my addr 172.21.15.150:0/3922811132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f947659c700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478100e20 0x7f94781994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9476d9d700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94781060a0 0x7f94781a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 -- 172.21.15.150:0/3922811132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94781060a0 msgr2=0x7f94781a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94781060a0 0x7f94781a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 -- 172.21.15.150:0/3922811132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478100e20 msgr2=0x7f94781994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9478100e20 0x7f94781994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 -- 172.21.15.150:0/3922811132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600095f0 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9476d9d700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94781060a0 0x7f94781a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.588+0000 7f9475d9b700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 0x7f9478199a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f94600098e0 tx=0x7f946000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.589+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946001d070 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.589+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94781a4d10 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.589+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9460022470 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.589+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460017650 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.589+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94781a5180 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.589+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94781022d0 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.591+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f94600225e0 con 0x7f9478105750 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.591+0000 7f946f7fe700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94640a11b0 0x7f94640a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f947659c700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94640a11b0 0x7f94640a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f94600ce590 con 0x7f9478105750 2024-05-10T04:38:56.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f946f7fe700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] conn(0x7f94640abd80 0x7f94640ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 --> [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94640ae8e0 con 0x7f94640abd80 2024-05-10T04:38:56.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94600ce9a0 con 0x7f9478105750 2024-05-10T04:38:56.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f9476d9d700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] conn(0x7f94640abd80 0x7f94640ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f947659c700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94640a11b0 0x7f94640a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9468005950 tx=0x7f946800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:56.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.592+0000 7f9476d9d700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] conn(0x7f94640abd80 0x7f94640ae210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:56.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.593+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== osd.6 v2:172.21.15.150:6818/3315977759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f94640ae8e0 con 0x7f94640abd80 2024-05-10T04:38:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.609+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 --> [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9478061950 con 0x7f94640abd80 2024-05-10T04:38:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.609+0000 7f946f7fe700 1 -- 172.21.15.150:0/3922811132 <== osd.6 v2:172.21.15.150:6818/3315977759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9478061950 con 0x7f94640abd80 2024-05-10T04:38:56.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] conn(0x7f94640abd80 msgr2=0x7f94640ae210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] conn(0x7f94640abd80 0x7f94640ae210 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94640a11b0 msgr2=0x7f94640a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94640a11b0 0x7f94640a36c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9468005950 tx=0x7f946800b410 comp rx=0 tx=0).stop 2024-05-10T04:38:56.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 msgr2=0x7f9478199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 --2- 172.21.15.150:0/3922811132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9478105750 0x7f9478199a20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f94600098e0 tx=0x7f946000f670 comp rx=0 tx=0).stop 2024-05-10T04:38:56.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f9476d9d700 1 -- 172.21.15.150:0/3922811132 reap_dead start 2024-05-10T04:38:56.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 shutdown_connections 2024-05-10T04:38:56.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 >> 172.21.15.150:0/3922811132 conn(0x7f94780fc7a0 msgr2=0x7f9478107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:56.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 shutdown_connections 2024-05-10T04:38:56.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.610+0000 7f947cb45700 1 -- 172.21.15.150:0/3922811132 wait complete. 2024-05-10T04:38:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:56 smithi181 ceph-mon[47314]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-10T04:38:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:56 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2280972099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T04:38:56.687 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:56.753 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/2356163630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec105730 msgr2=0x7fe6ec105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/2356163630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec105730 0x7fe6ec105b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009990 tx=0x7fe6d4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/2356163630 shutdown_connections 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/2356163630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6ec106080 0x7fe6ec10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/2356163630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6ec100e00 0x7fe6ec101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/2356163630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec105730 0x7fe6ec105b40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/2356163630 >> 172.21.15.150:0/2356163630 conn(0x7fe6ec0fc7a0 msgr2=0x7fe6ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/2356163630 shutdown_connections 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/2356163630 wait complete. 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.753+0000 7fe6f1c4a700 1 Processor -- start 2024-05-10T04:38:56.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6f1c4a700 1 -- start start 2024-05-10T04:38:56.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6f1c4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6ec100e00 0x7fe6ec1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6f1c4a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6ec105730 0x7fe6ec196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6f1c4a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec106080 0x7fe6ec19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6f1c4a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec19b2c0 con 0x7fe6ec106080 2024-05-10T04:38:56.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6f1c4a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec19b430 con 0x7fe6ec105730 2024-05-10T04:38:56.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6f1c4a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec19b5f0 con 0x7fe6ec100e00 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec106080 0x7fe6ec19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec106080 0x7fe6ec19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:51836/0 (socket says 172.21.15.150:51836) 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 -- 172.21.15.150:0/3175506659 learned_addr learned my addr 172.21.15.150:0/3175506659 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6eb7fe700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6ec100e00 0x7fe6ec1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6eaffd700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6ec105730 0x7fe6ec196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 -- 172.21.15.150:0/3175506659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6ec100e00 msgr2=0x7fe6ec1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6ec100e00 0x7fe6ec1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 -- 172.21.15.150:0/3175506659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6ec105730 msgr2=0x7fe6ec196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6ec105730 0x7fe6ec196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.754+0000 7fe6ebfff700 1 -- 172.21.15.150:0/3175506659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d40095f0 con 0x7fe6ec106080 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.755+0000 7fe6ebfff700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec106080 0x7fe6ec19ad50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009a20 tx=0x7fe6e4009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.755+0000 7fe6e8ff9700 1 -- 172.21.15.150:0/3175506659 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4012070 con 0x7fe6ec106080 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.755+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ec19b8d0 con 0x7fe6ec106080 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.755+0000 7fe6e8ff9700 1 -- 172.21.15.150:0/3175506659 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e400d670 con 0x7fe6ec106080 2024-05-10T04:38:56.757 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.755+0000 7fe6e8ff9700 1 -- 172.21.15.150:0/3175506659 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4005280 con 0x7fe6ec106080 2024-05-10T04:38:56.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.755+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec19bbb0 con 0x7fe6ec106080 2024-05-10T04:38:56.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.756+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ec109040 con 0x7fe6ec106080 2024-05-10T04:38:56.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.757+0000 7fe6e8ff9700 1 -- 172.21.15.150:0/3175506659 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe6e40053e0 con 0x7fe6ec106080 2024-05-10T04:38:56.758 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.757+0000 7fe6e8ff9700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6d80a11b0 0x7fe6d80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:56.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.758+0000 7fe6eb7fe700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6d80a11b0 0x7fe6d80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:56.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.758+0000 7fe6e8ff9700 1 -- 172.21.15.150:0/3175506659 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe6e4099630 con 0x7fe6ec106080 2024-05-10T04:38:56.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.758+0000 7fe6eb7fe700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6d80a11b0 0x7fe6d80a36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e7e0 tx=0x7fe6dc011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:56.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.762+0000 7fe6e8ff9700 1 -- 172.21.15.150:0/3175506659 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6e4095270 con 0x7fe6ec106080 2024-05-10T04:38:56.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:56 smithi161 ceph-mon[46912]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-10T04:38:56.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:56 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2280972099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-10T04:38:56.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.972+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe6ec1978b0 con 0x7fe6ec106080 2024-05-10T04:38:56.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.972+0000 7fe6e8ff9700 1 -- 172.21.15.150:0/3175506659 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6e4004e80 con 0x7fe6ec106080 2024-05-10T04:38:56.973 INFO:teuthology.orchestra.run.smithi150.stdout:214748364833 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6d80a11b0 msgr2=0x7fe6d80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6d80a11b0 0x7fe6d80a36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e7e0 tx=0x7fe6dc011040 comp rx=0 tx=0).stop 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec106080 msgr2=0x7fe6ec19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec106080 0x7fe6ec19ad50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009a20 tx=0x7fe6e4009d30 comp rx=0 tx=0).stop 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 shutdown_connections 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6ec106080 0x7fe6ec19ad50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6d80a11b0 0x7fe6d80a36c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6ec100e00 0x7fe6ec1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 --2- 172.21.15.150:0/3175506659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6ec105730 0x7fe6ec196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:56.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 >> 172.21.15.150:0/3175506659 conn(0x7fe6ec0fc7a0 msgr2=0x7fe6ec0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:56.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 shutdown_connections 2024-05-10T04:38:56.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:56.975+0000 7fe6f1c4a700 1 -- 172.21.15.150:0/3175506659 wait complete. 2024-05-10T04:38:57.571 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:57 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3175506659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T04:38:57.590 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364831 got 214748364833 for osd.7 2024-05-10T04:38:57.591 DEBUG:teuthology.parallel:result is None 2024-05-10T04:38:57.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:57 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3175506659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T04:38:57.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:57 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3175506659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-10T04:38:57.880 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:57.907 INFO:teuthology.orchestra.run.smithi150.stdout:193273528356 2024-05-10T04:38:57.907 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-10T04:38:57.982 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:58.448 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:58.520 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:58 smithi150 ceph-mon[42404]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-10T04:38:58.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:58 smithi181 ceph-mon[47314]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-10T04:38:58.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 -- 172.21.15.150:0/290409650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa04105780 msgr2=0x7faa04105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 --2- 172.21.15.150:0/290409650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa04105780 0x7faa04105b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80098f0 tx=0x7fa9f8009c00 comp rx=0 tx=0).stop 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 -- 172.21.15.150:0/290409650 shutdown_connections 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 --2- 172.21.15.150:0/290409650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa041060d0 0x7faa0410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 --2- 172.21.15.150:0/290409650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa04100e50 0x7faa04101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 --2- 172.21.15.150:0/290409650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa04105780 0x7faa04105b90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 -- 172.21.15.150:0/290409650 >> 172.21.15.150:0/290409650 conn(0x7faa040fc7a0 msgr2=0x7faa040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 -- 172.21.15.150:0/290409650 shutdown_connections 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.727+0000 7faa0a8af700 1 -- 172.21.15.150:0/290409650 wait complete. 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.728+0000 7faa0a8af700 1 Processor -- start 2024-05-10T04:38:58.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.728+0000 7faa0a8af700 1 -- start start 2024-05-10T04:38:58.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.728+0000 7faa0a8af700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa04100e50 0x7faa04198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.728+0000 7faa0a8af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa04105780 0x7faa04198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa0a8af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa041060d0 0x7faa0419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa0a8af700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419d530 con 0x7faa041060d0 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa0a8af700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419d6a0 con 0x7faa04100e50 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa0a8af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419d860 con 0x7faa04105780 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa041060d0 0x7faa0419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa041060d0 0x7faa0419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48470/0 (socket says 172.21.15.150:48470) 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 -- 172.21.15.150:0/717104154 learned_addr learned my addr 172.21.15.150:0/717104154 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa03fff700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa04100e50 0x7faa04198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa037fe700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa04105780 0x7faa04198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 -- 172.21.15.150:0/717104154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa04105780 msgr2=0x7faa04198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa04105780 0x7faa04198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 -- 172.21.15.150:0/717104154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa04100e50 msgr2=0x7faa04198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa04100e50 0x7faa04198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 -- 172.21.15.150:0/717104154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40095f0 con 0x7faa041060d0 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa03fff700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa04100e50 0x7faa04198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.729+0000 7faa08e4c700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa041060d0 0x7faa0419cfc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00ef30 tx=0x7fa9fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.730+0000 7faa017fa700 1 -- 172.21.15.150:0/717104154 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc018070 con 0x7faa041060d0 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.730+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f8009520 con 0x7faa041060d0 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.730+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0419de20 con 0x7faa041060d0 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.730+0000 7faa017fa700 1 -- 172.21.15.150:0/717104154 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9fc00f040 con 0x7faa041060d0 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.730+0000 7faa017fa700 1 -- 172.21.15.150:0/717104154 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00cd00 con 0x7faa041060d0 2024-05-10T04:38:58.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.731+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f00052f0 con 0x7faa041060d0 2024-05-10T04:38:58.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.732+0000 7faa017fa700 1 -- 172.21.15.150:0/717104154 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa9fc004ad0 con 0x7faa041060d0 2024-05-10T04:38:58.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.733+0000 7faa017fa700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9ec0a1230 0x7fa9ec0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.733+0000 7faa03fff700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9ec0a1230 0x7fa9ec0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:58.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.733+0000 7faa017fa700 1 -- 172.21.15.150:0/717104154 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa9fc0ccf80 con 0x7faa041060d0 2024-05-10T04:38:58.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.734+0000 7faa03fff700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9ec0a1230 0x7fa9ec0a3740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005fd0 tx=0x7fa9f4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:58.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.737+0000 7faa017fa700 1 -- 172.21.15.150:0/717104154 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa9fc096450 con 0x7faa041060d0 2024-05-10T04:38:58.779 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:58.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:58 smithi161 ceph-mon[46912]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-10T04:38:58.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 -- 172.21.15.150:0/1165491272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c102da0 msgr2=0x7f119c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:58.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/1165491272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c102da0 0x7f119c1031b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f11980098f0 tx=0x7f1198009c00 comp rx=0 tx=0).stop 2024-05-10T04:38:58.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 -- 172.21.15.150:0/1165491272 shutdown_connections 2024-05-10T04:38:58.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/1165491272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c10c210 0x7f119c10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/1165491272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c102da0 0x7f119c1031b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/1165491272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f119c1036f0 0x7f119c10bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 -- 172.21.15.150:0/1165491272 >> 172.21.15.150:0/1165491272 conn(0x7f119c0fc7a0 msgr2=0x7f119c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:58.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 -- 172.21.15.150:0/1165491272 shutdown_connections 2024-05-10T04:38:58.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.846+0000 7f11a3dd4700 1 -- 172.21.15.150:0/1165491272 wait complete. 2024-05-10T04:38:58.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.847+0000 7f11a3dd4700 1 Processor -- start 2024-05-10T04:38:58.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a3dd4700 1 -- start start 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a3dd4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f119c102da0 0x7f119c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a3dd4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c1036f0 0x7f119c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a3dd4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c10c210 0x7f119c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a3dd4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19d580 con 0x7f119c1036f0 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a3dd4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19d6f0 con 0x7f119c102da0 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a3dd4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19d8b0 con 0x7f119c10c210 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a2371700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c10c210 0x7f119c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a2371700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c10c210 0x7f119c19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:42398/0 (socket says 172.21.15.150:42398) 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a2371700 1 -- 172.21.15.150:0/584523411 learned_addr learned my addr 172.21.15.150:0/584523411 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.848+0000 7f11a136f700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c1036f0 0x7f119c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a2371700 1 -- 172.21.15.150:0/584523411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f119c102da0 msgr2=0x7f119c198560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a2371700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f119c102da0 0x7f119c198560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a2371700 1 -- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c1036f0 msgr2=0x7f119c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:58.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a2371700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c1036f0 0x7f119c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a2371700 1 -- 172.21.15.150:0/584523411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c009520 con 0x7f119c10c210 2024-05-10T04:38:58.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a2371700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c10c210 0x7f119c19d010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f119400e3c0 tx=0x7f119400e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:58.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f1192ffd700 1 -- 172.21.15.150:0/584523411 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11940092d0 con 0x7f119c10c210 2024-05-10T04:38:58.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f1192ffd700 1 -- 172.21.15.150:0/584523411 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119400f040 con 0x7f119c10c210 2024-05-10T04:38:58.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f1192ffd700 1 -- 172.21.15.150:0/584523411 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194007cc0 con 0x7f119c10c210 2024-05-10T04:38:58.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1194009d00 con 0x7f119c10c210 2024-05-10T04:38:58.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.849+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1198009520 con 0x7f119c10c210 2024-05-10T04:38:58.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.850+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c1072f0 con 0x7f119c10c210 2024-05-10T04:38:58.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.852+0000 7f1192ffd700 1 -- 172.21.15.150:0/584523411 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1194004ad0 con 0x7f119c10c210 2024-05-10T04:38:58.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.853+0000 7f1192ffd700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11880a1230 0x7f11880a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:58.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.853+0000 7f1192ffd700 1 -- 172.21.15.150:0/584523411 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f11940cca00 con 0x7f119c10c210 2024-05-10T04:38:58.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.856+0000 7f1192ffd700 1 -- 172.21.15.150:0/584523411 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1194095ed0 con 0x7f119c10c210 2024-05-10T04:38:58.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.862+0000 7f11a1b70700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11880a1230 0x7f11880a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:58.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.863+0000 7f11a1b70700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11880a1230 0x7f11880a3740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1198005f40 tx=0x7f1198009f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:58.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.975+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa9f0005160 con 0x7faa041060d0 2024-05-10T04:38:58.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.976+0000 7faa017fa700 1 -- 172.21.15.150:0/717104154 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa9fc09ee60 con 0x7faa041060d0 2024-05-10T04:38:58.976 INFO:teuthology.orchestra.run.smithi150.stdout:34359738426 2024-05-10T04:38:58.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9ec0a1230 msgr2=0x7fa9ec0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:58.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9ec0a1230 0x7fa9ec0a3740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4005fd0 tx=0x7fa9f4009310 comp rx=0 tx=0).stop 2024-05-10T04:38:58.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa041060d0 msgr2=0x7faa0419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:58.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa041060d0 0x7faa0419cfc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00ef30 tx=0x7fa9fc00c3a0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 shutdown_connections 2024-05-10T04:38:58.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa041060d0 0x7faa0419cfc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa04100e50 0x7faa04198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa9ec0a1230 0x7fa9ec0a3740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 --2- 172.21.15.150:0/717104154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa04105780 0x7faa04198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 >> 172.21.15.150:0/717104154 conn(0x7faa040fc7a0 msgr2=0x7faa04069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 shutdown_connections 2024-05-10T04:38:58.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:58.978+0000 7faa0a8af700 1 -- 172.21.15.150:0/717104154 wait complete. 2024-05-10T04:38:59.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.090+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f119c199b70 con 0x7f119c10c210 2024-05-10T04:38:59.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.093+0000 7f1192ffd700 1 -- 172.21.15.150:0/584523411 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11940994f0 con 0x7f119c10c210 2024-05-10T04:38:59.093 INFO:teuthology.orchestra.run.smithi150.stdout:111669149743 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11880a1230 msgr2=0x7f11880a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11880a1230 0x7f11880a3740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1198005f40 tx=0x7f1198009f70 comp rx=0 tx=0).stop 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c10c210 msgr2=0x7f119c19d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c10c210 0x7f119c19d010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f119400e3c0 tx=0x7f119400e6d0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 shutdown_connections 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11880a1230 0x7f11880a3740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c10c210 0x7f119c19d010 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f119c102da0 0x7f119c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 --2- 172.21.15.150:0/584523411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f119c1036f0 0x7f119c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.095+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 >> 172.21.15.150:0/584523411 conn(0x7f119c0fc7a0 msgr2=0x7f119c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.096+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 shutdown_connections 2024-05-10T04:38:59.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.096+0000 7f11a3dd4700 1 -- 172.21.15.150:0/584523411 wait complete. 2024-05-10T04:38:59.149 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411377 got 77309411379 for osd.2 2024-05-10T04:38:59.149 DEBUG:teuthology.parallel:result is None 2024-05-10T04:38:59.294 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:59.391 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:59 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/717104154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T04:38:59.391 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:38:59 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/584523411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T04:38:59.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:59 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/717104154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T04:38:59.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:38:59 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/584523411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T04:38:59.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 -- 172.21.15.150:0/28068044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 msgr2=0x7f27d4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 --2- 172.21.15.150:0/28068044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d4101270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009990 tx=0x7f27c4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 -- 172.21.15.150:0/28068044 shutdown_connections 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 --2- 172.21.15.150:0/28068044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41060e0 0x7f27d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 --2- 172.21.15.150:0/28068044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d4101270 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 --2- 172.21.15.150:0/28068044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d4105790 0x7f27d4105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 -- 172.21.15.150:0/28068044 >> 172.21.15.150:0/28068044 conn(0x7f27d40fc760 msgr2=0x7f27d40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 -- 172.21.15.150:0/28068044 shutdown_connections 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.444+0000 7f27d8e85700 1 -- 172.21.15.150:0/28068044 wait complete. 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 Processor -- start 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 -- start start 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d4105790 0x7f27d41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41060e0 0x7f27d41a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:59.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d41a48a0 con 0x7f27d4100e60 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d41a49e0 con 0x7f27d4105790 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d8e85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d41a4b20 con 0x7f27d41060e0 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d41994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48502/0 (socket says 172.21.15.150:48502) 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 -- 172.21.15.150:0/4024279447 learned_addr learned my addr 172.21.15.150:0/4024279447 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d2d9d700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41060e0 0x7f27d41a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d1d9b700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d4105790 0x7f27d41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:59.727 INFO:teuthology.orchestra.run.smithi150.stdout:274877906971 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 -- 172.21.15.150:0/4024279447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41060e0 msgr2=0x7f27d41a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41060e0 0x7f27d41a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 -- 172.21.15.150:0/4024279447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d4105790 msgr2=0x7f27d41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d4105790 0x7f27d41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d259c700 1 -- 172.21.15.150:0/4024279447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40095f0 con 0x7f27d4100e60 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.445+0000 7f27d2d9d700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41060e0 0x7f27d41a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.446+0000 7f27d259c700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d41994b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27c4004730 tx=0x7f27c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.446+0000 7f27c37fe700 1 -- 172.21.15.150:0/4024279447 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c401d070 con 0x7f27d4100e60 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.446+0000 7f27c37fe700 1 -- 172.21.15.150:0/4024279447 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c4022470 con 0x7f27d4100e60 2024-05-10T04:38:59.728 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.446+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d41a4d40 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.446+0000 7f27c37fe700 1 -- 172.21.15.150:0/4024279447 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4017650 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.446+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d41a51b0 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.447+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d4109130 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.448+0000 7f27c37fe700 1 -- 172.21.15.150:0/4024279447 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f27c40225e0 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.449+0000 7f27c37fe700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27bc0a12b0 0x7f27bc0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.449+0000 7f27d1d9b700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27bc0a12b0 0x7f27bc0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.449+0000 7f27c37fe700 1 -- 172.21.15.150:0/4024279447 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f27c40ce430 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.449+0000 7f27d1d9b700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27bc0a12b0 0x7f27bc0a37c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27d4073eb0 tx=0x7f27c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.452+0000 7f27c37fe700 1 -- 172.21.15.150:0/4024279447 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f27c40978a0 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.661+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f27d404edc0 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.661+0000 7f27c37fe700 1 -- 172.21.15.150:0/4024279447 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f27c4097030 con 0x7f27d4100e60 2024-05-10T04:38:59.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27bc0a12b0 msgr2=0x7f27bc0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27bc0a12b0 0x7f27bc0a37c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27d4073eb0 tx=0x7f27c8009260 comp rx=0 tx=0).stop 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 msgr2=0x7f27d41994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d41994b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f27c4004730 tx=0x7f27c4005e80 comp rx=0 tx=0).stop 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 shutdown_connections 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27bc0a12b0 0x7f27bc0a37c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d41060e0 0x7f27d41a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27d4100e60 0x7f27d41994b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 --2- 172.21.15.150:0/4024279447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27d4105790 0x7f27d41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.663+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 >> 172.21.15.150:0/4024279447 conn(0x7f27d40fc760 msgr2=0x7f27d4103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.664+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 shutdown_connections 2024-05-10T04:38:59.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.664+0000 7f27d8e85700 1 -- 172.21.15.150:0/4024279447 wait complete. 2024-05-10T04:38:59.731 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:38:59.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:59 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/717104154' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-10T04:38:59.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:38:59 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/584523411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-10T04:38:59.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738425 got 34359738426 for osd.0 2024-05-10T04:38:59.895 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:00.654 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:00 smithi150 ceph-mon[42404]: pgmap v250: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:39:00.654 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:00 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4024279447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T04:39:00.654 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:00 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/557801361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T04:39:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:00 smithi181 ceph-mon[47314]: pgmap v250: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:39:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:00 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4024279447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T04:39:00.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:00 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/557801361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T04:39:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:00 smithi161 ceph-mon[46912]: pgmap v250: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:39:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:00 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4024279447' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-10T04:39:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:00 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/557801361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-10T04:39:00.923 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149741 got 111669149743 for osd.3 2024-05-10T04:39:00.924 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:00.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.930+0000 7f3920c1c700 1 -- 172.21.15.150:0/2321540713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 msgr2=0x7f391c10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:00.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.930+0000 7f3920c1c700 1 --2- 172.21.15.150:0/2321540713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c10d9e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f391400b0a0 tx=0x7f391400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:00.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.930+0000 7f3920c1c700 1 -- 172.21.15.150:0/2321540713 shutdown_connections 2024-05-10T04:39:00.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.930+0000 7f3920c1c700 1 --2- 172.21.15.150:0/2321540713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c10d9e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:00.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.930+0000 7f3920c1c700 1 --2- 172.21.15.150:0/2321540713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391c100e00 0x7f391c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:00.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.930+0000 7f3920c1c700 1 --2- 172.21.15.150:0/2321540713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c105730 0x7f391c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:00.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.930+0000 7f3920c1c700 1 -- 172.21.15.150:0/2321540713 >> 172.21.15.150:0/2321540713 conn(0x7f391c0fc7a0 msgr2=0x7f391c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:00.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.931+0000 7f3920c1c700 1 -- 172.21.15.150:0/2321540713 shutdown_connections 2024-05-10T04:39:00.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.931+0000 7f3920c1c700 1 -- 172.21.15.150:0/2321540713 wait complete. 2024-05-10T04:39:00.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 Processor -- start 2024-05-10T04:39:00.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 -- start start 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c100e00 0x7f391c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391c105730 0x7f391c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c198a00 con 0x7f391c106080 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c198b70 con 0x7f391c100e00 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3920c1c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c198d30 con 0x7f391c105730 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48532/0 (socket says 172.21.15.150:48532) 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391ad9d700 1 -- 172.21.15.150:0/1888065351 learned_addr learned my addr 172.21.15.150:0/1888065351 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391a59c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c100e00 0x7f391c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f3919d9b700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391c105730 0x7f391c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:00.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391ad9d700 1 -- 172.21.15.150:0/1888065351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391c105730 msgr2=0x7f391c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391ad9d700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391c105730 0x7f391c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391ad9d700 1 -- 172.21.15.150:0/1888065351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c100e00 msgr2=0x7f391c19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.932+0000 7f391ad9d700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c100e00 0x7f391c19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.933+0000 7f391ad9d700 1 -- 172.21.15.150:0/1888065351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914009d00 con 0x7f391c106080 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.933+0000 7f391ad9d700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c198430 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f391400b070 tx=0x7f3914007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.933+0000 7f390b7fe700 1 -- 172.21.15.150:0/1888065351 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391400e040 con 0x7f391c106080 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.933+0000 7f390b7fe700 1 -- 172.21.15.150:0/1888065351 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3914009440 con 0x7f391c106080 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.933+0000 7f390b7fe700 1 -- 172.21.15.150:0/1888065351 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391401b450 con 0x7f391c106080 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.933+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c198fb0 con 0x7f391c106080 2024-05-10T04:39:00.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.933+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c199290 con 0x7f391c106080 2024-05-10T04:39:00.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.934+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c10a140 con 0x7f391c106080 2024-05-10T04:39:00.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.935+0000 7f390b7fe700 1 -- 172.21.15.150:0/1888065351 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f391401d480 con 0x7f391c106080 2024-05-10T04:39:00.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.936+0000 7f390b7fe700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f39040a11b0 0x7f39040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:00.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.936+0000 7f391a59c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f39040a11b0 0x7f39040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:00.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.937+0000 7f390b7fe700 1 -- 172.21.15.150:0/1888065351 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f39140cec50 con 0x7f391c106080 2024-05-10T04:39:00.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.937+0000 7f391a59c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f39040a11b0 0x7f39040a36c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f391c1998a0 tx=0x7f390c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:00.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:00.940+0000 7f390b7fe700 1 -- 172.21.15.150:0/1888065351 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f39140980c0 con 0x7f391c106080 2024-05-10T04:39:01.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.154+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f391c1022b0 con 0x7f391c106080 2024-05-10T04:39:01.155 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.155+0000 7f390b7fe700 1 -- 172.21.15.150:0/1888065351 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f391409b6e0 con 0x7f391c106080 2024-05-10T04:39:01.155 INFO:teuthology.orchestra.run.smithi150.stdout:244813135903 2024-05-10T04:39:01.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f39040a11b0 msgr2=0x7f39040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f39040a11b0 0x7f39040a36c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f391c1998a0 tx=0x7f390c00b410 comp rx=0 tx=0).stop 2024-05-10T04:39:01.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 msgr2=0x7f391c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.157 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c198430 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f391400b070 tx=0x7f3914007870 comp rx=0 tx=0).stop 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 shutdown_connections 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f391c106080 0x7f391c198430 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c100e00 0x7f391c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f39040a11b0 0x7f39040a36c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 --2- 172.21.15.150:0/1888065351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391c105730 0x7f391c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 >> 172.21.15.150:0/1888065351 conn(0x7f391c0fc7a0 msgr2=0x7f391c0690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 shutdown_connections 2024-05-10T04:39:01.158 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.157+0000 7f3920c1c700 1 -- 172.21.15.150:0/1888065351 wait complete. 2024-05-10T04:39:01.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 -- 172.21.15.150:0/3446830958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 msgr2=0x7f1034101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 --2- 172.21.15.150:0/3446830958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034101210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1030009990 tx=0x7f1030009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 -- 172.21.15.150:0/3446830958 shutdown_connections 2024-05-10T04:39:01.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 --2- 172.21.15.150:0/3446830958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034106080 0x7f103410d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 --2- 172.21.15.150:0/3446830958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034101210 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 --2- 172.21.15.150:0/3446830958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1034105730 0x7f1034105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 -- 172.21.15.150:0/3446830958 >> 172.21.15.150:0/3446830958 conn(0x7f10340fc7a0 msgr2=0x7f10340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 -- 172.21.15.150:0/3446830958 shutdown_connections 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.672+0000 7f103bc34700 1 -- 172.21.15.150:0/3446830958 wait complete. 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 Processor -- start 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 -- start start 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1034105730 0x7f1034199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034106080 0x7f10341a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103419a1b0 con 0x7f1034100e00 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341a48f0 con 0x7f1034106080 2024-05-10T04:39:01.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f103bc34700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341a4ab0 con 0x7f1034105730 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f10399d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f10399d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48520/0 (socket says 172.21.15.150:48520) 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.673+0000 7f10399d0700 1 -- 172.21.15.150:0/557801361 learned_addr learned my addr 172.21.15.150:0/557801361 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f10391cf700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1034105730 0x7f1034199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f103a1d1700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034106080 0x7f10341a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f10399d0700 1 -- 172.21.15.150:0/557801361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1034105730 msgr2=0x7f1034199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f10399d0700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1034105730 0x7f1034199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f10399d0700 1 -- 172.21.15.150:0/557801361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034106080 msgr2=0x7f10341a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f10399d0700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034106080 0x7f10341a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f10399d0700 1 -- 172.21.15.150:0/557801361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300095f0 con 0x7f1034100e00 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f103a1d1700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034106080 0x7f10341a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f10399d0700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034199500 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1030000c00 tx=0x7f1030005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f102affd700 1 -- 172.21.15.150:0/557801361 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103001d070 con 0x7f1034100e00 2024-05-10T04:39:01.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f102affd700 1 -- 172.21.15.150:0/557801361 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1030022470 con 0x7f1034100e00 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10341a4d30 con 0x7f1034100e00 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f102affd700 1 -- 172.21.15.150:0/557801361 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030017650 con 0x7f1034100e00 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.674+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10341a51a0 con 0x7f1034100e00 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.675+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10341090d0 con 0x7f1034100e00 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.677+0000 7f102affd700 1 -- 172.21.15.150:0/557801361 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f10300225e0 con 0x7f1034100e00 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.677+0000 7f102affd700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10200a1230 0x7f10200a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.678+0000 7f102affd700 1 -- 172.21.15.150:0/557801361 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f10300ce4a0 con 0x7f1034100e00 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.678+0000 7f10391cf700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10200a1230 0x7f10200a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.678+0000 7f10391cf700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10200a1230 0x7f10200a3740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1024005950 tx=0x7f10240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:01.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.682+0000 7f102affd700 1 -- 172.21.15.150:0/557801361 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1030097970 con 0x7f1034100e00 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stdout:55834574903 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.903+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f103404edc0 con 0x7f1034100e00 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.903+0000 7f102affd700 1 -- 172.21.15.150:0/557801361 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1030027020 con 0x7f1034100e00 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10200a1230 msgr2=0x7f10200a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10200a1230 0x7f10200a3740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1024005950 tx=0x7f10240058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 msgr2=0x7f1034199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034199500 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1030000c00 tx=0x7f1030005ef0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 shutdown_connections 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1034106080 0x7f10341a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1034100e00 0x7f1034199500 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10200a1230 0x7f10200a3740 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 --2- 172.21.15.150:0/557801361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1034105730 0x7f1034199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 >> 172.21.15.150:0/557801361 conn(0x7f10340fc7a0 msgr2=0x7f1034106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:01.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 shutdown_connections 2024-05-10T04:39:01.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:38:59.906+0000 7f103bc34700 1 -- 172.21.15.150:0/557801361 wait complete. 2024-05-10T04:39:01.468 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:01 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1888065351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 -- 172.21.15.150:0/3959363418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 msgr2=0x7f9ba410e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 --2- 172.21.15.150:0/3959363418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba410e720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00b0a0 tx=0x7f9b9c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 -- 172.21.15.150:0/3959363418 shutdown_connections 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 --2- 172.21.15.150:0/3959363418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba410e720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 --2- 172.21.15.150:0/3959363418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ba4102da0 0x7f9ba41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 --2- 172.21.15.150:0/3959363418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41036f0 0x7f9ba410bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 -- 172.21.15.150:0/3959363418 >> 172.21.15.150:0/3959363418 conn(0x7f9ba40fc7a0 msgr2=0x7f9ba40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 -- 172.21.15.150:0/3959363418 shutdown_connections 2024-05-10T04:39:01.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.488+0000 7f9ba903b700 1 -- 172.21.15.150:0/3959363418 wait complete. 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.489+0000 7f9ba903b700 1 Processor -- start 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.489+0000 7f9ba903b700 1 -- start start 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba903b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ba4102da0 0x7f9ba41a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba903b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41036f0 0x7f9ba41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba903b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba903b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419ac50 con 0x7f9ba410c210 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba903b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419adc0 con 0x7f9ba41036f0 2024-05-10T04:39:01.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba903b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419af80 con 0x7f9ba4102da0 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48540/0 (socket says 172.21.15.150:48540) 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 -- 172.21.15.150:0/167190526 learned_addr learned my addr 172.21.15.150:0/167190526 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba2d9d700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ba4102da0 0x7f9ba41a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba259c700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41036f0 0x7f9ba41a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 -- 172.21.15.150:0/167190526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ba4102da0 msgr2=0x7f9ba41a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ba4102da0 0x7f9ba41a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 -- 172.21.15.150:0/167190526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41036f0 msgr2=0x7f9ba41a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41036f0 0x7f9ba41a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.490+0000 7f9ba359e700 1 -- 172.21.15.150:0/167190526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c009d00 con 0x7f9ba410c210 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.491+0000 7f9ba359e700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba419a680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0077f0 tx=0x7f9b9c007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.491+0000 7f9b8bfff700 1 -- 172.21.15.150:0/167190526 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00e040 con 0x7f9ba410c210 2024-05-10T04:39:01.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.491+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba419b200 con 0x7f9ba410c210 2024-05-10T04:39:01.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.491+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba41a76f0 con 0x7f9ba410c210 2024-05-10T04:39:01.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.491+0000 7f9b8bfff700 1 -- 172.21.15.150:0/167190526 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9c004c60 con 0x7f9ba410c210 2024-05-10T04:39:01.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.491+0000 7f9b8bfff700 1 -- 172.21.15.150:0/167190526 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c0052a0 con 0x7f9ba410c210 2024-05-10T04:39:01.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.492+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b900052f0 con 0x7f9ba410c210 2024-05-10T04:39:01.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.494+0000 7f9b8bfff700 1 -- 172.21.15.150:0/167190526 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9b9c01b450 con 0x7f9ba410c210 2024-05-10T04:39:01.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.494+0000 7f9b8bfff700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b8c0a12b0 0x7f9b8c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.494+0000 7f9b8bfff700 1 -- 172.21.15.150:0/167190526 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9b9c0ce2e0 con 0x7f9ba410c210 2024-05-10T04:39:01.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.494+0000 7f9ba2d9d700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b8c0a12b0 0x7f9b8c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.495+0000 7f9ba2d9d700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b8c0a12b0 0x7f9b8c0a37c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ba419baf0 tx=0x7f9b9400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:01.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.498+0000 7f9b8bfff700 1 -- 172.21.15.150:0/167190526 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b9c097750 con 0x7f9ba410c210 2024-05-10T04:39:01.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:01 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1888065351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T04:39:01.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.704+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f9b90005160 con 0x7f9ba410c210 2024-05-10T04:39:01.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.704+0000 7f9b8bfff700 1 -- 172.21.15.150:0/167190526 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9b9c024e90 con 0x7f9ba410c210 2024-05-10T04:39:01.704 INFO:teuthology.orchestra.run.smithi150.stdout:296352743447 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b8c0a12b0 msgr2=0x7f9b8c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b8c0a12b0 0x7f9b8c0a37c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9ba419baf0 tx=0x7f9b9400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 msgr2=0x7f9ba419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba419a680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0077f0 tx=0x7f9b9c007820 comp rx=0 tx=0).stop 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 shutdown_connections 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ba410c210 0x7f9ba419a680 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b8c0a12b0 0x7f9b8c0a37c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ba4102da0 0x7f9ba41a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 --2- 172.21.15.150:0/167190526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ba41036f0 0x7f9ba41a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.707 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.706+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 >> 172.21.15.150:0/167190526 conn(0x7f9ba40fc7a0 msgr2=0x7f9ba40fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:01.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.707+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 shutdown_connections 2024-05-10T04:39:01.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.707+0000 7f9ba903b700 1 -- 172.21.15.150:0/167190526 wait complete. 2024-05-10T04:39:01.734 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2540726162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 msgr2=0x7fd1b4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2540726162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b4101210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00b0a0 tx=0x7fd1ac00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2540726162 shutdown_connections 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2540726162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4106080 0x7fd1b410d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2540726162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b4101210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2540726162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1b4105730 0x7fd1b4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2540726162 >> 172.21.15.150:0/2540726162 conn(0x7fd1b40fc7a0 msgr2=0x7fd1b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2540726162 shutdown_connections 2024-05-10T04:39:01.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.734+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2540726162 wait complete. 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 Processor -- start 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 -- start start 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b41995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4105730 0x7fd1b4199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1b4106080 0x7fd1b41a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b41a49b0 con 0x7fd1b4100e00 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b41a4af0 con 0x7fd1b4106080 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b94c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b41a4c30 con 0x7fd1b4105730 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.735+0000 7fd1b2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b41995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b41995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48562/0 (socket says 172.21.15.150:48562) 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 -- 172.21.15.150:0/2057553329 learned_addr learned my addr 172.21.15.150:0/2057553329 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b27fc700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4105730 0x7fd1b4199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b37fe700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1b4106080 0x7fd1b41a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 -- 172.21.15.150:0/2057553329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4105730 msgr2=0x7fd1b4199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4105730 0x7fd1b4199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 -- 172.21.15.150:0/2057553329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1b4106080 msgr2=0x7fd1b41a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1b4106080 0x7fd1b41a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 -- 172.21.15.150:0/2057553329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac009d00 con 0x7fd1b4100e00 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b37fe700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1b4106080 0x7fd1b41a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b2ffd700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b41995c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00b780 tx=0x7fd1ac007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1abfff700 1 -- 172.21.15.150:0/2057553329 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00e040 con 0x7fd1b4100e00 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b41a4e30 con 0x7fd1b4100e00 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1abfff700 1 -- 172.21.15.150:0/2057553329 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1ac01d470 con 0x7fd1b4100e00 2024-05-10T04:39:01.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1abfff700 1 -- 172.21.15.150:0/2057553329 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0052a0 con 0x7fd1b4100e00 2024-05-10T04:39:01.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.736+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b41a5240 con 0x7fd1b4100e00 2024-05-10T04:39:01.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.737+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b41090d0 con 0x7fd1b4100e00 2024-05-10T04:39:01.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.738+0000 7fd1abfff700 1 -- 172.21.15.150:0/2057553329 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd1ac01d5e0 con 0x7fd1b4100e00 2024-05-10T04:39:01.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.739+0000 7fd1abfff700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1a00a11b0 0x7fd1a00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:01.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.739+0000 7fd1b27fc700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1a00a11b0 0x7fd1a00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:01.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.739+0000 7fd1abfff700 1 -- 172.21.15.150:0/2057553329 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd1ac0cdd30 con 0x7fd1b4100e00 2024-05-10T04:39:01.740 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.740+0000 7fd1b27fc700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1a00a11b0 0x7fd1a00a36c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1b41002b0 tx=0x7fd19c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:01.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.744+0000 7fd1abfff700 1 -- 172.21.15.150:0/2057553329 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1ac097200 con 0x7fd1b4100e00 2024-05-10T04:39:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:01 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1888065351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-10T04:39:01.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906969 got 274877906971 for osd.9 2024-05-10T04:39:01.891 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:01.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.950+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd1b41022b0 con 0x7fd1b4100e00 2024-05-10T04:39:01.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.950+0000 7fd1abfff700 1 -- 172.21.15.150:0/2057553329 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1b41022b0 con 0x7fd1b4100e00 2024-05-10T04:39:01.951 INFO:teuthology.orchestra.run.smithi150.stdout:141733920813 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1a00a11b0 msgr2=0x7fd1a00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1a00a11b0 0x7fd1a00a36c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd1b41002b0 tx=0x7fd19c009260 comp rx=0 tx=0).stop 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 msgr2=0x7fd1b41995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b41995c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00b780 tx=0x7fd1ac007940 comp rx=0 tx=0).stop 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 shutdown_connections 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1b4106080 0x7fd1b41a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b4100e00 0x7fd1b41995c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd1a00a11b0 0x7fd1a00a36c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 --2- 172.21.15.150:0/2057553329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1b4105730 0x7fd1b4199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:01.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.952+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 >> 172.21.15.150:0/2057553329 conn(0x7fd1b40fc7a0 msgr2=0x7fd1b4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:01.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.953+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 shutdown_connections 2024-05-10T04:39:01.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:01.953+0000 7fd1b94c8700 1 -- 172.21.15.150:0/2057553329 wait complete. 2024-05-10T04:39:01.991 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135901 got 244813135903 for osd.8 2024-05-10T04:39:01.992 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:02.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574901 got 55834574903 for osd.1 2024-05-10T04:39:02.147 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:02.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920811 got 141733920813 for osd.4 2024-05-10T04:39:02.525 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:02.575 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:02 smithi150 ceph-mon[42404]: pgmap v251: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:02.576 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:02 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/167190526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T04:39:02.576 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:02 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2057553329' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T04:39:02.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:02 smithi181 ceph-mon[47314]: pgmap v251: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:02.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:02 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/167190526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T04:39:02.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:02 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2057553329' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T04:39:02.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:02 smithi161 ceph-mon[46912]: pgmap v251: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:02.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:02 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/167190526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-10T04:39:02.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:02 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2057553329' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-10T04:39:02.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743447 for osd.10 2024-05-10T04:39:02.884 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 -- 172.21.15.150:0/3643466646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101340 msgr2=0x7ff73c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 --2- 172.21.15.150:0/3643466646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101340 0x7ff73c101750 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009990 tx=0x7ff72c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 -- 172.21.15.150:0/3643466646 shutdown_connections 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 --2- 172.21.15.150:0/3643466646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c101c90 0x7ff73c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 --2- 172.21.15.150:0/3643466646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73c106c30 0x7ff73c107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 --2- 172.21.15.150:0/3643466646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101340 0x7ff73c101750 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 -- 172.21.15.150:0/3643466646 >> 172.21.15.150:0/3643466646 conn(0x7ff73c0fbed0 msgr2=0x7ff73c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 -- 172.21.15.150:0/3643466646 shutdown_connections 2024-05-10T04:39:03.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.129+0000 7ff7410ce700 1 -- 172.21.15.150:0/3643466646 wait complete. 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.130+0000 7ff7410ce700 1 Processor -- start 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.130+0000 7ff7410ce700 1 -- start start 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff7410ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c101340 0x7ff73c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff7410ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101c90 0x7ff73c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff7410ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73c106c30 0x7ff73c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff7410ce700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c19d520 con 0x7ff73c101c90 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff7410ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c19d690 con 0x7ff73c106c30 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff7410ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c19d850 con 0x7ff73c101340 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101c90 0x7ff73c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101c90 0x7ff73c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48590/0 (socket says 172.21.15.150:48590) 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 -- 172.21.15.150:0/4102160072 learned_addr learned my addr 172.21.15.150:0/4102160072 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73ad9d700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c101340 0x7ff73c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73b59e700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73c106c30 0x7ff73c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 -- 172.21.15.150:0/4102160072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c101340 msgr2=0x7ff73c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c101340 0x7ff73c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 -- 172.21.15.150:0/4102160072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73c106c30 msgr2=0x7ff73c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73c106c30 0x7ff73c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 -- 172.21.15.150:0/4102160072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0095f0 con 0x7ff73c101c90 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73b59e700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73c106c30 0x7ff73c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.131+0000 7ff73a59c700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101c90 0x7ff73c198aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0098e0 tx=0x7ff72c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.132+0000 7ff733fff700 1 -- 172.21.15.150:0/4102160072 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c01d070 con 0x7ff73c101c90 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.132+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c19dad0 con 0x7ff73c101c90 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.132+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c1a5260 con 0x7ff73c101c90 2024-05-10T04:39:03.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.132+0000 7ff733fff700 1 -- 172.21.15.150:0/4102160072 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff72c022470 con 0x7ff73c101c90 2024-05-10T04:39:03.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.132+0000 7ff733fff700 1 -- 172.21.15.150:0/4102160072 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c017650 con 0x7ff73c101c90 2024-05-10T04:39:03.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.133+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c0052f0 con 0x7ff73c101c90 2024-05-10T04:39:03.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.135+0000 7ff733fff700 1 -- 172.21.15.150:0/4102160072 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff72c00fae0 con 0x7ff73c101c90 2024-05-10T04:39:03.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.135+0000 7ff733fff700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7280a1230 0x7ff7280a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.136+0000 7ff733fff700 1 -- 172.21.15.150:0/4102160072 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff72c0cdb50 con 0x7ff73c101c90 2024-05-10T04:39:03.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.136+0000 7ff73ad9d700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7280a1230 0x7ff7280a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.136+0000 7ff73ad9d700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7280a1230 0x7ff7280a3740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff724005950 tx=0x7ff72400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:03.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.140+0000 7ff733fff700 1 -- 172.21.15.150:0/4102160072 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff72c097020 con 0x7ff73c101c90 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 -- 172.21.15.150:0/1411583970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8100a20 msgr2=0x7f7ca8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 --2- 172.21.15.150:0/1411583970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8100a20 0x7f7ca8100e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009990 tx=0x7f7c9c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 -- 172.21.15.150:0/1411583970 shutdown_connections 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 --2- 172.21.15.150:0/1411583970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ca810e5d0 0x7f7ca81109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 --2- 172.21.15.150:0/1411583970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8100a20 0x7f7ca8100e30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 --2- 172.21.15.150:0/1411583970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8101370 0x7f7ca810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 -- 172.21.15.150:0/1411583970 >> 172.21.15.150:0/1411583970 conn(0x7f7ca80fc7a0 msgr2=0x7f7ca80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 -- 172.21.15.150:0/1411583970 shutdown_connections 2024-05-10T04:39:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.300+0000 7f7caee4f700 1 -- 172.21.15.150:0/1411583970 wait complete. 2024-05-10T04:39:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.301+0000 7f7caee4f700 1 Processor -- start 2024-05-10T04:39:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.301+0000 7f7caee4f700 1 -- start start 2024-05-10T04:39:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.301+0000 7f7caee4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8100a20 0x7f7ca81a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.301+0000 7f7caee4f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8101370 0x7f7ca81a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7caee4f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ca810e5d0 0x7f7ca819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7caee4f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819a9f0 con 0x7f7ca8101370 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7caee4f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819ab60 con 0x7f7ca810e5d0 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7caee4f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819ad20 con 0x7f7ca8100a20 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8101370 0x7f7ca81a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8101370 0x7f7ca81a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48612/0 (socket says 172.21.15.150:48612) 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 -- 172.21.15.150:0/4034519638 learned_addr learned my addr 172.21.15.150:0/4034519638 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7cacbeb700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8100a20 0x7f7ca81a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7cad3ec700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ca810e5d0 0x7f7ca819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 -- 172.21.15.150:0/4034519638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8100a20 msgr2=0x7f7ca81a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8100a20 0x7f7ca81a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 -- 172.21.15.150:0/4034519638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ca810e5d0 msgr2=0x7f7ca819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ca810e5d0 0x7f7ca819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 -- 172.21.15.150:0/4034519638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0095f0 con 0x7f7ca8101370 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca7fff700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8101370 0x7f7ca81a0a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400d980 tx=0x7f7c9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.302+0000 7f7ca5ffb700 1 -- 172.21.15.150:0/4034519638 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400f890 con 0x7f7ca8101370 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.303+0000 7f7ca5ffb700 1 -- 172.21.15.150:0/4034519638 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c94010460 con 0x7f7ca8101370 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.303+0000 7f7ca5ffb700 1 -- 172.21.15.150:0/4034519638 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400e610 con 0x7f7ca8101370 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.303+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca819b000 con 0x7f7ca8101370 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.303+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81a7350 con 0x7f7ca8101370 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.304+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca81097f0 con 0x7f7ca8101370 2024-05-10T04:39:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.305+0000 7f7ca5ffb700 1 -- 172.21.15.150:0/4034519638 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7c940105d0 con 0x7f7ca8101370 2024-05-10T04:39:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.306+0000 7f7ca5ffb700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c980a11b0 0x7f7c980a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.306+0000 7f7ca5ffb700 1 -- 172.21.15.150:0/4034519638 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7c940cc640 con 0x7f7ca8101370 2024-05-10T04:39:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.306+0000 7f7cacbeb700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c980a11b0 0x7f7c980a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.307+0000 7f7cacbeb700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c980a11b0 0x7f7c980a36c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00b5c0 tx=0x7f7c9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:03.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.311+0000 7f7ca5ffb700 1 -- 172.21.15.150:0/4034519638 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c94095a80 con 0x7f7ca8101370 2024-05-10T04:39:03.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.356+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff71c005160 con 0x7ff73c101c90 2024-05-10T04:39:03.358 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.357+0000 7ff733fff700 1 -- 172.21.15.150:0/4102160072 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff72c09a640 con 0x7ff73c101c90 2024-05-10T04:39:03.358 INFO:teuthology.orchestra.run.smithi150.stdout:163208757290 2024-05-10T04:39:03.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7280a1230 msgr2=0x7ff7280a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7280a1230 0x7ff7280a3740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff724005950 tx=0x7ff72400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101c90 msgr2=0x7ff73c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101c90 0x7ff73c198aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0098e0 tx=0x7ff72c00f670 comp rx=0 tx=0).stop 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 shutdown_connections 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff73c106c30 0x7ff73c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff7280a1230 0x7ff7280a3740 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c101340 0x7ff73c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 --2- 172.21.15.150:0/4102160072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff73c101c90 0x7ff73c198aa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 >> 172.21.15.150:0/4102160072 conn(0x7ff73c0fbed0 msgr2=0x7ff73c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 shutdown_connections 2024-05-10T04:39:03.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.360+0000 7ff7410ce700 1 -- 172.21.15.150:0/4102160072 wait complete. 2024-05-10T04:39:03.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 -- 172.21.15.150:0/1305307028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 msgr2=0x7f2af41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/1305307028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af41031b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009990 tx=0x7f2af0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 -- 172.21.15.150:0/1305307028 shutdown_connections 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/1305307028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2af410c210 0x7f2af410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/1305307028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af41031b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/1305307028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41036f0 0x7f2af410bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 -- 172.21.15.150:0/1305307028 >> 172.21.15.150:0/1305307028 conn(0x7f2af40fc7a0 msgr2=0x7f2af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 -- 172.21.15.150:0/1305307028 shutdown_connections 2024-05-10T04:39:03.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.424+0000 7f2afc3cd700 1 -- 172.21.15.150:0/1305307028 wait complete. 2024-05-10T04:39:03.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 Processor -- start 2024-05-10T04:39:03.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 -- start start 2024-05-10T04:39:03.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af4100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41036f0 0x7f2af4100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2af410c210 0x7f2af41a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4101840 con 0x7f2af4102da0 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41a6c20 con 0x7f2af41036f0 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afc3cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41a6d60 con 0x7f2af410c210 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afa169700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af4100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.425+0000 7f2afa169700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af4100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48632/0 (socket says 172.21.15.150:48632) 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa169700 1 -- 172.21.15.150:0/3553942331 learned_addr learned my addr 172.21.15.150:0/3553942331 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa96a700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2af410c210 0x7f2af41a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2af9968700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41036f0 0x7f2af4100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa169700 1 -- 172.21.15.150:0/3553942331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2af410c210 msgr2=0x7f2af41a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa169700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2af410c210 0x7f2af41a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa169700 1 -- 172.21.15.150:0/3553942331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41036f0 msgr2=0x7f2af4100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa169700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41036f0 0x7f2af4100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa169700 1 -- 172.21.15.150:0/3553942331 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00095f0 con 0x7f2af4102da0 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2af9968700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41036f0 0x7f2af4100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afa169700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af4100a90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098e0 tx=0x7f2af000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2aeb7fe700 1 -- 172.21.15.150:0/3553942331 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af001d070 con 0x7f2af4102da0 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2aeb7fe700 1 -- 172.21.15.150:0/3553942331 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af0022470 con 0x7f2af4102da0 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.426+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af41a6f90 con 0x7f2af4102da0 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.427+0000 7f2aeb7fe700 1 -- 172.21.15.150:0/3553942331 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0017650 con 0x7f2af4102da0 2024-05-10T04:39:03.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.427+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af41a7340 con 0x7f2af4102da0 2024-05-10T04:39:03.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.428+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af41072f0 con 0x7f2af4102da0 2024-05-10T04:39:03.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.429+0000 7f2aeb7fe700 1 -- 172.21.15.150:0/3553942331 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2af00225e0 con 0x7f2af4102da0 2024-05-10T04:39:03.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.430+0000 7f2aeb7fe700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ae00a11b0 0x7f2ae00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:03.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.430+0000 7f2aeb7fe700 1 -- 172.21.15.150:0/3553942331 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2af00cdd00 con 0x7f2af4102da0 2024-05-10T04:39:03.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.431+0000 7f2af9968700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ae00a11b0 0x7f2ae00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:03.432 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.431+0000 7f2af9968700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ae00a11b0 0x7f2ae00a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4005950 tx=0x7f2ae40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:03.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.435+0000 7f2aeb7fe700 1 -- 172.21.15.150:0/3553942331 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2af0097170 con 0x7f2af4102da0 2024-05-10T04:39:03.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.547+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f7ca819bbd0 con 0x7f7ca8101370 2024-05-10T04:39:03.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.547+0000 7f7ca5ffb700 1 -- 172.21.15.150:0/4034519638 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f7c940990a0 con 0x7f7ca8101370 2024-05-10T04:39:03.549 INFO:teuthology.orchestra.run.smithi150.stdout:330712481813 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c980a11b0 msgr2=0x7f7c980a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c980a11b0 0x7f7c980a36c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00b5c0 tx=0x7f7c9c005960 comp rx=0 tx=0).stop 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8101370 msgr2=0x7f7ca81a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8101370 0x7f7ca81a0a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400d980 tx=0x7f7c9400dc90 comp rx=0 tx=0).stop 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 shutdown_connections 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ca810e5d0 0x7f7ca819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c980a11b0 0x7f7c980a36c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8100a20 0x7f7ca81a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 --2- 172.21.15.150:0/4034519638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ca8101370 0x7f7ca81a0a90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 >> 172.21.15.150:0/4034519638 conn(0x7f7ca80fc7a0 msgr2=0x7f7ca81101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.549+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 shutdown_connections 2024-05-10T04:39:03.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.550+0000 7f7caee4f700 1 -- 172.21.15.150:0/4034519638 wait complete. 2024-05-10T04:39:03.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.645+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2af41021d0 con 0x7f2af4102da0 2024-05-10T04:39:03.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.646+0000 7f2aeb7fe700 1 -- 172.21.15.150:0/3553942331 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2af009a790 con 0x7f2af4102da0 2024-05-10T04:39:03.647 INFO:teuthology.orchestra.run.smithi150.stdout:193273528358 2024-05-10T04:39:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.648+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ae00a11b0 msgr2=0x7f2ae00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.648+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ae00a11b0 0x7f2ae00a36c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4005950 tx=0x7f2ae40058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.648+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 msgr2=0x7f2af4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:03.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.648+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af4100a90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098e0 tx=0x7f2af000f670 comp rx=0 tx=0).stop 2024-05-10T04:39:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 shutdown_connections 2024-05-10T04:39:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2ae00a11b0 0x7f2ae00a36c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2af410c210 0x7f2af41a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af4102da0 0x7f2af4100a90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 --2- 172.21.15.150:0/3553942331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41036f0 0x7f2af4100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 >> 172.21.15.150:0/3553942331 conn(0x7f2af40fc7a0 msgr2=0x7f2af40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:03.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 shutdown_connections 2024-05-10T04:39:03.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:03.651+0000 7f2afc3cd700 1 -- 172.21.15.150:0/3553942331 wait complete. 2024-05-10T04:39:03.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:03 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4102160072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T04:39:03.753 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757288 got 163208757290 for osd.5 2024-05-10T04:39:03.753 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:03.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:03 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4102160072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T04:39:03.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:03 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4102160072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-10T04:39:04.193 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528356 got 193273528358 for osd.6 2024-05-10T04:39:04.193 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:04.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481811 got 330712481813 for osd.11 2024-05-10T04:39:04.201 DEBUG:teuthology.parallel:result is None 2024-05-10T04:39:04.201 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-10T04:39:04.201 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T04:39:04.469 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:04.565 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:04 smithi150 ceph-mon[42404]: pgmap v252: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:39:04.565 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:04 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/4034519638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T04:39:04.565 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:04 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3553942331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T04:39:04.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:04 smithi161 ceph-mon[46912]: pgmap v252: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:39:04.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:04 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/4034519638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T04:39:04.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:04 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3553942331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T04:39:04.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:04 smithi181 ceph-mon[47314]: pgmap v252: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:39:04.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:04 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/4034519638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-10T04:39:04.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:04 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3553942331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 -- 172.21.15.150:0/301230190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744100a40 msgr2=0x7f9744100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 --2- 172.21.15.150:0/301230190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744100a40 0x7f9744100e50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9738009990 tx=0x7f9738009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 -- 172.21.15.150:0/301230190 shutdown_connections 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 --2- 172.21.15.150:0/301230190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f974410e5f0 0x7f9744110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 --2- 172.21.15.150:0/301230190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744100a40 0x7f9744100e50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 --2- 172.21.15.150:0/301230190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9744101390 0x7f974410e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 -- 172.21.15.150:0/301230190 >> 172.21.15.150:0/301230190 conn(0x7f97440fc7a0 msgr2=0x7f97440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 -- 172.21.15.150:0/301230190 shutdown_connections 2024-05-10T04:39:05.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.378+0000 7f97492f7700 1 -- 172.21.15.150:0/301230190 wait complete. 2024-05-10T04:39:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.379+0000 7f97492f7700 1 Processor -- start 2024-05-10T04:39:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.379+0000 7f97492f7700 1 -- start start 2024-05-10T04:39:05.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.379+0000 7f97492f7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9744100a40 0x7f97441a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.379+0000 7f97492f7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744101390 0x7f97441a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.379+0000 7f97492f7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974410e5f0 0x7f974419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97492f7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974419aac0 con 0x7f9744101390 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97492f7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974419ac30 con 0x7f974410e5f0 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97492f7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974419adf0 con 0x7f9744100a40 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744101390 0x7f97441a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744101390 0x7f97441a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48662/0 (socket says 172.21.15.150:48662) 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 -- 172.21.15.150:0/2178011752 learned_addr learned my addr 172.21.15.150:0/2178011752 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f9742ffd700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9744100a40 0x7f97441a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97437fe700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974410e5f0 0x7f974419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 -- 172.21.15.150:0/2178011752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9744100a40 msgr2=0x7f97441a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9744100a40 0x7f97441a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 -- 172.21.15.150:0/2178011752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974410e5f0 msgr2=0x7f974419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974410e5f0 0x7f974419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 -- 172.21.15.150:0/2178011752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380095f0 con 0x7f9744101390 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97437fe700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974410e5f0 0x7f974419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97427fc700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744101390 0x7f97441a0ad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f973400d980 tx=0x7f973400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f972bfff700 1 -- 172.21.15.150:0/2178011752 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400b950 con 0x7f9744101390 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f972bfff700 1 -- 172.21.15.150:0/2178011752 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9734007620 con 0x7f9744101390 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974419b0d0 con 0x7f9744101390 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f972bfff700 1 -- 172.21.15.150:0/2178011752 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400f3c0 con 0x7f9744101390 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.380+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97441a7560 con 0x7f9744101390 2024-05-10T04:39:05.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.382+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744109810 con 0x7f9744101390 2024-05-10T04:39:05.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.383+0000 7f972bfff700 1 -- 172.21.15.150:0/2178011752 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9734010640 con 0x7f9744101390 2024-05-10T04:39:05.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.384+0000 7f972bfff700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f972c0a1230 0x7f972c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:05.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.384+0000 7f9742ffd700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f972c0a1230 0x7f972c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:05.385 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.384+0000 7f972bfff700 1 -- 172.21.15.150:0/2178011752 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f97340cd310 con 0x7f9744101390 2024-05-10T04:39:05.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.385+0000 7f9742ffd700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f972c0a1230 0x7f972c0a3740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f973800b5c0 tx=0x7f9738005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:05.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.387+0000 7f972bfff700 1 -- 172.21.15.150:0/2178011752 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97340966f0 con 0x7f9744101390 2024-05-10T04:39:05.577 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.576+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f974419bc10 con 0x7f972c0a1230 2024-05-10T04:39:05.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.588+0000 7f972bfff700 1 -- 172.21.15.150:0/2178011752 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422962 (secure 0 0 0) 0x7f974419bc10 con 0x7f972c0a1230 2024-05-10T04:39:05.595 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:05.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f972c0a1230 msgr2=0x7f972c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:05.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f972c0a1230 0x7f972c0a3740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f973800b5c0 tx=0x7f9738005960 comp rx=0 tx=0).stop 2024-05-10T04:39:05.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744101390 msgr2=0x7f97441a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:05.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744101390 0x7f97441a0ad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f973400d980 tx=0x7f973400dc90 comp rx=0 tx=0).stop 2024-05-10T04:39:05.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 shutdown_connections 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f974410e5f0 0x7f974419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f972c0a1230 0x7f972c0a3740 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9744100a40 0x7f97441a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 --2- 172.21.15.150:0/2178011752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9744101390 0x7f97441a0ad0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.591+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 >> 172.21.15.150:0/2178011752 conn(0x7f97440fc7a0 msgr2=0x7f9744110120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.592+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 shutdown_connections 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:05.592+0000 7f97492f7700 1 -- 172.21.15.150:0/2178011752 wait complete. 2024-05-10T04:39:05.599 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:39:05.982 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-05-10T04:39:05.318967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658885,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":449,"num_read_kb":341,"num_write":416,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360332,"kb_used_data":11564,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581492,"statfs":{"total":1151940427776,"available":1151571447808,"internally_reserved":0,"allocated":11841536,"data_stored":7536330,"data_compressed":251892,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19127,"internal_metadata":357021001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3564,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":54,"num_write":2,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003670"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787827+0000","last_change":"2024-05-10T04:38:08.784019+0000","last_active":"2024-05-10T04:38:12.787827+0000","last_peered":"2024-05-10T04:38:12.787827+0000","last_clean":"2024-05-10T04:38:12.787827+0000","last_became_active":"2024-05-10T04:38:08.783585+0000","last_became_peered":"2024-05-10T04:38:08.783585+0000","last_unstale":"2024-05-10T04:38:12.787827+0000","last_undegraded":"2024-05-10T04:38:12.787827+0000","last_fullsized":"2024-05-10T04:38:12.787827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:56.963068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788295+0000","last_change":"2024-05-10T04:38:08.777364+0000","last_active":"2024-05-10T04:38:12.788295+0000","last_peered":"2024-05-10T04:38:12.788295+0000","last_clean":"2024-05-10T04:38:12.788295+0000","last_became_active":"2024-05-10T04:38:08.776488+0000","last_became_peered":"2024-05-10T04:38:08.776488+0000","last_unstale":"2024-05-10T04:38:12.788295+0000","last_undegraded":"2024-05-10T04:38:12.788295+0000","last_fullsized":"2024-05-10T04:38:12.788295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:40.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787110+0000","last_change":"2024-05-10T04:38:08.777169+0000","last_active":"2024-05-10T04:38:12.787110+0000","last_peered":"2024-05-10T04:38:12.787110+0000","last_clean":"2024-05-10T04:38:12.787110+0000","last_became_active":"2024-05-10T04:38:08.776713+0000","last_became_peered":"2024-05-10T04:38:08.776713+0000","last_unstale":"2024-05-10T04:38:12.787110+0000","last_undegraded":"2024-05-10T04:38:12.787110+0000","last_fullsized":"2024-05-10T04:38:12.787110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:08:41.947577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791723+0000","last_change":"2024-05-10T04:38:08.788464+0000","last_active":"2024-05-10T04:38:12.791723+0000","last_peered":"2024-05-10T04:38:12.791723+0000","last_clean":"2024-05-10T04:38:12.791723+0000","last_became_active":"2024-05-10T04:38:08.787811+0000","last_became_peered":"2024-05-10T04:38:08.787811+0000","last_unstale":"2024-05-10T04:38:12.791723+0000","last_undegraded":"2024-05-10T04:38:12.791723+0000","last_fullsized":"2024-05-10T04:38:12.791723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:22.574882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156610+0000","last_change":"2024-05-10T04:38:08.772904+0000","last_active":"2024-05-10T04:38:13.156610+0000","last_peered":"2024-05-10T04:38:13.156610+0000","last_clean":"2024-05-10T04:38:13.156610+0000","last_became_active":"2024-05-10T04:38:08.772503+0000","last_became_peered":"2024-05-10T04:38:08.772503+0000","last_unstale":"2024-05-10T04:38:13.156610+0000","last_undegraded":"2024-05-10T04:38:13.156610+0000","last_fullsized":"2024-05-10T04:38:13.156610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:58.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809064+0000","last_change":"2024-05-10T04:38:08.779442+0000","last_active":"2024-05-10T04:38:12.809064+0000","last_peered":"2024-05-10T04:38:12.809064+0000","last_clean":"2024-05-10T04:38:12.809064+0000","last_became_active":"2024-05-10T04:38:08.777348+0000","last_became_peered":"2024-05-10T04:38:08.777348+0000","last_unstale":"2024-05-10T04:38:12.809064+0000","last_undegraded":"2024-05-10T04:38:12.809064+0000","last_fullsized":"2024-05-10T04:38:12.809064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:25.619031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786546+0000","last_change":"2024-05-10T04:38:08.775646+0000","last_active":"2024-05-10T04:38:12.786546+0000","last_peered":"2024-05-10T04:38:12.786546+0000","last_clean":"2024-05-10T04:38:12.786546+0000","last_became_active":"2024-05-10T04:38:08.768229+0000","last_became_peered":"2024-05-10T04:38:08.768229+0000","last_unstale":"2024-05-10T04:38:12.786546+0000","last_undegraded":"2024-05-10T04:38:12.786546+0000","last_fullsized":"2024-05-10T04:38:12.786546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:14.653065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787545+0000","last_change":"2024-05-10T04:38:08.771012+0000","last_active":"2024-05-10T04:38:12.787545+0000","last_peered":"2024-05-10T04:38:12.787545+0000","last_clean":"2024-05-10T04:38:12.787545+0000","last_became_active":"2024-05-10T04:38:08.770674+0000","last_became_peered":"2024-05-10T04:38:08.770674+0000","last_unstale":"2024-05-10T04:38:12.787545+0000","last_undegraded":"2024-05-10T04:38:12.787545+0000","last_fullsized":"2024-05-10T04:38:12.787545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:14.389263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784668+0000","last_change":"2024-05-10T04:38:08.782109+0000","last_active":"2024-05-10T04:38:12.784668+0000","last_peered":"2024-05-10T04:38:12.784668+0000","last_clean":"2024-05-10T04:38:12.784668+0000","last_became_active":"2024-05-10T04:38:08.781717+0000","last_became_peered":"2024-05-10T04:38:08.781717+0000","last_unstale":"2024-05-10T04:38:12.784668+0000","last_undegraded":"2024-05-10T04:38:12.784668+0000","last_fullsized":"2024-05-10T04:38:12.784668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:17.505448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809930+0000","last_change":"2024-05-10T04:38:08.778429+0000","last_active":"2024-05-10T04:38:12.809930+0000","last_peered":"2024-05-10T04:38:12.809930+0000","last_clean":"2024-05-10T04:38:12.809930+0000","last_became_active":"2024-05-10T04:38:08.777874+0000","last_became_peered":"2024-05-10T04:38:08.777874+0000","last_unstale":"2024-05-10T04:38:12.809930+0000","last_undegraded":"2024-05-10T04:38:12.809930+0000","last_fullsized":"2024-05-10T04:38:12.809930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:20:19.704678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786985+0000","last_change":"2024-05-10T04:38:08.780104+0000","last_active":"2024-05-10T04:38:12.786985+0000","last_peered":"2024-05-10T04:38:12.786985+0000","last_clean":"2024-05-10T04:38:12.786985+0000","last_became_active":"2024-05-10T04:38:08.779298+0000","last_became_peered":"2024-05-10T04:38:08.779298+0000","last_unstale":"2024-05-10T04:38:12.786985+0000","last_undegraded":"2024-05-10T04:38:12.786985+0000","last_fullsized":"2024-05-10T04:38:12.786985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:41.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791619+0000","last_change":"2024-05-10T04:38:08.788605+0000","last_active":"2024-05-10T04:38:12.791619+0000","last_peered":"2024-05-10T04:38:12.791619+0000","last_clean":"2024-05-10T04:38:12.791619+0000","last_became_active":"2024-05-10T04:38:08.788207+0000","last_became_peered":"2024-05-10T04:38:08.788207+0000","last_unstale":"2024-05-10T04:38:12.791619+0000","last_undegraded":"2024-05-10T04:38:12.791619+0000","last_fullsized":"2024-05-10T04:38:12.791619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:35.000980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782473+0000","last_change":"2024-05-10T04:38:08.761953+0000","last_active":"2024-05-10T04:38:12.782473+0000","last_peered":"2024-05-10T04:38:12.782473+0000","last_clean":"2024-05-10T04:38:12.782473+0000","last_became_active":"2024-05-10T04:38:08.761585+0000","last_became_peered":"2024-05-10T04:38:08.761585+0000","last_unstale":"2024-05-10T04:38:12.782473+0000","last_undegraded":"2024-05-10T04:38:12.782473+0000","last_fullsized":"2024-05-10T04:38:12.782473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:25.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786743+0000","last_change":"2024-05-10T04:38:08.777656+0000","last_active":"2024-05-10T04:38:12.786743+0000","last_peered":"2024-05-10T04:38:12.786743+0000","last_clean":"2024-05-10T04:38:12.786743+0000","last_became_active":"2024-05-10T04:38:08.777332+0000","last_became_peered":"2024-05-10T04:38:08.777332+0000","last_unstale":"2024-05-10T04:38:12.786743+0000","last_undegraded":"2024-05-10T04:38:12.786743+0000","last_fullsized":"2024-05-10T04:38:12.786743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:17:52.878943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787112+0000","last_change":"2024-05-10T04:38:08.784934+0000","last_active":"2024-05-10T04:38:12.787112+0000","last_peered":"2024-05-10T04:38:12.787112+0000","last_clean":"2024-05-10T04:38:12.787112+0000","last_became_active":"2024-05-10T04:38:08.784644+0000","last_became_peered":"2024-05-10T04:38:08.784644+0000","last_unstale":"2024-05-10T04:38:12.787112+0000","last_undegraded":"2024-05-10T04:38:12.787112+0000","last_fullsized":"2024-05-10T04:38:12.787112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:43.889305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783537+0000","last_change":"2024-05-10T04:38:08.784110+0000","last_active":"2024-05-10T04:38:12.783537+0000","last_peered":"2024-05-10T04:38:12.783537+0000","last_clean":"2024-05-10T04:38:12.783537+0000","last_became_active":"2024-05-10T04:38:08.783779+0000","last_became_peered":"2024-05-10T04:38:08.783779+0000","last_unstale":"2024-05-10T04:38:12.783537+0000","last_undegraded":"2024-05-10T04:38:12.783537+0000","last_fullsized":"2024-05-10T04:38:12.783537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:24.233024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787536+0000","last_change":"2024-05-10T04:38:08.788685+0000","last_active":"2024-05-10T04:38:12.787536+0000","last_peered":"2024-05-10T04:38:12.787536+0000","last_clean":"2024-05-10T04:38:12.787536+0000","last_became_active":"2024-05-10T04:38:08.788122+0000","last_became_peered":"2024-05-10T04:38:08.788122+0000","last_unstale":"2024-05-10T04:38:12.787536+0000","last_undegraded":"2024-05-10T04:38:12.787536+0000","last_fullsized":"2024-05-10T04:38:12.787536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:21:01.098784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802717+0000","last_change":"2024-05-10T04:38:08.776330+0000","last_active":"2024-05-10T04:38:12.802717+0000","last_peered":"2024-05-10T04:38:12.802717+0000","last_clean":"2024-05-10T04:38:12.802717+0000","last_became_active":"2024-05-10T04:38:08.776005+0000","last_became_peered":"2024-05-10T04:38:08.776005+0000","last_unstale":"2024-05-10T04:38:12.802717+0000","last_undegraded":"2024-05-10T04:38:12.802717+0000","last_fullsized":"2024-05-10T04:38:12.802717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:46:14.407766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787933+0000","last_change":"2024-05-10T04:38:08.772811+0000","last_active":"2024-05-10T04:38:12.787933+0000","last_peered":"2024-05-10T04:38:12.787933+0000","last_clean":"2024-05-10T04:38:12.787933+0000","last_became_active":"2024-05-10T04:38:08.772462+0000","last_became_peered":"2024-05-10T04:38:08.772462+0000","last_unstale":"2024-05-10T04:38:12.787933+0000","last_undegraded":"2024-05-10T04:38:12.787933+0000","last_fullsized":"2024-05-10T04:38:12.787933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:39.703137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410816+0000","last_change":"2024-05-10T04:38:12.799483+0000","last_active":"2024-05-10T04:38:13.410816+0000","last_peered":"2024-05-10T04:38:13.410816+0000","last_clean":"2024-05-10T04:38:13.410816+0000","last_became_active":"2024-05-10T04:38:12.799049+0000","last_became_peered":"2024-05-10T04:38:12.799049+0000","last_unstale":"2024-05-10T04:38:13.410816+0000","last_undegraded":"2024-05-10T04:38:13.410816+0000","last_fullsized":"2024-05-10T04:38:13.410816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:00.380645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":55,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:59.017819+0000","last_change":"2024-05-10T04:38:08.345935+0000","last_active":"2024-05-10T04:38:59.017819+0000","last_peered":"2024-05-10T04:38:59.017819+0000","last_clean":"2024-05-10T04:38:59.017819+0000","last_became_active":"2024-05-10T04:38:08.345580+0000","last_became_peered":"2024-05-10T04:38:08.345580+0000","last_unstale":"2024-05-10T04:38:59.017819+0000","last_undegraded":"2024-05-10T04:38:59.017819+0000","last_fullsized":"2024-05-10T04:38:59.017819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:11.527710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787511+0000","last_change":"2024-05-10T04:38:08.771072+0000","last_active":"2024-05-10T04:38:12.787511+0000","last_peered":"2024-05-10T04:38:12.787511+0000","last_clean":"2024-05-10T04:38:12.787511+0000","last_became_active":"2024-05-10T04:38:08.770711+0000","last_became_peered":"2024-05-10T04:38:08.770711+0000","last_unstale":"2024-05-10T04:38:12.787511+0000","last_undegraded":"2024-05-10T04:38:12.787511+0000","last_fullsized":"2024-05-10T04:38:12.787511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:44.803219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220274+0000","last_change":"2024-05-10T04:38:12.801374+0000","last_active":"2024-05-10T04:38:13.220274+0000","last_peered":"2024-05-10T04:38:13.220274+0000","last_clean":"2024-05-10T04:38:13.220274+0000","last_became_active":"2024-05-10T04:38:12.800889+0000","last_became_peered":"2024-05-10T04:38:12.800889+0000","last_unstale":"2024-05-10T04:38:13.220274+0000","last_undegraded":"2024-05-10T04:38:13.220274+0000","last_fullsized":"2024-05-10T04:38:13.220274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:35:51.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217645+0000","last_change":"2024-05-10T04:38:08.348806+0000","last_active":"2024-05-10T04:38:13.217645+0000","last_peered":"2024-05-10T04:38:13.217645+0000","last_clean":"2024-05-10T04:38:13.217645+0000","last_became_active":"2024-05-10T04:38:08.348513+0000","last_became_peered":"2024-05-10T04:38:08.348513+0000","last_unstale":"2024-05-10T04:38:13.217645+0000","last_undegraded":"2024-05-10T04:38:13.217645+0000","last_fullsized":"2024-05-10T04:38:13.217645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:45.474403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787353+0000","last_change":"2024-05-10T04:38:08.782738+0000","last_active":"2024-05-10T04:38:12.787353+0000","last_peered":"2024-05-10T04:38:12.787353+0000","last_clean":"2024-05-10T04:38:12.787353+0000","last_became_active":"2024-05-10T04:38:08.782375+0000","last_became_peered":"2024-05-10T04:38:08.782375+0000","last_unstale":"2024-05-10T04:38:12.787353+0000","last_undegraded":"2024-05-10T04:38:12.787353+0000","last_fullsized":"2024-05-10T04:38:12.787353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:16.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220388+0000","last_change":"2024-05-10T04:38:12.801322+0000","last_active":"2024-05-10T04:38:13.220388+0000","last_peered":"2024-05-10T04:38:13.220388+0000","last_clean":"2024-05-10T04:38:13.220388+0000","last_became_active":"2024-05-10T04:38:12.800388+0000","last_became_peered":"2024-05-10T04:38:12.800388+0000","last_unstale":"2024-05-10T04:38:13.220388+0000","last_undegraded":"2024-05-10T04:38:13.220388+0000","last_fullsized":"2024-05-10T04:38:13.220388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:57.678691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.985782+0000","last_change":"2024-05-10T04:38:08.347997+0000","last_active":"2024-05-10T04:38:35.985782+0000","last_peered":"2024-05-10T04:38:35.985782+0000","last_clean":"2024-05-10T04:38:35.985782+0000","last_became_active":"2024-05-10T04:38:08.347469+0000","last_became_peered":"2024-05-10T04:38:08.347469+0000","last_unstale":"2024-05-10T04:38:35.985782+0000","last_undegraded":"2024-05-10T04:38:35.985782+0000","last_fullsized":"2024-05-10T04:38:35.985782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:39.122148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783940+0000","last_change":"2024-05-10T04:38:08.781068+0000","last_active":"2024-05-10T04:38:12.783940+0000","last_peered":"2024-05-10T04:38:12.783940+0000","last_clean":"2024-05-10T04:38:12.783940+0000","last_became_active":"2024-05-10T04:38:08.780671+0000","last_became_peered":"2024-05-10T04:38:08.780671+0000","last_unstale":"2024-05-10T04:38:12.783940+0000","last_undegraded":"2024-05-10T04:38:12.783940+0000","last_fullsized":"2024-05-10T04:38:12.783940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220322+0000","last_change":"2024-05-10T04:38:12.801702+0000","last_active":"2024-05-10T04:38:13.220322+0000","last_peered":"2024-05-10T04:38:13.220322+0000","last_clean":"2024-05-10T04:38:13.220322+0000","last_became_active":"2024-05-10T04:38:12.801031+0000","last_became_peered":"2024-05-10T04:38:12.801031+0000","last_unstale":"2024-05-10T04:38:13.220322+0000","last_undegraded":"2024-05-10T04:38:13.220322+0000","last_fullsized":"2024-05-10T04:38:13.220322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:05:07.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970534+0000","last_change":"2024-05-10T04:38:08.347981+0000","last_active":"2024-05-10T04:38:34.970534+0000","last_peered":"2024-05-10T04:38:34.970534+0000","last_clean":"2024-05-10T04:38:34.970534+0000","last_became_active":"2024-05-10T04:38:08.346915+0000","last_became_peered":"2024-05-10T04:38:08.346915+0000","last_unstale":"2024-05-10T04:38:34.970534+0000","last_undegraded":"2024-05-10T04:38:34.970534+0000","last_fullsized":"2024-05-10T04:38:34.970534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:52.383734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780699+0000","last_change":"2024-05-10T04:38:08.779164+0000","last_active":"2024-05-10T04:38:12.780699+0000","last_peered":"2024-05-10T04:38:12.780699+0000","last_clean":"2024-05-10T04:38:12.780699+0000","last_became_active":"2024-05-10T04:38:08.778851+0000","last_became_peered":"2024-05-10T04:38:08.778851+0000","last_unstale":"2024-05-10T04:38:12.780699+0000","last_undegraded":"2024-05-10T04:38:12.780699+0000","last_fullsized":"2024-05-10T04:38:12.780699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:01.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411122+0000","last_change":"2024-05-10T04:38:12.796175+0000","last_active":"2024-05-10T04:38:13.411122+0000","last_peered":"2024-05-10T04:38:13.411122+0000","last_clean":"2024-05-10T04:38:13.411122+0000","last_became_active":"2024-05-10T04:38:12.795752+0000","last_became_peered":"2024-05-10T04:38:12.795752+0000","last_unstale":"2024-05-10T04:38:13.411122+0000","last_undegraded":"2024-05-10T04:38:13.411122+0000","last_fullsized":"2024-05-10T04:38:13.411122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:48.077369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:03.992208+0000","last_change":"2024-05-10T04:38:08.341633+0000","last_active":"2024-05-10T04:39:03.992208+0000","last_peered":"2024-05-10T04:39:03.992208+0000","last_clean":"2024-05-10T04:39:03.992208+0000","last_became_active":"2024-05-10T04:38:08.341220+0000","last_became_peered":"2024-05-10T04:38:08.341220+0000","last_unstale":"2024-05-10T04:39:03.992208+0000","last_undegraded":"2024-05-10T04:39:03.992208+0000","last_fullsized":"2024-05-10T04:39:03.992208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:36.861191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156596+0000","last_change":"2024-05-10T04:38:08.790969+0000","last_active":"2024-05-10T04:38:13.156596+0000","last_peered":"2024-05-10T04:38:13.156596+0000","last_clean":"2024-05-10T04:38:13.156596+0000","last_became_active":"2024-05-10T04:38:08.790539+0000","last_became_peered":"2024-05-10T04:38:08.790539+0000","last_unstale":"2024-05-10T04:38:13.156596+0000","last_undegraded":"2024-05-10T04:38:13.156596+0000","last_fullsized":"2024-05-10T04:38:13.156596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:50.174798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411137+0000","last_change":"2024-05-10T04:38:12.800901+0000","last_active":"2024-05-10T04:38:13.411137+0000","last_peered":"2024-05-10T04:38:13.411137+0000","last_clean":"2024-05-10T04:38:13.411137+0000","last_became_active":"2024-05-10T04:38:12.800287+0000","last_became_peered":"2024-05-10T04:38:12.800287+0000","last_unstale":"2024-05-10T04:38:13.411137+0000","last_undegraded":"2024-05-10T04:38:13.411137+0000","last_fullsized":"2024-05-10T04:38:13.411137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:23:25.463044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.965633+0000","last_change":"2024-05-10T04:38:08.342449+0000","last_active":"2024-05-10T04:38:33.965633+0000","last_peered":"2024-05-10T04:38:33.965633+0000","last_clean":"2024-05-10T04:38:33.965633+0000","last_became_active":"2024-05-10T04:38:08.342043+0000","last_became_peered":"2024-05-10T04:38:08.342043+0000","last_unstale":"2024-05-10T04:38:33.965633+0000","last_undegraded":"2024-05-10T04:38:33.965633+0000","last_fullsized":"2024-05-10T04:38:33.965633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:31.161203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780542+0000","last_change":"2024-05-10T04:38:08.783759+0000","last_active":"2024-05-10T04:38:12.780542+0000","last_peered":"2024-05-10T04:38:12.780542+0000","last_clean":"2024-05-10T04:38:12.780542+0000","last_became_active":"2024-05-10T04:38:08.783419+0000","last_became_peered":"2024-05-10T04:38:08.783419+0000","last_unstale":"2024-05-10T04:38:12.780542+0000","last_undegraded":"2024-05-10T04:38:12.780542+0000","last_fullsized":"2024-05-10T04:38:12.780542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:16.114017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218482+0000","last_change":"2024-05-10T04:38:12.803018+0000","last_active":"2024-05-10T04:38:13.218482+0000","last_peered":"2024-05-10T04:38:13.218482+0000","last_clean":"2024-05-10T04:38:13.218482+0000","last_became_active":"2024-05-10T04:38:12.802546+0000","last_became_peered":"2024-05-10T04:38:12.802546+0000","last_unstale":"2024-05-10T04:38:13.218482+0000","last_undegraded":"2024-05-10T04:38:13.218482+0000","last_fullsized":"2024-05-10T04:38:13.218482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:40.676335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950654+0000","last_change":"2024-05-10T04:38:08.341404+0000","last_active":"2024-05-10T04:38:32.950654+0000","last_peered":"2024-05-10T04:38:32.950654+0000","last_clean":"2024-05-10T04:38:32.950654+0000","last_became_active":"2024-05-10T04:38:08.341013+0000","last_became_peered":"2024-05-10T04:38:08.341013+0000","last_unstale":"2024-05-10T04:38:32.950654+0000","last_undegraded":"2024-05-10T04:38:32.950654+0000","last_fullsized":"2024-05-10T04:38:32.950654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:43.513989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787639+0000","last_change":"2024-05-10T04:38:08.775650+0000","last_active":"2024-05-10T04:38:12.787639+0000","last_peered":"2024-05-10T04:38:12.787639+0000","last_clean":"2024-05-10T04:38:12.787639+0000","last_became_active":"2024-05-10T04:38:08.775320+0000","last_became_peered":"2024-05-10T04:38:08.775320+0000","last_unstale":"2024-05-10T04:38:12.787639+0000","last_undegraded":"2024-05-10T04:38:12.787639+0000","last_fullsized":"2024-05-10T04:38:12.787639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:49:07.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.341020+0000","last_change":"2024-05-10T04:38:12.797168+0000","last_active":"2024-05-10T04:38:13.341020+0000","last_peered":"2024-05-10T04:38:13.341020+0000","last_clean":"2024-05-10T04:38:13.341020+0000","last_became_active":"2024-05-10T04:38:12.796776+0000","last_became_peered":"2024-05-10T04:38:12.796776+0000","last_unstale":"2024-05-10T04:38:13.341020+0000","last_undegraded":"2024-05-10T04:38:13.341020+0000","last_fullsized":"2024-05-10T04:38:13.341020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:59:02.363398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970039+0000","last_change":"2024-05-10T04:38:08.344086+0000","last_active":"2024-05-10T04:38:34.970039+0000","last_peered":"2024-05-10T04:38:34.970039+0000","last_clean":"2024-05-10T04:38:34.970039+0000","last_became_active":"2024-05-10T04:38:08.343744+0000","last_became_peered":"2024-05-10T04:38:08.343744+0000","last_unstale":"2024-05-10T04:38:34.970039+0000","last_undegraded":"2024-05-10T04:38:34.970039+0000","last_fullsized":"2024-05-10T04:38:34.970039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:01.800590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802611+0000","last_change":"2024-05-10T04:38:08.773638+0000","last_active":"2024-05-10T04:38:12.802611+0000","last_peered":"2024-05-10T04:38:12.802611+0000","last_clean":"2024-05-10T04:38:12.802611+0000","last_became_active":"2024-05-10T04:38:08.773218+0000","last_became_peered":"2024-05-10T04:38:08.773218+0000","last_unstale":"2024-05-10T04:38:12.802611+0000","last_undegraded":"2024-05-10T04:38:12.802611+0000","last_fullsized":"2024-05-10T04:38:12.802611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:54:38.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217138+0000","last_change":"2024-05-10T04:38:12.796344+0000","last_active":"2024-05-10T04:38:13.217138+0000","last_peered":"2024-05-10T04:38:13.217138+0000","last_clean":"2024-05-10T04:38:13.217138+0000","last_became_active":"2024-05-10T04:38:12.795958+0000","last_became_peered":"2024-05-10T04:38:12.795958+0000","last_unstale":"2024-05-10T04:38:13.217138+0000","last_undegraded":"2024-05-10T04:38:13.217138+0000","last_fullsized":"2024-05-10T04:38:13.217138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:45.708078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:01.988367+0000","last_change":"2024-05-10T04:38:08.341327+0000","last_active":"2024-05-10T04:39:01.988367+0000","last_peered":"2024-05-10T04:39:01.988367+0000","last_clean":"2024-05-10T04:39:01.988367+0000","last_became_active":"2024-05-10T04:38:08.339606+0000","last_became_peered":"2024-05-10T04:38:08.339606+0000","last_unstale":"2024-05-10T04:39:01.988367+0000","last_undegraded":"2024-05-10T04:39:01.988367+0000","last_fullsized":"2024-05-10T04:39:01.988367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:22:40.212212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784430+0000","last_change":"2024-05-10T04:38:08.781195+0000","last_active":"2024-05-10T04:38:12.784430+0000","last_peered":"2024-05-10T04:38:12.784430+0000","last_clean":"2024-05-10T04:38:12.784430+0000","last_became_active":"2024-05-10T04:38:08.780779+0000","last_became_peered":"2024-05-10T04:38:08.780779+0000","last_unstale":"2024-05-10T04:38:12.784430+0000","last_undegraded":"2024-05-10T04:38:12.784430+0000","last_fullsized":"2024-05-10T04:38:12.784430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:24.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930221+0000","last_change":"2024-05-10T04:38:12.799494+0000","last_active":"2024-05-10T04:38:13.930221+0000","last_peered":"2024-05-10T04:38:13.930221+0000","last_clean":"2024-05-10T04:38:13.930221+0000","last_became_active":"2024-05-10T04:38:12.799071+0000","last_became_peered":"2024-05-10T04:38:12.799071+0000","last_unstale":"2024-05-10T04:38:13.930221+0000","last_undegraded":"2024-05-10T04:38:13.930221+0000","last_fullsized":"2024-05-10T04:38:13.930221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:00.814875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970501+0000","last_change":"2024-05-10T04:38:08.348569+0000","last_active":"2024-05-10T04:38:34.970501+0000","last_peered":"2024-05-10T04:38:34.970501+0000","last_clean":"2024-05-10T04:38:34.970501+0000","last_became_active":"2024-05-10T04:38:08.347942+0000","last_became_peered":"2024-05-10T04:38:08.347942+0000","last_unstale":"2024-05-10T04:38:34.970501+0000","last_undegraded":"2024-05-10T04:38:34.970501+0000","last_fullsized":"2024-05-10T04:38:34.970501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.865369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787640+0000","last_change":"2024-05-10T04:38:08.784447+0000","last_active":"2024-05-10T04:38:12.787640+0000","last_peered":"2024-05-10T04:38:12.787640+0000","last_clean":"2024-05-10T04:38:12.787640+0000","last_became_active":"2024-05-10T04:38:08.783989+0000","last_became_peered":"2024-05-10T04:38:08.783989+0000","last_unstale":"2024-05-10T04:38:12.787640+0000","last_undegraded":"2024-05-10T04:38:12.787640+0000","last_fullsized":"2024-05-10T04:38:12.787640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:37.511400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220978+0000","last_change":"2024-05-10T04:38:12.807478+0000","last_active":"2024-05-10T04:38:13.220978+0000","last_peered":"2024-05-10T04:38:13.220978+0000","last_clean":"2024-05-10T04:38:13.220978+0000","last_became_active":"2024-05-10T04:38:12.806801+0000","last_became_peered":"2024-05-10T04:38:12.806801+0000","last_unstale":"2024-05-10T04:38:13.220978+0000","last_undegraded":"2024-05-10T04:38:13.220978+0000","last_fullsized":"2024-05-10T04:38:13.220978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:32.168836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.943959+0000","last_change":"2024-05-10T04:38:08.343610+0000","last_active":"2024-05-10T04:38:32.943959+0000","last_peered":"2024-05-10T04:38:32.943959+0000","last_clean":"2024-05-10T04:38:32.943959+0000","last_became_active":"2024-05-10T04:38:08.342880+0000","last_became_peered":"2024-05-10T04:38:08.342880+0000","last_unstale":"2024-05-10T04:38:32.943959+0000","last_undegraded":"2024-05-10T04:38:32.943959+0000","last_fullsized":"2024-05-10T04:38:32.943959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:45:11.502258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784342+0000","last_change":"2024-05-10T04:38:08.780354+0000","last_active":"2024-05-10T04:38:12.784342+0000","last_peered":"2024-05-10T04:38:12.784342+0000","last_clean":"2024-05-10T04:38:12.784342+0000","last_became_active":"2024-05-10T04:38:08.779777+0000","last_became_peered":"2024-05-10T04:38:08.779777+0000","last_unstale":"2024-05-10T04:38:12.784342+0000","last_undegraded":"2024-05-10T04:38:12.784342+0000","last_fullsized":"2024-05-10T04:38:12.784342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:28.243474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931246+0000","last_change":"2024-05-10T04:38:12.800759+0000","last_active":"2024-05-10T04:38:13.931246+0000","last_peered":"2024-05-10T04:38:13.931246+0000","last_clean":"2024-05-10T04:38:13.931246+0000","last_became_active":"2024-05-10T04:38:12.800417+0000","last_became_peered":"2024-05-10T04:38:12.800417+0000","last_unstale":"2024-05-10T04:38:13.931246+0000","last_undegraded":"2024-05-10T04:38:13.931246+0000","last_fullsized":"2024-05-10T04:38:13.931246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:04.734817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'33","reported_seq":116,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:01.988113+0000","last_change":"2024-05-10T04:38:08.343873+0000","last_active":"2024-05-10T04:39:01.988113+0000","last_peered":"2024-05-10T04:39:01.988113+0000","last_clean":"2024-05-10T04:39:01.988113+0000","last_became_active":"2024-05-10T04:38:08.343360+0000","last_became_peered":"2024-05-10T04:38:08.343360+0000","last_unstale":"2024-05-10T04:39:01.988113+0000","last_undegraded":"2024-05-10T04:39:01.988113+0000","last_fullsized":"2024-05-10T04:39:01.988113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:06:10.816343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61660,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":94,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156650+0000","last_change":"2024-05-10T04:38:08.783955+0000","last_active":"2024-05-10T04:38:13.156650+0000","last_peered":"2024-05-10T04:38:13.156650+0000","last_clean":"2024-05-10T04:38:13.156650+0000","last_became_active":"2024-05-10T04:38:08.779131+0000","last_became_peered":"2024-05-10T04:38:08.779131+0000","last_unstale":"2024-05-10T04:38:13.156650+0000","last_undegraded":"2024-05-10T04:38:13.156650+0000","last_fullsized":"2024-05-10T04:38:13.156650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:58.619571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930607+0000","last_change":"2024-05-10T04:38:12.795140+0000","last_active":"2024-05-10T04:38:13.930607+0000","last_peered":"2024-05-10T04:38:13.930607+0000","last_clean":"2024-05-10T04:38:13.930607+0000","last_became_active":"2024-05-10T04:38:12.794666+0000","last_became_peered":"2024-05-10T04:38:12.794666+0000","last_unstale":"2024-05-10T04:38:13.930607+0000","last_undegraded":"2024-05-10T04:38:13.930607+0000","last_fullsized":"2024-05-10T04:38:13.930607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:44.468321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.971051+0000","last_change":"2024-05-10T04:38:08.338866+0000","last_active":"2024-05-10T04:38:34.971051+0000","last_peered":"2024-05-10T04:38:34.971051+0000","last_clean":"2024-05-10T04:38:34.971051+0000","last_became_active":"2024-05-10T04:38:08.338434+0000","last_became_peered":"2024-05-10T04:38:08.338434+0000","last_unstale":"2024-05-10T04:38:34.971051+0000","last_undegraded":"2024-05-10T04:38:34.971051+0000","last_fullsized":"2024-05-10T04:38:34.971051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:49.858034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781395+0000","last_change":"2024-05-10T04:38:08.768998+0000","last_active":"2024-05-10T04:38:12.781395+0000","last_peered":"2024-05-10T04:38:12.781395+0000","last_clean":"2024-05-10T04:38:12.781395+0000","last_became_active":"2024-05-10T04:38:08.768687+0000","last_became_peered":"2024-05-10T04:38:08.768687+0000","last_unstale":"2024-05-10T04:38:12.781395+0000","last_undegraded":"2024-05-10T04:38:12.781395+0000","last_fullsized":"2024-05-10T04:38:12.781395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:19.390746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218211+0000","last_change":"2024-05-10T04:38:12.801223+0000","last_active":"2024-05-10T04:38:13.218211+0000","last_peered":"2024-05-10T04:38:13.218211+0000","last_clean":"2024-05-10T04:38:13.218211+0000","last_became_active":"2024-05-10T04:38:12.800750+0000","last_became_peered":"2024-05-10T04:38:12.800750+0000","last_unstale":"2024-05-10T04:38:13.218211+0000","last_undegraded":"2024-05-10T04:38:13.218211+0000","last_fullsized":"2024-05-10T04:38:13.218211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:33.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968494+0000","last_change":"2024-05-10T04:38:08.344579+0000","last_active":"2024-05-10T04:38:34.968494+0000","last_peered":"2024-05-10T04:38:34.968494+0000","last_clean":"2024-05-10T04:38:34.968494+0000","last_became_active":"2024-05-10T04:38:08.344250+0000","last_became_peered":"2024-05-10T04:38:08.344250+0000","last_unstale":"2024-05-10T04:38:34.968494+0000","last_undegraded":"2024-05-10T04:38:34.968494+0000","last_fullsized":"2024-05-10T04:38:34.968494+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:56.491732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786793+0000","last_change":"2024-05-10T04:38:08.784556+0000","last_active":"2024-05-10T04:38:12.786793+0000","last_peered":"2024-05-10T04:38:12.786793+0000","last_clean":"2024-05-10T04:38:12.786793+0000","last_became_active":"2024-05-10T04:38:08.784229+0000","last_became_peered":"2024-05-10T04:38:08.784229+0000","last_unstale":"2024-05-10T04:38:12.786793+0000","last_undegraded":"2024-05-10T04:38:12.786793+0000","last_fullsized":"2024-05-10T04:38:12.786793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:48:46.519837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930516+0000","last_change":"2024-05-10T04:38:12.805795+0000","last_active":"2024-05-10T04:38:13.930516+0000","last_peered":"2024-05-10T04:38:13.930516+0000","last_clean":"2024-05-10T04:38:13.930516+0000","last_became_active":"2024-05-10T04:38:12.805366+0000","last_became_peered":"2024-05-10T04:38:12.805366+0000","last_unstale":"2024-05-10T04:38:13.930516+0000","last_undegraded":"2024-05-10T04:38:13.930516+0000","last_fullsized":"2024-05-10T04:38:13.930516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:38:11.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:02.022159+0000","last_change":"2024-05-10T04:38:08.345396+0000","last_active":"2024-05-10T04:39:02.022159+0000","last_peered":"2024-05-10T04:39:02.022159+0000","last_clean":"2024-05-10T04:39:02.022159+0000","last_became_active":"2024-05-10T04:38:08.345076+0000","last_became_peered":"2024-05-10T04:38:08.345076+0000","last_unstale":"2024-05-10T04:39:02.022159+0000","last_undegraded":"2024-05-10T04:39:02.022159+0000","last_fullsized":"2024-05-10T04:39:02.022159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:38.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787293+0000","last_change":"2024-05-10T04:38:08.791089+0000","last_active":"2024-05-10T04:38:12.787293+0000","last_peered":"2024-05-10T04:38:12.787293+0000","last_clean":"2024-05-10T04:38:12.787293+0000","last_became_active":"2024-05-10T04:38:08.790813+0000","last_became_peered":"2024-05-10T04:38:08.790813+0000","last_unstale":"2024-05-10T04:38:12.787293+0000","last_undegraded":"2024-05-10T04:38:12.787293+0000","last_fullsized":"2024-05-10T04:38:12.787293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:52:45.753196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809384+0000","last_change":"2024-05-10T04:38:12.803896+0000","last_active":"2024-05-10T04:38:12.809384+0000","last_peered":"2024-05-10T04:38:12.809384+0000","last_clean":"2024-05-10T04:38:12.809384+0000","last_became_active":"2024-05-10T04:38:12.803512+0000","last_became_peered":"2024-05-10T04:38:12.803512+0000","last_unstale":"2024-05-10T04:38:12.809384+0000","last_undegraded":"2024-05-10T04:38:12.809384+0000","last_fullsized":"2024-05-10T04:38:12.809384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:01:23.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970681+0000","last_change":"2024-05-10T04:38:08.348187+0000","last_active":"2024-05-10T04:38:34.970681+0000","last_peered":"2024-05-10T04:38:34.970681+0000","last_clean":"2024-05-10T04:38:34.970681+0000","last_became_active":"2024-05-10T04:38:08.346986+0000","last_became_peered":"2024-05-10T04:38:08.346986+0000","last_unstale":"2024-05-10T04:38:34.970681+0000","last_undegraded":"2024-05-10T04:38:34.970681+0000","last_fullsized":"2024-05-10T04:38:34.970681+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:46.245042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780738+0000","last_change":"2024-05-10T04:38:08.782905+0000","last_active":"2024-05-10T04:38:12.780738+0000","last_peered":"2024-05-10T04:38:12.780738+0000","last_clean":"2024-05-10T04:38:12.780738+0000","last_became_active":"2024-05-10T04:38:08.782508+0000","last_became_peered":"2024-05-10T04:38:08.782508+0000","last_unstale":"2024-05-10T04:38:12.780738+0000","last_undegraded":"2024-05-10T04:38:12.780738+0000","last_fullsized":"2024-05-10T04:38:12.780738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:29.026282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930246+0000","last_change":"2024-05-10T04:38:12.796046+0000","last_active":"2024-05-10T04:38:13.930246+0000","last_peered":"2024-05-10T04:38:13.930246+0000","last_clean":"2024-05-10T04:38:13.930246+0000","last_became_active":"2024-05-10T04:38:12.795635+0000","last_became_peered":"2024-05-10T04:38:12.795635+0000","last_unstale":"2024-05-10T04:38:13.930246+0000","last_undegraded":"2024-05-10T04:38:13.930246+0000","last_fullsized":"2024-05-10T04:38:13.930246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:50.822404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.941305+0000","last_change":"2024-05-10T04:38:08.348701+0000","last_active":"2024-05-10T04:38:32.941305+0000","last_peered":"2024-05-10T04:38:32.941305+0000","last_clean":"2024-05-10T04:38:32.941305+0000","last_became_active":"2024-05-10T04:38:08.348345+0000","last_became_peered":"2024-05-10T04:38:08.348345+0000","last_unstale":"2024-05-10T04:38:32.941305+0000","last_undegraded":"2024-05-10T04:38:32.941305+0000","last_fullsized":"2024-05-10T04:38:32.941305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:31.505996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787204+0000","last_change":"2024-05-10T04:38:08.793070+0000","last_active":"2024-05-10T04:38:12.787204+0000","last_peered":"2024-05-10T04:38:12.787204+0000","last_clean":"2024-05-10T04:38:12.787204+0000","last_became_active":"2024-05-10T04:38:08.792694+0000","last_became_peered":"2024-05-10T04:38:08.792694+0000","last_unstale":"2024-05-10T04:38:12.787204+0000","last_undegraded":"2024-05-10T04:38:12.787204+0000","last_fullsized":"2024-05-10T04:38:12.787204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:23.636222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156578+0000","last_change":"2024-05-10T04:38:12.809215+0000","last_active":"2024-05-10T04:38:13.156578+0000","last_peered":"2024-05-10T04:38:13.156578+0000","last_clean":"2024-05-10T04:38:13.156578+0000","last_became_active":"2024-05-10T04:38:12.808717+0000","last_became_peered":"2024-05-10T04:38:12.808717+0000","last_unstale":"2024-05-10T04:38:13.156578+0000","last_undegraded":"2024-05-10T04:38:13.156578+0000","last_fullsized":"2024-05-10T04:38:13.156578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:40.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931223+0000","last_change":"2024-05-10T04:38:08.347770+0000","last_active":"2024-05-10T04:38:13.931223+0000","last_peered":"2024-05-10T04:38:13.931223+0000","last_clean":"2024-05-10T04:38:13.931223+0000","last_became_active":"2024-05-10T04:38:08.347444+0000","last_became_peered":"2024-05-10T04:38:08.347444+0000","last_unstale":"2024-05-10T04:38:13.931223+0000","last_undegraded":"2024-05-10T04:38:13.931223+0000","last_fullsized":"2024-05-10T04:38:13.931223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:49.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809464+0000","last_change":"2024-05-10T04:38:08.765358+0000","last_active":"2024-05-10T04:38:12.809464+0000","last_peered":"2024-05-10T04:38:12.809464+0000","last_clean":"2024-05-10T04:38:12.809464+0000","last_became_active":"2024-05-10T04:38:08.765090+0000","last_became_peered":"2024-05-10T04:38:08.765090+0000","last_unstale":"2024-05-10T04:38:12.809464+0000","last_undegraded":"2024-05-10T04:38:12.809464+0000","last_fullsized":"2024-05-10T04:38:12.809464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:55:25.722331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.798587+0000","last_change":"2024-05-10T04:38:12.794765+0000","last_active":"2024-05-10T04:38:12.798587+0000","last_peered":"2024-05-10T04:38:12.798587+0000","last_clean":"2024-05-10T04:38:12.798587+0000","last_became_active":"2024-05-10T04:38:12.794400+0000","last_became_peered":"2024-05-10T04:38:12.794400+0000","last_unstale":"2024-05-10T04:38:12.798587+0000","last_undegraded":"2024-05-10T04:38:12.798587+0000","last_fullsized":"2024-05-10T04:38:12.798587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:15:16.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:03.023719+0000","last_change":"2024-05-10T04:38:08.348812+0000","last_active":"2024-05-10T04:39:03.023719+0000","last_peered":"2024-05-10T04:39:03.023719+0000","last_clean":"2024-05-10T04:39:03.023719+0000","last_became_active":"2024-05-10T04:38:08.348207+0000","last_became_peered":"2024-05-10T04:38:08.348207+0000","last_unstale":"2024-05-10T04:39:03.023719+0000","last_undegraded":"2024-05-10T04:39:03.023719+0000","last_fullsized":"2024-05-10T04:39:03.023719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:17.350094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809431+0000","last_change":"2024-05-10T04:38:08.785502+0000","last_active":"2024-05-10T04:38:12.809431+0000","last_peered":"2024-05-10T04:38:12.809431+0000","last_clean":"2024-05-10T04:38:12.809431+0000","last_became_active":"2024-05-10T04:38:08.785073+0000","last_became_peered":"2024-05-10T04:38:08.785073+0000","last_unstale":"2024-05-10T04:38:12.809431+0000","last_undegraded":"2024-05-10T04:38:12.809431+0000","last_fullsized":"2024-05-10T04:38:12.809431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:08:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.810257+0000","last_change":"2024-05-10T04:38:12.809599+0000","last_active":"2024-05-10T04:38:12.810257+0000","last_peered":"2024-05-10T04:38:12.810257+0000","last_clean":"2024-05-10T04:38:12.810257+0000","last_became_active":"2024-05-10T04:38:12.809200+0000","last_became_peered":"2024-05-10T04:38:12.809200+0000","last_unstale":"2024-05-10T04:38:12.810257+0000","last_undegraded":"2024-05-10T04:38:12.810257+0000","last_fullsized":"2024-05-10T04:38:12.810257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:54.940517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781249+0000","last_change":"2024-05-10T04:38:08.779837+0000","last_active":"2024-05-10T04:38:12.781249+0000","last_peered":"2024-05-10T04:38:12.781249+0000","last_clean":"2024-05-10T04:38:12.781249+0000","last_became_active":"2024-05-10T04:38:08.779501+0000","last_became_peered":"2024-05-10T04:38:08.779501+0000","last_unstale":"2024-05-10T04:38:12.781249+0000","last_undegraded":"2024-05-10T04:38:12.781249+0000","last_fullsized":"2024-05-10T04:38:12.781249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:00.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:00.018997+0000","last_change":"2024-05-10T04:38:08.343649+0000","last_active":"2024-05-10T04:39:00.018997+0000","last_peered":"2024-05-10T04:39:00.018997+0000","last_clean":"2024-05-10T04:39:00.018997+0000","last_became_active":"2024-05-10T04:38:08.343292+0000","last_became_peered":"2024-05-10T04:38:08.343292+0000","last_unstale":"2024-05-10T04:39:00.018997+0000","last_undegraded":"2024-05-10T04:39:00.018997+0000","last_fullsized":"2024-05-10T04:39:00.018997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:53.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:14.390654+0000","last_change":"2024-05-10T04:38:12.796552+0000","last_active":"2024-05-10T04:38:14.390654+0000","last_peered":"2024-05-10T04:38:14.390654+0000","last_clean":"2024-05-10T04:38:14.390654+0000","last_became_active":"2024-05-10T04:38:12.796151+0000","last_became_peered":"2024-05-10T04:38:12.796151+0000","last_unstale":"2024-05-10T04:38:14.390654+0000","last_undegraded":"2024-05-10T04:38:14.390654+0000","last_fullsized":"2024-05-10T04:38:14.390654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:05.520859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787088+0000","last_change":"2024-05-10T04:38:08.770957+0000","last_active":"2024-05-10T04:38:12.787088+0000","last_peered":"2024-05-10T04:38:12.787088+0000","last_clean":"2024-05-10T04:38:12.787088+0000","last_became_active":"2024-05-10T04:38:08.770572+0000","last_became_peered":"2024-05-10T04:38:08.770572+0000","last_unstale":"2024-05-10T04:38:12.787088+0000","last_undegraded":"2024-05-10T04:38:12.787088+0000","last_fullsized":"2024-05-10T04:38:12.787088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:21.421033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.985828+0000","last_change":"2024-05-10T04:38:08.340788+0000","last_active":"2024-05-10T04:38:35.985828+0000","last_peered":"2024-05-10T04:38:35.985828+0000","last_clean":"2024-05-10T04:38:35.985828+0000","last_became_active":"2024-05-10T04:38:08.340407+0000","last_became_peered":"2024-05-10T04:38:08.340407+0000","last_unstale":"2024-05-10T04:38:35.985828+0000","last_undegraded":"2024-05-10T04:38:35.985828+0000","last_fullsized":"2024-05-10T04:38:35.985828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:30:05.276951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930055+0000","last_change":"2024-05-10T04:37:20.961282+0000","last_active":"2024-05-10T04:38:13.930055+0000","last_peered":"2024-05-10T04:38:13.930055+0000","last_clean":"2024-05-10T04:38:13.930055+0000","last_became_active":"2024-05-10T04:37:20.960828+0000","last_became_peered":"2024-05-10T04:37:20.960828+0000","last_unstale":"2024-05-10T04:38:13.930055+0000","last_undegraded":"2024-05-10T04:38:13.930055+0000","last_fullsized":"2024-05-10T04:38:13.930055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_clean_scrub_stamp":"2024-05-10T04:35:00.248380+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:08:04.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930338+0000","last_change":"2024-05-10T04:38:12.809899+0000","last_active":"2024-05-10T04:38:13.930338+0000","last_peered":"2024-05-10T04:38:13.930338+0000","last_clean":"2024-05-10T04:38:13.930338+0000","last_became_active":"2024-05-10T04:38:12.809441+0000","last_became_peered":"2024-05-10T04:38:12.809441+0000","last_unstale":"2024-05-10T04:38:13.930338+0000","last_undegraded":"2024-05-10T04:38:13.930338+0000","last_fullsized":"2024-05-10T04:38:13.930338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:48.378890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781228+0000","last_change":"2024-05-10T04:38:08.784838+0000","last_active":"2024-05-10T04:38:12.781228+0000","last_peered":"2024-05-10T04:38:12.781228+0000","last_clean":"2024-05-10T04:38:12.781228+0000","last_became_active":"2024-05-10T04:38:08.784464+0000","last_became_peered":"2024-05-10T04:38:08.784464+0000","last_unstale":"2024-05-10T04:38:12.781228+0000","last_undegraded":"2024-05-10T04:38:12.781228+0000","last_fullsized":"2024-05-10T04:38:12.781228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:00.335350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.953106+0000","last_change":"2024-05-10T04:38:08.348639+0000","last_active":"2024-05-10T04:38:32.953106+0000","last_peered":"2024-05-10T04:38:32.953106+0000","last_clean":"2024-05-10T04:38:32.953106+0000","last_became_active":"2024-05-10T04:38:08.348271+0000","last_became_peered":"2024-05-10T04:38:08.348271+0000","last_unstale":"2024-05-10T04:38:32.953106+0000","last_undegraded":"2024-05-10T04:38:32.953106+0000","last_fullsized":"2024-05-10T04:38:32.953106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:18.486150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218189+0000","last_change":"2024-05-10T04:38:12.801079+0000","last_active":"2024-05-10T04:38:13.218189+0000","last_peered":"2024-05-10T04:38:13.218189+0000","last_clean":"2024-05-10T04:38:13.218189+0000","last_became_active":"2024-05-10T04:38:12.800630+0000","last_became_peered":"2024-05-10T04:38:12.800630+0000","last_unstale":"2024-05-10T04:38:13.218189+0000","last_undegraded":"2024-05-10T04:38:13.218189+0000","last_fullsized":"2024-05-10T04:38:13.218189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:05.213620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.964694+0000","last_change":"2024-05-10T04:38:08.348764+0000","last_active":"2024-05-10T04:38:33.964694+0000","last_peered":"2024-05-10T04:38:33.964694+0000","last_clean":"2024-05-10T04:38:33.964694+0000","last_became_active":"2024-05-10T04:38:08.347939+0000","last_became_peered":"2024-05-10T04:38:08.347939+0000","last_unstale":"2024-05-10T04:38:33.964694+0000","last_undegraded":"2024-05-10T04:38:33.964694+0000","last_fullsized":"2024-05-10T04:38:33.964694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:01.308255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782312+0000","last_change":"2024-05-10T04:38:08.771382+0000","last_active":"2024-05-10T04:38:12.782312+0000","last_peered":"2024-05-10T04:38:12.782312+0000","last_clean":"2024-05-10T04:38:12.782312+0000","last_became_active":"2024-05-10T04:38:08.771108+0000","last_became_peered":"2024-05-10T04:38:08.771108+0000","last_unstale":"2024-05-10T04:38:12.782312+0000","last_undegraded":"2024-05-10T04:38:12.782312+0000","last_fullsized":"2024-05-10T04:38:12.782312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:32.664640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217051+0000","last_change":"2024-05-10T04:38:12.809582+0000","last_active":"2024-05-10T04:38:13.217051+0000","last_peered":"2024-05-10T04:38:13.217051+0000","last_clean":"2024-05-10T04:38:13.217051+0000","last_became_active":"2024-05-10T04:38:12.809134+0000","last_became_peered":"2024-05-10T04:38:12.809134+0000","last_unstale":"2024-05-10T04:38:13.217051+0000","last_undegraded":"2024-05-10T04:38:13.217051+0000","last_fullsized":"2024-05-10T04:38:13.217051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:31:07.694368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:37.782703+0000","last_change":"2024-05-10T04:38:08.345903+0000","last_active":"2024-05-10T04:38:37.782703+0000","last_peered":"2024-05-10T04:38:37.782703+0000","last_clean":"2024-05-10T04:38:37.782703+0000","last_became_active":"2024-05-10T04:38:08.345371+0000","last_became_peered":"2024-05-10T04:38:08.345371+0000","last_unstale":"2024-05-10T04:38:37.782703+0000","last_undegraded":"2024-05-10T04:38:37.782703+0000","last_fullsized":"2024-05-10T04:38:37.782703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:03.581112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789703+0000","last_change":"2024-05-10T04:38:08.779853+0000","last_active":"2024-05-10T04:38:12.789703+0000","last_peered":"2024-05-10T04:38:12.789703+0000","last_clean":"2024-05-10T04:38:12.789703+0000","last_became_active":"2024-05-10T04:38:08.779531+0000","last_became_peered":"2024-05-10T04:38:08.779531+0000","last_unstale":"2024-05-10T04:38:12.789703+0000","last_undegraded":"2024-05-10T04:38:12.789703+0000","last_fullsized":"2024-05-10T04:38:12.789703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:05.079540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410643+0000","last_change":"2024-05-10T04:38:12.795970+0000","last_active":"2024-05-10T04:38:13.410643+0000","last_peered":"2024-05-10T04:38:13.410643+0000","last_clean":"2024-05-10T04:38:13.410643+0000","last_became_active":"2024-05-10T04:38:12.795497+0000","last_became_peered":"2024-05-10T04:38:12.795497+0000","last_unstale":"2024-05-10T04:38:13.410643+0000","last_undegraded":"2024-05-10T04:38:13.410643+0000","last_fullsized":"2024-05-10T04:38:13.410643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:11.174672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970885+0000","last_change":"2024-05-10T04:38:08.343268+0000","last_active":"2024-05-10T04:38:34.970885+0000","last_peered":"2024-05-10T04:38:34.970885+0000","last_clean":"2024-05-10T04:38:34.970885+0000","last_became_active":"2024-05-10T04:38:08.342952+0000","last_became_peered":"2024-05-10T04:38:08.342952+0000","last_unstale":"2024-05-10T04:38:34.970885+0000","last_undegraded":"2024-05-10T04:38:34.970885+0000","last_fullsized":"2024-05-10T04:38:34.970885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:49.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788390+0000","last_change":"2024-05-10T04:38:08.784276+0000","last_active":"2024-05-10T04:38:12.788390+0000","last_peered":"2024-05-10T04:38:12.788390+0000","last_clean":"2024-05-10T04:38:12.788390+0000","last_became_active":"2024-05-10T04:38:08.783951+0000","last_became_peered":"2024-05-10T04:38:08.783951+0000","last_unstale":"2024-05-10T04:38:12.788390+0000","last_undegraded":"2024-05-10T04:38:12.788390+0000","last_fullsized":"2024-05-10T04:38:12.788390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:45:43.992567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220551+0000","last_change":"2024-05-10T04:38:12.811689+0000","last_active":"2024-05-10T04:38:13.220551+0000","last_peered":"2024-05-10T04:38:13.220551+0000","last_clean":"2024-05-10T04:38:13.220551+0000","last_became_active":"2024-05-10T04:38:12.811138+0000","last_became_peered":"2024-05-10T04:38:12.811138+0000","last_unstale":"2024-05-10T04:38:13.220551+0000","last_undegraded":"2024-05-10T04:38:13.220551+0000","last_fullsized":"2024-05-10T04:38:13.220551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:09.291137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410146+0000","last_change":"2024-05-10T04:38:08.348738+0000","last_active":"2024-05-10T04:38:32.410146+0000","last_peered":"2024-05-10T04:38:32.410146+0000","last_clean":"2024-05-10T04:38:32.410146+0000","last_became_active":"2024-05-10T04:38:08.348382+0000","last_became_peered":"2024-05-10T04:38:08.348382+0000","last_unstale":"2024-05-10T04:38:32.410146+0000","last_undegraded":"2024-05-10T04:38:32.410146+0000","last_fullsized":"2024-05-10T04:38:32.410146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:02.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156551+0000","last_change":"2024-05-10T04:38:08.781483+0000","last_active":"2024-05-10T04:38:13.156551+0000","last_peered":"2024-05-10T04:38:13.156551+0000","last_clean":"2024-05-10T04:38:13.156551+0000","last_became_active":"2024-05-10T04:38:08.777713+0000","last_became_peered":"2024-05-10T04:38:08.777713+0000","last_unstale":"2024-05-10T04:38:13.156551+0000","last_undegraded":"2024-05-10T04:38:13.156551+0000","last_fullsized":"2024-05-10T04:38:13.156551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:41.050901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930627+0000","last_change":"2024-05-10T04:38:12.800656+0000","last_active":"2024-05-10T04:38:13.930627+0000","last_peered":"2024-05-10T04:38:13.930627+0000","last_clean":"2024-05-10T04:38:13.930627+0000","last_became_active":"2024-05-10T04:38:12.800191+0000","last_became_peered":"2024-05-10T04:38:12.800191+0000","last_unstale":"2024-05-10T04:38:13.930627+0000","last_undegraded":"2024-05-10T04:38:13.930627+0000","last_fullsized":"2024-05-10T04:38:13.930627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:47.017642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968707+0000","last_change":"2024-05-10T04:38:08.344017+0000","last_active":"2024-05-10T04:38:34.968707+0000","last_peered":"2024-05-10T04:38:34.968707+0000","last_clean":"2024-05-10T04:38:34.968707+0000","last_became_active":"2024-05-10T04:38:08.343646+0000","last_became_peered":"2024-05-10T04:38:08.343646+0000","last_unstale":"2024-05-10T04:38:34.968707+0000","last_undegraded":"2024-05-10T04:38:34.968707+0000","last_fullsized":"2024-05-10T04:38:34.968707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:02:30.455019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786221+0000","last_change":"2024-05-10T04:38:08.779065+0000","last_active":"2024-05-10T04:38:12.786221+0000","last_peered":"2024-05-10T04:38:12.786221+0000","last_clean":"2024-05-10T04:38:12.786221+0000","last_became_active":"2024-05-10T04:38:08.778685+0000","last_became_peered":"2024-05-10T04:38:08.778685+0000","last_unstale":"2024-05-10T04:38:12.786221+0000","last_undegraded":"2024-05-10T04:38:12.786221+0000","last_fullsized":"2024-05-10T04:38:12.786221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:25:58.780080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.353532+0000","last_change":"2024-05-10T04:38:12.801524+0000","last_active":"2024-05-10T04:38:13.353532+0000","last_peered":"2024-05-10T04:38:13.353532+0000","last_clean":"2024-05-10T04:38:13.353532+0000","last_became_active":"2024-05-10T04:38:12.800777+0000","last_became_peered":"2024-05-10T04:38:12.800777+0000","last_unstale":"2024-05-10T04:38:13.353532+0000","last_undegraded":"2024-05-10T04:38:13.353532+0000","last_fullsized":"2024-05-10T04:38:13.353532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:14.139236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.277600+0000","last_change":"2024-05-10T04:38:08.344785+0000","last_active":"2024-05-10T04:38:35.277600+0000","last_peered":"2024-05-10T04:38:35.277600+0000","last_clean":"2024-05-10T04:38:35.277600+0000","last_became_active":"2024-05-10T04:38:08.344337+0000","last_became_peered":"2024-05-10T04:38:08.344337+0000","last_unstale":"2024-05-10T04:38:35.277600+0000","last_undegraded":"2024-05-10T04:38:35.277600+0000","last_fullsized":"2024-05-10T04:38:35.277600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:47.682532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788214+0000","last_change":"2024-05-10T04:38:08.768686+0000","last_active":"2024-05-10T04:38:12.788214+0000","last_peered":"2024-05-10T04:38:12.788214+0000","last_clean":"2024-05-10T04:38:12.788214+0000","last_became_active":"2024-05-10T04:38:08.768317+0000","last_became_peered":"2024-05-10T04:38:08.768317+0000","last_unstale":"2024-05-10T04:38:12.788214+0000","last_undegraded":"2024-05-10T04:38:12.788214+0000","last_fullsized":"2024-05-10T04:38:12.788214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:42:33.670912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220479+0000","last_change":"2024-05-10T04:38:12.809825+0000","last_active":"2024-05-10T04:38:13.220479+0000","last_peered":"2024-05-10T04:38:13.220479+0000","last_clean":"2024-05-10T04:38:13.220479+0000","last_became_active":"2024-05-10T04:38:12.809378+0000","last_became_peered":"2024-05-10T04:38:12.809378+0000","last_unstale":"2024-05-10T04:38:13.220479+0000","last_undegraded":"2024-05-10T04:38:13.220479+0000","last_fullsized":"2024-05-10T04:38:13.220479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:28.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410164+0000","last_change":"2024-05-10T04:38:08.347871+0000","last_active":"2024-05-10T04:38:32.410164+0000","last_peered":"2024-05-10T04:38:32.410164+0000","last_clean":"2024-05-10T04:38:32.410164+0000","last_became_active":"2024-05-10T04:38:08.347618+0000","last_became_peered":"2024-05-10T04:38:08.347618+0000","last_unstale":"2024-05-10T04:38:32.410164+0000","last_undegraded":"2024-05-10T04:38:32.410164+0000","last_fullsized":"2024-05-10T04:38:32.410164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:09:42.060082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787115+0000","last_change":"2024-05-10T04:38:08.779191+0000","last_active":"2024-05-10T04:38:12.787115+0000","last_peered":"2024-05-10T04:38:12.787115+0000","last_clean":"2024-05-10T04:38:12.787115+0000","last_became_active":"2024-05-10T04:38:08.778829+0000","last_became_peered":"2024-05-10T04:38:08.778829+0000","last_unstale":"2024-05-10T04:38:12.787115+0000","last_undegraded":"2024-05-10T04:38:12.787115+0000","last_fullsized":"2024-05-10T04:38:12.787115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:07.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931626+0000","last_change":"2024-05-10T04:38:12.788693+0000","last_active":"2024-05-10T04:38:13.931626+0000","last_peered":"2024-05-10T04:38:13.931626+0000","last_clean":"2024-05-10T04:38:13.931626+0000","last_became_active":"2024-05-10T04:38:12.788156+0000","last_became_peered":"2024-05-10T04:38:12.788156+0000","last_unstale":"2024-05-10T04:38:13.931626+0000","last_undegraded":"2024-05-10T04:38:13.931626+0000","last_fullsized":"2024-05-10T04:38:13.931626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:30.119523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:02.033680+0000","last_change":"2024-05-10T04:38:08.344211+0000","last_active":"2024-05-10T04:39:02.033680+0000","last_peered":"2024-05-10T04:39:02.033680+0000","last_clean":"2024-05-10T04:39:02.033680+0000","last_became_active":"2024-05-10T04:38:08.343443+0000","last_became_peered":"2024-05-10T04:38:08.343443+0000","last_unstale":"2024-05-10T04:39:02.033680+0000","last_undegraded":"2024-05-10T04:39:02.033680+0000","last_fullsized":"2024-05-10T04:39:02.033680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:33:06.960942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787186+0000","last_change":"2024-05-10T04:38:08.780352+0000","last_active":"2024-05-10T04:38:12.787186+0000","last_peered":"2024-05-10T04:38:12.787186+0000","last_clean":"2024-05-10T04:38:12.787186+0000","last_became_active":"2024-05-10T04:38:08.779779+0000","last_became_peered":"2024-05-10T04:38:08.779779+0000","last_unstale":"2024-05-10T04:38:12.787186+0000","last_undegraded":"2024-05-10T04:38:12.787186+0000","last_fullsized":"2024-05-10T04:38:12.787186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:54:31.973896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.340967+0000","last_change":"2024-05-10T04:38:12.805310+0000","last_active":"2024-05-10T04:38:13.340967+0000","last_peered":"2024-05-10T04:38:13.340967+0000","last_clean":"2024-05-10T04:38:13.340967+0000","last_became_active":"2024-05-10T04:38:12.804827+0000","last_became_peered":"2024-05-10T04:38:12.804827+0000","last_unstale":"2024-05-10T04:38:13.340967+0000","last_undegraded":"2024-05-10T04:38:13.340967+0000","last_fullsized":"2024-05-10T04:38:13.340967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:01.655788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963212+0000","last_change":"2024-05-10T04:38:08.343813+0000","last_active":"2024-05-10T04:38:33.963212+0000","last_peered":"2024-05-10T04:38:33.963212+0000","last_clean":"2024-05-10T04:38:33.963212+0000","last_became_active":"2024-05-10T04:38:08.343309+0000","last_became_peered":"2024-05-10T04:38:08.343309+0000","last_unstale":"2024-05-10T04:38:33.963212+0000","last_undegraded":"2024-05-10T04:38:33.963212+0000","last_fullsized":"2024-05-10T04:38:33.963212+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:07.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786851+0000","last_change":"2024-05-10T04:38:08.780598+0000","last_active":"2024-05-10T04:38:12.786851+0000","last_peered":"2024-05-10T04:38:12.786851+0000","last_clean":"2024-05-10T04:38:12.786851+0000","last_became_active":"2024-05-10T04:38:08.775367+0000","last_became_peered":"2024-05-10T04:38:08.775367+0000","last_unstale":"2024-05-10T04:38:12.786851+0000","last_undegraded":"2024-05-10T04:38:12.786851+0000","last_fullsized":"2024-05-10T04:38:12.786851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:33.826116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.929645+0000","last_change":"2024-05-10T04:38:12.795975+0000","last_active":"2024-05-10T04:38:13.929645+0000","last_peered":"2024-05-10T04:38:13.929645+0000","last_clean":"2024-05-10T04:38:13.929645+0000","last_became_active":"2024-05-10T04:38:12.795582+0000","last_became_peered":"2024-05-10T04:38:12.795582+0000","last_unstale":"2024-05-10T04:38:13.929645+0000","last_undegraded":"2024-05-10T04:38:13.929645+0000","last_fullsized":"2024-05-10T04:38:13.929645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:42:30.734833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.969664+0000","last_change":"2024-05-10T04:38:08.346338+0000","last_active":"2024-05-10T04:38:34.969664+0000","last_peered":"2024-05-10T04:38:34.969664+0000","last_clean":"2024-05-10T04:38:34.969664+0000","last_became_active":"2024-05-10T04:38:08.345994+0000","last_became_peered":"2024-05-10T04:38:08.345994+0000","last_unstale":"2024-05-10T04:38:34.969664+0000","last_undegraded":"2024-05-10T04:38:34.969664+0000","last_fullsized":"2024-05-10T04:38:34.969664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:22.962514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787140+0000","last_change":"2024-05-10T04:38:08.780925+0000","last_active":"2024-05-10T04:38:12.787140+0000","last_peered":"2024-05-10T04:38:12.787140+0000","last_clean":"2024-05-10T04:38:12.787140+0000","last_became_active":"2024-05-10T04:38:08.778777+0000","last_became_peered":"2024-05-10T04:38:08.778777+0000","last_unstale":"2024-05-10T04:38:12.787140+0000","last_undegraded":"2024-05-10T04:38:12.787140+0000","last_fullsized":"2024-05-10T04:38:12.787140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:57.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786327+0000","last_change":"2024-05-10T04:38:08.780281+0000","last_active":"2024-05-10T04:38:12.786327+0000","last_peered":"2024-05-10T04:38:12.786327+0000","last_clean":"2024-05-10T04:38:12.786327+0000","last_became_active":"2024-05-10T04:38:08.779927+0000","last_became_peered":"2024-05-10T04:38:08.779927+0000","last_unstale":"2024-05-10T04:38:12.786327+0000","last_undegraded":"2024-05-10T04:38:12.786327+0000","last_fullsized":"2024-05-10T04:38:12.786327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:33.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789688+0000","last_change":"2024-05-10T04:38:08.784916+0000","last_active":"2024-05-10T04:38:12.789688+0000","last_peered":"2024-05-10T04:38:12.789688+0000","last_clean":"2024-05-10T04:38:12.789688+0000","last_became_active":"2024-05-10T04:38:08.784544+0000","last_became_peered":"2024-05-10T04:38:08.784544+0000","last_unstale":"2024-05-10T04:38:12.789688+0000","last_undegraded":"2024-05-10T04:38:12.789688+0000","last_fullsized":"2024-05-10T04:38:12.789688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:03.669897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786689+0000","last_change":"2024-05-10T04:38:08.769149+0000","last_active":"2024-05-10T04:38:12.786689+0000","last_peered":"2024-05-10T04:38:12.786689+0000","last_clean":"2024-05-10T04:38:12.786689+0000","last_became_active":"2024-05-10T04:38:08.768808+0000","last_became_peered":"2024-05-10T04:38:08.768808+0000","last_unstale":"2024-05-10T04:38:12.786689+0000","last_undegraded":"2024-05-10T04:38:12.786689+0000","last_fullsized":"2024-05-10T04:38:12.786689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:42:46.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787125+0000","last_change":"2024-05-10T04:38:08.784411+0000","last_active":"2024-05-10T04:38:12.787125+0000","last_peered":"2024-05-10T04:38:12.787125+0000","last_clean":"2024-05-10T04:38:12.787125+0000","last_became_active":"2024-05-10T04:38:08.784064+0000","last_became_peered":"2024-05-10T04:38:08.784064+0000","last_unstale":"2024-05-10T04:38:12.787125+0000","last_undegraded":"2024-05-10T04:38:12.787125+0000","last_fullsized":"2024-05-10T04:38:12.787125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:32:46.493888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782409+0000","last_change":"2024-05-10T04:38:08.779643+0000","last_active":"2024-05-10T04:38:12.782409+0000","last_peered":"2024-05-10T04:38:12.782409+0000","last_clean":"2024-05-10T04:38:12.782409+0000","last_became_active":"2024-05-10T04:38:08.779291+0000","last_became_peered":"2024-05-10T04:38:08.779291+0000","last_unstale":"2024-05-10T04:38:12.782409+0000","last_undegraded":"2024-05-10T04:38:12.782409+0000","last_fullsized":"2024-05-10T04:38:12.782409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:24.535515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786192+0000","last_change":"2024-05-10T04:38:08.789382+0000","last_active":"2024-05-10T04:38:12.786192+0000","last_peered":"2024-05-10T04:38:12.786192+0000","last_clean":"2024-05-10T04:38:12.786192+0000","last_became_active":"2024-05-10T04:38:08.789003+0000","last_became_peered":"2024-05-10T04:38:08.789003+0000","last_unstale":"2024-05-10T04:38:12.786192+0000","last_undegraded":"2024-05-10T04:38:12.786192+0000","last_fullsized":"2024-05-10T04:38:12.786192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:03.031319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788582+0000","last_change":"2024-05-10T04:38:08.777290+0000","last_active":"2024-05-10T04:38:12.788582+0000","last_peered":"2024-05-10T04:38:12.788582+0000","last_clean":"2024-05-10T04:38:12.788582+0000","last_became_active":"2024-05-10T04:38:08.776331+0000","last_became_peered":"2024-05-10T04:38:08.776331+0000","last_unstale":"2024-05-10T04:38:12.788582+0000","last_undegraded":"2024-05-10T04:38:12.788582+0000","last_fullsized":"2024-05-10T04:38:12.788582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:51.074991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789825+0000","last_change":"2024-05-10T04:38:08.780937+0000","last_active":"2024-05-10T04:38:12.789825+0000","last_peered":"2024-05-10T04:38:12.789825+0000","last_clean":"2024-05-10T04:38:12.789825+0000","last_became_active":"2024-05-10T04:38:08.780620+0000","last_became_peered":"2024-05-10T04:38:08.780620+0000","last_unstale":"2024-05-10T04:38:12.789825+0000","last_undegraded":"2024-05-10T04:38:12.789825+0000","last_fullsized":"2024-05-10T04:38:12.789825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:27.697571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781208+0000","last_change":"2024-05-10T04:38:08.773314+0000","last_active":"2024-05-10T04:38:12.781208+0000","last_peered":"2024-05-10T04:38:12.781208+0000","last_clean":"2024-05-10T04:38:12.781208+0000","last_became_active":"2024-05-10T04:38:08.772973+0000","last_became_peered":"2024-05-10T04:38:08.772973+0000","last_unstale":"2024-05-10T04:38:12.781208+0000","last_undegraded":"2024-05-10T04:38:12.781208+0000","last_fullsized":"2024-05-10T04:38:12.781208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:53.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788890+0000","last_change":"2024-05-10T04:38:08.775632+0000","last_active":"2024-05-10T04:38:12.788890+0000","last_peered":"2024-05-10T04:38:12.788890+0000","last_clean":"2024-05-10T04:38:12.788890+0000","last_became_active":"2024-05-10T04:38:08.775308+0000","last_became_peered":"2024-05-10T04:38:08.775308+0000","last_unstale":"2024-05-10T04:38:12.788890+0000","last_undegraded":"2024-05-10T04:38:12.788890+0000","last_fullsized":"2024-05-10T04:38:12.788890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:58.758863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783887+0000","last_change":"2024-05-10T04:38:08.787354+0000","last_active":"2024-05-10T04:38:12.783887+0000","last_peered":"2024-05-10T04:38:12.783887+0000","last_clean":"2024-05-10T04:38:12.783887+0000","last_became_active":"2024-05-10T04:38:08.786645+0000","last_became_peered":"2024-05-10T04:38:08.786645+0000","last_unstale":"2024-05-10T04:38:12.783887+0000","last_undegraded":"2024-05-10T04:38:12.783887+0000","last_fullsized":"2024-05-10T04:38:12.783887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:35.060271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782372+0000","last_change":"2024-05-10T04:38:08.769233+0000","last_active":"2024-05-10T04:38:12.782372+0000","last_peered":"2024-05-10T04:38:12.782372+0000","last_clean":"2024-05-10T04:38:12.782372+0000","last_became_active":"2024-05-10T04:38:08.768936+0000","last_became_peered":"2024-05-10T04:38:08.768936+0000","last_unstale":"2024-05-10T04:38:12.782372+0000","last_undegraded":"2024-05-10T04:38:12.782372+0000","last_fullsized":"2024-05-10T04:38:12.782372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:08.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786960+0000","last_change":"2024-05-10T04:38:08.788189+0000","last_active":"2024-05-10T04:38:12.786960+0000","last_peered":"2024-05-10T04:38:12.786960+0000","last_clean":"2024-05-10T04:38:12.786960+0000","last_became_active":"2024-05-10T04:38:08.787806+0000","last_became_peered":"2024-05-10T04:38:08.787806+0000","last_unstale":"2024-05-10T04:38:12.786960+0000","last_undegraded":"2024-05-10T04:38:12.786960+0000","last_fullsized":"2024-05-10T04:38:12.786960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:00:51.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280050,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68283,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":323,"num_read_kb":232,"num_write":233,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":206501,"data_compressed":4791,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481813,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":942080,"data_stored":466157,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.629}]},{"osd":7,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.56499999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":9,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.89200000000000002}]},{"osd":10,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.70999999999999996}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":1113909,"data_compressed":48903,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91500000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.89800000000000002}]},{"osd":9,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.88400000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906972,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":469838,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.748}]},{"osd":3,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84199999999999997}]},{"osd":6,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.76000000000000001}]},{"osd":8,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.91500000000000004}]},{"osd":10,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":962560,"data_stored":466120,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.497,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.66900000000000004,"15min":0.64400000000000002},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.59399999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.86399999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.70199999999999996},"min":{"1min":0.59099999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.67400000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.877}]},{"osd":7,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":0.96099999999999997}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":1.04}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364835,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1062082,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.532,"5min":3.532,"15min":3.532},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69399999999999995,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.3759999999999999,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.60799999999999998,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.625,"15min":0.59899999999999998},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.9609999999999999,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6179999999999999,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.755}]},{"osd":5,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.72899999999999998,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4860000000000002,"5min":3.4860000000000002,"15min":3.4860000000000002},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.70699999999999996,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":3.996,"5min":3.996,"15min":3.996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.626}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":2.028,"15min":2.028},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91400000000000003},"min":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95299999999999996,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.90400000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.2670000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":0.95299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528358,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":461198,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.625},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.435},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.36499999999999999}]},{"osd":4,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.63,"15min":0.63},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.504}]},{"osd":5,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri May 10 04:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70799999999999996}]},{"osd":8,"last update":"Fri May 10 04:38:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.65500000000000003}]},{"osd":9,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57399999999999995}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":1003520,"data_stored":523888,"data_compressed":13284,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53500000000000003,"15min":0.5},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.505}]},{"osd":2,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.47899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri May 10 04:38:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46100000000000002,"15min":0.438},"min":{"1min":0.33200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.47999999999999998,"15min":0.44700000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.621}]},{"osd":5,"last update":"Fri May 10 04:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.58499999999999996,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":1.123,"15min":1.123},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.69199999999999995,"15min":0.64900000000000002},"min":{"1min":0.55100000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1100000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.749}]},{"osd":6,"last update":"Fri May 10 04:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.008,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78400000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.52800000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.626,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1040000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.85099999999999998}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83699999999999997}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":463047,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.673,"5min":2.673,"15min":2.673},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.55800000000000005,"15min":0.48699999999999999},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.621}]},{"osd":2,"last update":"Fri May 10 04:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.68500000000000005,"5min":1.123,"15min":1.123},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri May 10 04:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.37}]},{"osd":4,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0089999999999999,"5min":1.093,"15min":1.093},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri May 10 04:38:07 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.41599999999999998}]},{"osd":7,"last update":"Fri May 10 04:37:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.61199999999999999}]},{"osd":8,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.58899999999999997}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69599999999999995,"15min":0.67300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.7190000000000003,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.56200000000000006}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77300000000000002}]},{"osd":11,"last update":"Fri May 10 04:38:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":526105,"data_compressed":13284,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.502,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.47999999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.67800000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44}]},{"osd":1,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80000000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":6,"last update":"Fri May 10 04:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri May 10 04:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.75700000000000001}]},{"osd":8,"last update":"Fri May 10 04:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58099999999999996}]},{"osd":9,"last update":"Fri May 10 04:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68899999999999995}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":460954,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35799999999999998,"15min":0.34000000000000002},"min":{"1min":0.31900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri May 10 04:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.45500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81499999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.501}]},{"osd":5,"last update":"Fri May 10 04:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.379},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri May 10 04:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78300000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri May 10 04:38:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69599999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96799999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60499999999999998}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91600000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.64800000000000002}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.621}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920814,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":465272,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":1.143,"15min":1.143},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.36099999999999999},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.441},{"interface":"front","average":{"1min":0.434,"5min":0.38200000000000001,"15min":0.36199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.627,"5min":0.748,"15min":0.748},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":1.03,"15min":1.03},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88300000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90700000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67200000000000004,"5min":1.069,"15min":1.069},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":1.079,"15min":1.079},"last":0.622}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56599999999999995}]},{"osd":8,"last update":"Fri May 10 04:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.52800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88500000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.86399999999999999}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.80300000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73199999999999998}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1024000,"data_stored":1057760,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68999999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85499999999999998,"5min":1.008,"15min":1.008},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81599999999999995,"5min":1.089,"15min":1.089},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84499999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.48399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.80600000000000005}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63700000000000001}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.745},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63364,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65581,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63348,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:39:05.989 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-10T04:39:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:06 smithi150 ceph-mon[42404]: pgmap v253: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:06 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:39:06.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:06 smithi161 ceph-mon[46912]: pgmap v253: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:06.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:06 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:39:06.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:06 smithi181 ceph-mon[47314]: pgmap v253: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:06.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:06 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:39:07.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 -- 172.21.15.150:0/2412831537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee81000f0 msgr2=0x7f0ee8100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:07.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/2412831537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee81000f0 0x7f0ee8100500 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009990 tx=0x7f0ed8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 -- 172.21.15.150:0/2412831537 shutdown_connections 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/2412831537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee8100a40 0x7f0ee810ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/2412831537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee8106c60 0x7f0ee8107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/2412831537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee81000f0 0x7f0ee8100500 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 -- 172.21.15.150:0/2412831537 >> 172.21.15.150:0/2412831537 conn(0x7f0ee80fbed0 msgr2=0x7f0ee80fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 -- 172.21.15.150:0/2412831537 shutdown_connections 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 -- 172.21.15.150:0/2412831537 wait complete. 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.544+0000 7f0ef00fb700 1 Processor -- start 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0ef00fb700 1 -- start start 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0ef00fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee81000f0 0x7f0ee819e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0ef00fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee8100a40 0x7f0ee819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0ef00fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8106c60 0x7f0ee8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0ef00fb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8198940 con 0x7f0ee8106c60 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0ef00fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8198ab0 con 0x7f0ee81000f0 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0ef00fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8198c70 con 0x7f0ee8100a40 2024-05-10T04:39:07.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8106c60 0x7f0ee8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eede97700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee81000f0 0x7f0ee819e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8106c60 0x7f0ee8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59150/0 (socket says 172.21.15.150:59150) 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 -- 172.21.15.150:0/99450535 learned_addr learned my addr 172.21.15.150:0/99450535 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eed696700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee8100a40 0x7f0ee819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 -- 172.21.15.150:0/99450535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee8100a40 msgr2=0x7f0ee819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee8100a40 0x7f0ee819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 -- 172.21.15.150:0/99450535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee81000f0 msgr2=0x7f0ee819e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee81000f0 0x7f0ee819e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eee698700 1 -- 172.21.15.150:0/99450535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80095f0 con 0x7f0ee8106c60 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.545+0000 7f0eede97700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee81000f0 0x7f0ee819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.546+0000 7f0eee698700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8106c60 0x7f0ee8198400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0007d70 tx=0x7f0ee000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.546+0000 7f0edeffd700 1 -- 172.21.15.150:0/99450535 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee000ac90 con 0x7f0ee8106c60 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.546+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee8198f50 con 0x7f0ee8106c60 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.546+0000 7f0edeffd700 1 -- 172.21.15.150:0/99450535 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee000adf0 con 0x7f0ee8106c60 2024-05-10T04:39:07.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.546+0000 7f0edeffd700 1 -- 172.21.15.150:0/99450535 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee00115e0 con 0x7f0ee8106c60 2024-05-10T04:39:07.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.546+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee81a52a0 con 0x7f0ee8106c60 2024-05-10T04:39:07.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.547+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee810a220 con 0x7f0ee8106c60 2024-05-10T04:39:07.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.548+0000 7f0edeffd700 1 -- 172.21.15.150:0/99450535 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f0ee0004ad0 con 0x7f0ee8106c60 2024-05-10T04:39:07.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.549+0000 7f0edeffd700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0ed40a11b0 0x7f0ed40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:07.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.549+0000 7f0eede97700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0ed40a11b0 0x7f0ed40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:07.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.549+0000 7f0edeffd700 1 -- 172.21.15.150:0/99450535 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0ee00cd170 con 0x7f0ee8106c60 2024-05-10T04:39:07.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.549+0000 7f0eede97700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0ed40a11b0 0x7f0ed40a36c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8199870 tx=0x7f0ee400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:07.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.552+0000 7f0edeffd700 1 -- 172.21.15.150:0/99450535 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0ee0096660 con 0x7f0ee8106c60 2024-05-10T04:39:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:07 smithi150 ceph-mon[42404]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:07 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.779 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.777+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ee8199b20 con 0x7f0ed40a11b0 2024-05-10T04:39:07.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.790+0000 7f0edeffd700 1 -- 172.21.15.150:0/99450535 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423078 (secure 0 0 0) 0x7f0ee8199b20 con 0x7f0ed40a11b0 2024-05-10T04:39:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:07.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0ed40a11b0 msgr2=0x7f0ed40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0ed40a11b0 0x7f0ed40a36c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8199870 tx=0x7f0ee400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8106c60 msgr2=0x7f0ee8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8106c60 0x7f0ee8198400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0007d70 tx=0x7f0ee000a340 comp rx=0 tx=0).stop 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 shutdown_connections 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8106c60 0x7f0ee8198400 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ee81000f0 0x7f0ee819e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0ed40a11b0 0x7f0ed40a36c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 --2- 172.21.15.150:0/99450535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee8100a40 0x7f0ee819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.794+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 >> 172.21.15.150:0/99450535 conn(0x7f0ee80fbed0 msgr2=0x7f0ee80fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.795+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 shutdown_connections 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:07.795+0000 7f0ef00fb700 1 -- 172.21.15.150:0/99450535 wait complete. 2024-05-10T04:39:07.801 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:39:07.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:07 smithi161 ceph-mon[46912]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:07.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:07 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:07 smithi181 ceph-mon[47314]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:07.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:07.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:07 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.209 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-05-10T04:39:07.319553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658885,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":459,"num_read_kb":351,"num_write":416,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360336,"kb_used_data":11568,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581488,"statfs":{"total":1151940427776,"available":1151571443712,"internally_reserved":0,"allocated":11845632,"data_stored":7537982,"data_compressed":251892,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19127,"internal_metadata":357021001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3564,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":64,"num_write":2,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787827+0000","last_change":"2024-05-10T04:38:08.784019+0000","last_active":"2024-05-10T04:38:12.787827+0000","last_peered":"2024-05-10T04:38:12.787827+0000","last_clean":"2024-05-10T04:38:12.787827+0000","last_became_active":"2024-05-10T04:38:08.783585+0000","last_became_peered":"2024-05-10T04:38:08.783585+0000","last_unstale":"2024-05-10T04:38:12.787827+0000","last_undegraded":"2024-05-10T04:38:12.787827+0000","last_fullsized":"2024-05-10T04:38:12.787827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:56.963068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788295+0000","last_change":"2024-05-10T04:38:08.777364+0000","last_active":"2024-05-10T04:38:12.788295+0000","last_peered":"2024-05-10T04:38:12.788295+0000","last_clean":"2024-05-10T04:38:12.788295+0000","last_became_active":"2024-05-10T04:38:08.776488+0000","last_became_peered":"2024-05-10T04:38:08.776488+0000","last_unstale":"2024-05-10T04:38:12.788295+0000","last_undegraded":"2024-05-10T04:38:12.788295+0000","last_fullsized":"2024-05-10T04:38:12.788295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:40.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787110+0000","last_change":"2024-05-10T04:38:08.777169+0000","last_active":"2024-05-10T04:38:12.787110+0000","last_peered":"2024-05-10T04:38:12.787110+0000","last_clean":"2024-05-10T04:38:12.787110+0000","last_became_active":"2024-05-10T04:38:08.776713+0000","last_became_peered":"2024-05-10T04:38:08.776713+0000","last_unstale":"2024-05-10T04:38:12.787110+0000","last_undegraded":"2024-05-10T04:38:12.787110+0000","last_fullsized":"2024-05-10T04:38:12.787110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:08:41.947577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791723+0000","last_change":"2024-05-10T04:38:08.788464+0000","last_active":"2024-05-10T04:38:12.791723+0000","last_peered":"2024-05-10T04:38:12.791723+0000","last_clean":"2024-05-10T04:38:12.791723+0000","last_became_active":"2024-05-10T04:38:08.787811+0000","last_became_peered":"2024-05-10T04:38:08.787811+0000","last_unstale":"2024-05-10T04:38:12.791723+0000","last_undegraded":"2024-05-10T04:38:12.791723+0000","last_fullsized":"2024-05-10T04:38:12.791723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:22.574882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156610+0000","last_change":"2024-05-10T04:38:08.772904+0000","last_active":"2024-05-10T04:38:13.156610+0000","last_peered":"2024-05-10T04:38:13.156610+0000","last_clean":"2024-05-10T04:38:13.156610+0000","last_became_active":"2024-05-10T04:38:08.772503+0000","last_became_peered":"2024-05-10T04:38:08.772503+0000","last_unstale":"2024-05-10T04:38:13.156610+0000","last_undegraded":"2024-05-10T04:38:13.156610+0000","last_fullsized":"2024-05-10T04:38:13.156610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:58.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809064+0000","last_change":"2024-05-10T04:38:08.779442+0000","last_active":"2024-05-10T04:38:12.809064+0000","last_peered":"2024-05-10T04:38:12.809064+0000","last_clean":"2024-05-10T04:38:12.809064+0000","last_became_active":"2024-05-10T04:38:08.777348+0000","last_became_peered":"2024-05-10T04:38:08.777348+0000","last_unstale":"2024-05-10T04:38:12.809064+0000","last_undegraded":"2024-05-10T04:38:12.809064+0000","last_fullsized":"2024-05-10T04:38:12.809064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:25.619031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786546+0000","last_change":"2024-05-10T04:38:08.775646+0000","last_active":"2024-05-10T04:38:12.786546+0000","last_peered":"2024-05-10T04:38:12.786546+0000","last_clean":"2024-05-10T04:38:12.786546+0000","last_became_active":"2024-05-10T04:38:08.768229+0000","last_became_peered":"2024-05-10T04:38:08.768229+0000","last_unstale":"2024-05-10T04:38:12.786546+0000","last_undegraded":"2024-05-10T04:38:12.786546+0000","last_fullsized":"2024-05-10T04:38:12.786546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:14.653065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787545+0000","last_change":"2024-05-10T04:38:08.771012+0000","last_active":"2024-05-10T04:38:12.787545+0000","last_peered":"2024-05-10T04:38:12.787545+0000","last_clean":"2024-05-10T04:38:12.787545+0000","last_became_active":"2024-05-10T04:38:08.770674+0000","last_became_peered":"2024-05-10T04:38:08.770674+0000","last_unstale":"2024-05-10T04:38:12.787545+0000","last_undegraded":"2024-05-10T04:38:12.787545+0000","last_fullsized":"2024-05-10T04:38:12.787545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:14.389263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784668+0000","last_change":"2024-05-10T04:38:08.782109+0000","last_active":"2024-05-10T04:38:12.784668+0000","last_peered":"2024-05-10T04:38:12.784668+0000","last_clean":"2024-05-10T04:38:12.784668+0000","last_became_active":"2024-05-10T04:38:08.781717+0000","last_became_peered":"2024-05-10T04:38:08.781717+0000","last_unstale":"2024-05-10T04:38:12.784668+0000","last_undegraded":"2024-05-10T04:38:12.784668+0000","last_fullsized":"2024-05-10T04:38:12.784668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:17.505448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809930+0000","last_change":"2024-05-10T04:38:08.778429+0000","last_active":"2024-05-10T04:38:12.809930+0000","last_peered":"2024-05-10T04:38:12.809930+0000","last_clean":"2024-05-10T04:38:12.809930+0000","last_became_active":"2024-05-10T04:38:08.777874+0000","last_became_peered":"2024-05-10T04:38:08.777874+0000","last_unstale":"2024-05-10T04:38:12.809930+0000","last_undegraded":"2024-05-10T04:38:12.809930+0000","last_fullsized":"2024-05-10T04:38:12.809930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:20:19.704678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786985+0000","last_change":"2024-05-10T04:38:08.780104+0000","last_active":"2024-05-10T04:38:12.786985+0000","last_peered":"2024-05-10T04:38:12.786985+0000","last_clean":"2024-05-10T04:38:12.786985+0000","last_became_active":"2024-05-10T04:38:08.779298+0000","last_became_peered":"2024-05-10T04:38:08.779298+0000","last_unstale":"2024-05-10T04:38:12.786985+0000","last_undegraded":"2024-05-10T04:38:12.786985+0000","last_fullsized":"2024-05-10T04:38:12.786985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:41.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.791619+0000","last_change":"2024-05-10T04:38:08.788605+0000","last_active":"2024-05-10T04:38:12.791619+0000","last_peered":"2024-05-10T04:38:12.791619+0000","last_clean":"2024-05-10T04:38:12.791619+0000","last_became_active":"2024-05-10T04:38:08.788207+0000","last_became_peered":"2024-05-10T04:38:08.788207+0000","last_unstale":"2024-05-10T04:38:12.791619+0000","last_undegraded":"2024-05-10T04:38:12.791619+0000","last_fullsized":"2024-05-10T04:38:12.791619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:35.000980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782473+0000","last_change":"2024-05-10T04:38:08.761953+0000","last_active":"2024-05-10T04:38:12.782473+0000","last_peered":"2024-05-10T04:38:12.782473+0000","last_clean":"2024-05-10T04:38:12.782473+0000","last_became_active":"2024-05-10T04:38:08.761585+0000","last_became_peered":"2024-05-10T04:38:08.761585+0000","last_unstale":"2024-05-10T04:38:12.782473+0000","last_undegraded":"2024-05-10T04:38:12.782473+0000","last_fullsized":"2024-05-10T04:38:12.782473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:25.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786743+0000","last_change":"2024-05-10T04:38:08.777656+0000","last_active":"2024-05-10T04:38:12.786743+0000","last_peered":"2024-05-10T04:38:12.786743+0000","last_clean":"2024-05-10T04:38:12.786743+0000","last_became_active":"2024-05-10T04:38:08.777332+0000","last_became_peered":"2024-05-10T04:38:08.777332+0000","last_unstale":"2024-05-10T04:38:12.786743+0000","last_undegraded":"2024-05-10T04:38:12.786743+0000","last_fullsized":"2024-05-10T04:38:12.786743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:17:52.878943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787112+0000","last_change":"2024-05-10T04:38:08.784934+0000","last_active":"2024-05-10T04:38:12.787112+0000","last_peered":"2024-05-10T04:38:12.787112+0000","last_clean":"2024-05-10T04:38:12.787112+0000","last_became_active":"2024-05-10T04:38:08.784644+0000","last_became_peered":"2024-05-10T04:38:08.784644+0000","last_unstale":"2024-05-10T04:38:12.787112+0000","last_undegraded":"2024-05-10T04:38:12.787112+0000","last_fullsized":"2024-05-10T04:38:12.787112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:43.889305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783537+0000","last_change":"2024-05-10T04:38:08.784110+0000","last_active":"2024-05-10T04:38:12.783537+0000","last_peered":"2024-05-10T04:38:12.783537+0000","last_clean":"2024-05-10T04:38:12.783537+0000","last_became_active":"2024-05-10T04:38:08.783779+0000","last_became_peered":"2024-05-10T04:38:08.783779+0000","last_unstale":"2024-05-10T04:38:12.783537+0000","last_undegraded":"2024-05-10T04:38:12.783537+0000","last_fullsized":"2024-05-10T04:38:12.783537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:24.233024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787536+0000","last_change":"2024-05-10T04:38:08.788685+0000","last_active":"2024-05-10T04:38:12.787536+0000","last_peered":"2024-05-10T04:38:12.787536+0000","last_clean":"2024-05-10T04:38:12.787536+0000","last_became_active":"2024-05-10T04:38:08.788122+0000","last_became_peered":"2024-05-10T04:38:08.788122+0000","last_unstale":"2024-05-10T04:38:12.787536+0000","last_undegraded":"2024-05-10T04:38:12.787536+0000","last_fullsized":"2024-05-10T04:38:12.787536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:21:01.098784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802717+0000","last_change":"2024-05-10T04:38:08.776330+0000","last_active":"2024-05-10T04:38:12.802717+0000","last_peered":"2024-05-10T04:38:12.802717+0000","last_clean":"2024-05-10T04:38:12.802717+0000","last_became_active":"2024-05-10T04:38:08.776005+0000","last_became_peered":"2024-05-10T04:38:08.776005+0000","last_unstale":"2024-05-10T04:38:12.802717+0000","last_undegraded":"2024-05-10T04:38:12.802717+0000","last_fullsized":"2024-05-10T04:38:12.802717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:46:14.407766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787933+0000","last_change":"2024-05-10T04:38:08.772811+0000","last_active":"2024-05-10T04:38:12.787933+0000","last_peered":"2024-05-10T04:38:12.787933+0000","last_clean":"2024-05-10T04:38:12.787933+0000","last_became_active":"2024-05-10T04:38:08.772462+0000","last_became_peered":"2024-05-10T04:38:08.772462+0000","last_unstale":"2024-05-10T04:38:12.787933+0000","last_undegraded":"2024-05-10T04:38:12.787933+0000","last_fullsized":"2024-05-10T04:38:12.787933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:39.703137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410816+0000","last_change":"2024-05-10T04:38:12.799483+0000","last_active":"2024-05-10T04:38:13.410816+0000","last_peered":"2024-05-10T04:38:13.410816+0000","last_clean":"2024-05-10T04:38:13.410816+0000","last_became_active":"2024-05-10T04:38:12.799049+0000","last_became_peered":"2024-05-10T04:38:12.799049+0000","last_unstale":"2024-05-10T04:38:13.410816+0000","last_undegraded":"2024-05-10T04:38:13.410816+0000","last_fullsized":"2024-05-10T04:38:13.410816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:00.380645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:04.024330+0000","last_change":"2024-05-10T04:38:08.345935+0000","last_active":"2024-05-10T04:39:04.024330+0000","last_peered":"2024-05-10T04:39:04.024330+0000","last_clean":"2024-05-10T04:39:04.024330+0000","last_became_active":"2024-05-10T04:38:08.345580+0000","last_became_peered":"2024-05-10T04:38:08.345580+0000","last_unstale":"2024-05-10T04:39:04.024330+0000","last_undegraded":"2024-05-10T04:39:04.024330+0000","last_fullsized":"2024-05-10T04:39:04.024330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:11.527710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787511+0000","last_change":"2024-05-10T04:38:08.771072+0000","last_active":"2024-05-10T04:38:12.787511+0000","last_peered":"2024-05-10T04:38:12.787511+0000","last_clean":"2024-05-10T04:38:12.787511+0000","last_became_active":"2024-05-10T04:38:08.770711+0000","last_became_peered":"2024-05-10T04:38:08.770711+0000","last_unstale":"2024-05-10T04:38:12.787511+0000","last_undegraded":"2024-05-10T04:38:12.787511+0000","last_fullsized":"2024-05-10T04:38:12.787511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:44.803219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220274+0000","last_change":"2024-05-10T04:38:12.801374+0000","last_active":"2024-05-10T04:38:13.220274+0000","last_peered":"2024-05-10T04:38:13.220274+0000","last_clean":"2024-05-10T04:38:13.220274+0000","last_became_active":"2024-05-10T04:38:12.800889+0000","last_became_peered":"2024-05-10T04:38:12.800889+0000","last_unstale":"2024-05-10T04:38:13.220274+0000","last_undegraded":"2024-05-10T04:38:13.220274+0000","last_fullsized":"2024-05-10T04:38:13.220274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:35:51.641742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217645+0000","last_change":"2024-05-10T04:38:08.348806+0000","last_active":"2024-05-10T04:38:13.217645+0000","last_peered":"2024-05-10T04:38:13.217645+0000","last_clean":"2024-05-10T04:38:13.217645+0000","last_became_active":"2024-05-10T04:38:08.348513+0000","last_became_peered":"2024-05-10T04:38:08.348513+0000","last_unstale":"2024-05-10T04:38:13.217645+0000","last_undegraded":"2024-05-10T04:38:13.217645+0000","last_fullsized":"2024-05-10T04:38:13.217645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:45.474403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787353+0000","last_change":"2024-05-10T04:38:08.782738+0000","last_active":"2024-05-10T04:38:12.787353+0000","last_peered":"2024-05-10T04:38:12.787353+0000","last_clean":"2024-05-10T04:38:12.787353+0000","last_became_active":"2024-05-10T04:38:08.782375+0000","last_became_peered":"2024-05-10T04:38:08.782375+0000","last_unstale":"2024-05-10T04:38:12.787353+0000","last_undegraded":"2024-05-10T04:38:12.787353+0000","last_fullsized":"2024-05-10T04:38:12.787353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:16.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220388+0000","last_change":"2024-05-10T04:38:12.801322+0000","last_active":"2024-05-10T04:38:13.220388+0000","last_peered":"2024-05-10T04:38:13.220388+0000","last_clean":"2024-05-10T04:38:13.220388+0000","last_became_active":"2024-05-10T04:38:12.800388+0000","last_became_peered":"2024-05-10T04:38:12.800388+0000","last_unstale":"2024-05-10T04:38:13.220388+0000","last_undegraded":"2024-05-10T04:38:13.220388+0000","last_fullsized":"2024-05-10T04:38:13.220388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:57.678691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.985782+0000","last_change":"2024-05-10T04:38:08.347997+0000","last_active":"2024-05-10T04:38:35.985782+0000","last_peered":"2024-05-10T04:38:35.985782+0000","last_clean":"2024-05-10T04:38:35.985782+0000","last_became_active":"2024-05-10T04:38:08.347469+0000","last_became_peered":"2024-05-10T04:38:08.347469+0000","last_unstale":"2024-05-10T04:38:35.985782+0000","last_undegraded":"2024-05-10T04:38:35.985782+0000","last_fullsized":"2024-05-10T04:38:35.985782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:39.122148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783940+0000","last_change":"2024-05-10T04:38:08.781068+0000","last_active":"2024-05-10T04:38:12.783940+0000","last_peered":"2024-05-10T04:38:12.783940+0000","last_clean":"2024-05-10T04:38:12.783940+0000","last_became_active":"2024-05-10T04:38:08.780671+0000","last_became_peered":"2024-05-10T04:38:08.780671+0000","last_unstale":"2024-05-10T04:38:12.783940+0000","last_undegraded":"2024-05-10T04:38:12.783940+0000","last_fullsized":"2024-05-10T04:38:12.783940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220322+0000","last_change":"2024-05-10T04:38:12.801702+0000","last_active":"2024-05-10T04:38:13.220322+0000","last_peered":"2024-05-10T04:38:13.220322+0000","last_clean":"2024-05-10T04:38:13.220322+0000","last_became_active":"2024-05-10T04:38:12.801031+0000","last_became_peered":"2024-05-10T04:38:12.801031+0000","last_unstale":"2024-05-10T04:38:13.220322+0000","last_undegraded":"2024-05-10T04:38:13.220322+0000","last_fullsized":"2024-05-10T04:38:13.220322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:05:07.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970534+0000","last_change":"2024-05-10T04:38:08.347981+0000","last_active":"2024-05-10T04:38:34.970534+0000","last_peered":"2024-05-10T04:38:34.970534+0000","last_clean":"2024-05-10T04:38:34.970534+0000","last_became_active":"2024-05-10T04:38:08.346915+0000","last_became_peered":"2024-05-10T04:38:08.346915+0000","last_unstale":"2024-05-10T04:38:34.970534+0000","last_undegraded":"2024-05-10T04:38:34.970534+0000","last_fullsized":"2024-05-10T04:38:34.970534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:52.383734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780699+0000","last_change":"2024-05-10T04:38:08.779164+0000","last_active":"2024-05-10T04:38:12.780699+0000","last_peered":"2024-05-10T04:38:12.780699+0000","last_clean":"2024-05-10T04:38:12.780699+0000","last_became_active":"2024-05-10T04:38:08.778851+0000","last_became_peered":"2024-05-10T04:38:08.778851+0000","last_unstale":"2024-05-10T04:38:12.780699+0000","last_undegraded":"2024-05-10T04:38:12.780699+0000","last_fullsized":"2024-05-10T04:38:12.780699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:01.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411122+0000","last_change":"2024-05-10T04:38:12.796175+0000","last_active":"2024-05-10T04:38:13.411122+0000","last_peered":"2024-05-10T04:38:13.411122+0000","last_clean":"2024-05-10T04:38:13.411122+0000","last_became_active":"2024-05-10T04:38:12.795752+0000","last_became_peered":"2024-05-10T04:38:12.795752+0000","last_unstale":"2024-05-10T04:38:13.411122+0000","last_undegraded":"2024-05-10T04:38:13.411122+0000","last_fullsized":"2024-05-10T04:38:13.411122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:44:48.077369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:03.992208+0000","last_change":"2024-05-10T04:38:08.341633+0000","last_active":"2024-05-10T04:39:03.992208+0000","last_peered":"2024-05-10T04:39:03.992208+0000","last_clean":"2024-05-10T04:39:03.992208+0000","last_became_active":"2024-05-10T04:38:08.341220+0000","last_became_peered":"2024-05-10T04:38:08.341220+0000","last_unstale":"2024-05-10T04:39:03.992208+0000","last_undegraded":"2024-05-10T04:39:03.992208+0000","last_fullsized":"2024-05-10T04:39:03.992208+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:36.861191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156596+0000","last_change":"2024-05-10T04:38:08.790969+0000","last_active":"2024-05-10T04:38:13.156596+0000","last_peered":"2024-05-10T04:38:13.156596+0000","last_clean":"2024-05-10T04:38:13.156596+0000","last_became_active":"2024-05-10T04:38:08.790539+0000","last_became_peered":"2024-05-10T04:38:08.790539+0000","last_unstale":"2024-05-10T04:38:13.156596+0000","last_undegraded":"2024-05-10T04:38:13.156596+0000","last_fullsized":"2024-05-10T04:38:13.156596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:50.174798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.411137+0000","last_change":"2024-05-10T04:38:12.800901+0000","last_active":"2024-05-10T04:38:13.411137+0000","last_peered":"2024-05-10T04:38:13.411137+0000","last_clean":"2024-05-10T04:38:13.411137+0000","last_became_active":"2024-05-10T04:38:12.800287+0000","last_became_peered":"2024-05-10T04:38:12.800287+0000","last_unstale":"2024-05-10T04:38:13.411137+0000","last_undegraded":"2024-05-10T04:38:13.411137+0000","last_fullsized":"2024-05-10T04:38:13.411137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:23:25.463044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.965633+0000","last_change":"2024-05-10T04:38:08.342449+0000","last_active":"2024-05-10T04:38:33.965633+0000","last_peered":"2024-05-10T04:38:33.965633+0000","last_clean":"2024-05-10T04:38:33.965633+0000","last_became_active":"2024-05-10T04:38:08.342043+0000","last_became_peered":"2024-05-10T04:38:08.342043+0000","last_unstale":"2024-05-10T04:38:33.965633+0000","last_undegraded":"2024-05-10T04:38:33.965633+0000","last_fullsized":"2024-05-10T04:38:33.965633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:31.161203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780542+0000","last_change":"2024-05-10T04:38:08.783759+0000","last_active":"2024-05-10T04:38:12.780542+0000","last_peered":"2024-05-10T04:38:12.780542+0000","last_clean":"2024-05-10T04:38:12.780542+0000","last_became_active":"2024-05-10T04:38:08.783419+0000","last_became_peered":"2024-05-10T04:38:08.783419+0000","last_unstale":"2024-05-10T04:38:12.780542+0000","last_undegraded":"2024-05-10T04:38:12.780542+0000","last_fullsized":"2024-05-10T04:38:12.780542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:16.114017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218482+0000","last_change":"2024-05-10T04:38:12.803018+0000","last_active":"2024-05-10T04:38:13.218482+0000","last_peered":"2024-05-10T04:38:13.218482+0000","last_clean":"2024-05-10T04:38:13.218482+0000","last_became_active":"2024-05-10T04:38:12.802546+0000","last_became_peered":"2024-05-10T04:38:12.802546+0000","last_unstale":"2024-05-10T04:38:13.218482+0000","last_undegraded":"2024-05-10T04:38:13.218482+0000","last_fullsized":"2024-05-10T04:38:13.218482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:40.676335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.950654+0000","last_change":"2024-05-10T04:38:08.341404+0000","last_active":"2024-05-10T04:38:32.950654+0000","last_peered":"2024-05-10T04:38:32.950654+0000","last_clean":"2024-05-10T04:38:32.950654+0000","last_became_active":"2024-05-10T04:38:08.341013+0000","last_became_peered":"2024-05-10T04:38:08.341013+0000","last_unstale":"2024-05-10T04:38:32.950654+0000","last_undegraded":"2024-05-10T04:38:32.950654+0000","last_fullsized":"2024-05-10T04:38:32.950654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:43.513989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787639+0000","last_change":"2024-05-10T04:38:08.775650+0000","last_active":"2024-05-10T04:38:12.787639+0000","last_peered":"2024-05-10T04:38:12.787639+0000","last_clean":"2024-05-10T04:38:12.787639+0000","last_became_active":"2024-05-10T04:38:08.775320+0000","last_became_peered":"2024-05-10T04:38:08.775320+0000","last_unstale":"2024-05-10T04:38:12.787639+0000","last_undegraded":"2024-05-10T04:38:12.787639+0000","last_fullsized":"2024-05-10T04:38:12.787639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:49:07.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.341020+0000","last_change":"2024-05-10T04:38:12.797168+0000","last_active":"2024-05-10T04:38:13.341020+0000","last_peered":"2024-05-10T04:38:13.341020+0000","last_clean":"2024-05-10T04:38:13.341020+0000","last_became_active":"2024-05-10T04:38:12.796776+0000","last_became_peered":"2024-05-10T04:38:12.796776+0000","last_unstale":"2024-05-10T04:38:13.341020+0000","last_undegraded":"2024-05-10T04:38:13.341020+0000","last_fullsized":"2024-05-10T04:38:13.341020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:59:02.363398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970039+0000","last_change":"2024-05-10T04:38:08.344086+0000","last_active":"2024-05-10T04:38:34.970039+0000","last_peered":"2024-05-10T04:38:34.970039+0000","last_clean":"2024-05-10T04:38:34.970039+0000","last_became_active":"2024-05-10T04:38:08.343744+0000","last_became_peered":"2024-05-10T04:38:08.343744+0000","last_unstale":"2024-05-10T04:38:34.970039+0000","last_undegraded":"2024-05-10T04:38:34.970039+0000","last_fullsized":"2024-05-10T04:38:34.970039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:01.800590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.802611+0000","last_change":"2024-05-10T04:38:08.773638+0000","last_active":"2024-05-10T04:38:12.802611+0000","last_peered":"2024-05-10T04:38:12.802611+0000","last_clean":"2024-05-10T04:38:12.802611+0000","last_became_active":"2024-05-10T04:38:08.773218+0000","last_became_peered":"2024-05-10T04:38:08.773218+0000","last_unstale":"2024-05-10T04:38:12.802611+0000","last_undegraded":"2024-05-10T04:38:12.802611+0000","last_fullsized":"2024-05-10T04:38:12.802611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:54:38.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217138+0000","last_change":"2024-05-10T04:38:12.796344+0000","last_active":"2024-05-10T04:38:13.217138+0000","last_peered":"2024-05-10T04:38:13.217138+0000","last_clean":"2024-05-10T04:38:13.217138+0000","last_became_active":"2024-05-10T04:38:12.795958+0000","last_became_peered":"2024-05-10T04:38:12.795958+0000","last_unstale":"2024-05-10T04:38:13.217138+0000","last_undegraded":"2024-05-10T04:38:13.217138+0000","last_fullsized":"2024-05-10T04:38:13.217138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:45.708078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:01.988367+0000","last_change":"2024-05-10T04:38:08.341327+0000","last_active":"2024-05-10T04:39:01.988367+0000","last_peered":"2024-05-10T04:39:01.988367+0000","last_clean":"2024-05-10T04:39:01.988367+0000","last_became_active":"2024-05-10T04:38:08.339606+0000","last_became_peered":"2024-05-10T04:38:08.339606+0000","last_unstale":"2024-05-10T04:39:01.988367+0000","last_undegraded":"2024-05-10T04:39:01.988367+0000","last_fullsized":"2024-05-10T04:39:01.988367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:22:40.212212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784430+0000","last_change":"2024-05-10T04:38:08.781195+0000","last_active":"2024-05-10T04:38:12.784430+0000","last_peered":"2024-05-10T04:38:12.784430+0000","last_clean":"2024-05-10T04:38:12.784430+0000","last_became_active":"2024-05-10T04:38:08.780779+0000","last_became_peered":"2024-05-10T04:38:08.780779+0000","last_unstale":"2024-05-10T04:38:12.784430+0000","last_undegraded":"2024-05-10T04:38:12.784430+0000","last_fullsized":"2024-05-10T04:38:12.784430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:24.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930221+0000","last_change":"2024-05-10T04:38:12.799494+0000","last_active":"2024-05-10T04:38:13.930221+0000","last_peered":"2024-05-10T04:38:13.930221+0000","last_clean":"2024-05-10T04:38:13.930221+0000","last_became_active":"2024-05-10T04:38:12.799071+0000","last_became_peered":"2024-05-10T04:38:12.799071+0000","last_unstale":"2024-05-10T04:38:13.930221+0000","last_undegraded":"2024-05-10T04:38:13.930221+0000","last_fullsized":"2024-05-10T04:38:13.930221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:00.814875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970501+0000","last_change":"2024-05-10T04:38:08.348569+0000","last_active":"2024-05-10T04:38:34.970501+0000","last_peered":"2024-05-10T04:38:34.970501+0000","last_clean":"2024-05-10T04:38:34.970501+0000","last_became_active":"2024-05-10T04:38:08.347942+0000","last_became_peered":"2024-05-10T04:38:08.347942+0000","last_unstale":"2024-05-10T04:38:34.970501+0000","last_undegraded":"2024-05-10T04:38:34.970501+0000","last_fullsized":"2024-05-10T04:38:34.970501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.865369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787640+0000","last_change":"2024-05-10T04:38:08.784447+0000","last_active":"2024-05-10T04:38:12.787640+0000","last_peered":"2024-05-10T04:38:12.787640+0000","last_clean":"2024-05-10T04:38:12.787640+0000","last_became_active":"2024-05-10T04:38:08.783989+0000","last_became_peered":"2024-05-10T04:38:08.783989+0000","last_unstale":"2024-05-10T04:38:12.787640+0000","last_undegraded":"2024-05-10T04:38:12.787640+0000","last_fullsized":"2024-05-10T04:38:12.787640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:37.511400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220978+0000","last_change":"2024-05-10T04:38:12.807478+0000","last_active":"2024-05-10T04:38:13.220978+0000","last_peered":"2024-05-10T04:38:13.220978+0000","last_clean":"2024-05-10T04:38:13.220978+0000","last_became_active":"2024-05-10T04:38:12.806801+0000","last_became_peered":"2024-05-10T04:38:12.806801+0000","last_unstale":"2024-05-10T04:38:13.220978+0000","last_undegraded":"2024-05-10T04:38:13.220978+0000","last_fullsized":"2024-05-10T04:38:13.220978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:32.168836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.943959+0000","last_change":"2024-05-10T04:38:08.343610+0000","last_active":"2024-05-10T04:38:32.943959+0000","last_peered":"2024-05-10T04:38:32.943959+0000","last_clean":"2024-05-10T04:38:32.943959+0000","last_became_active":"2024-05-10T04:38:08.342880+0000","last_became_peered":"2024-05-10T04:38:08.342880+0000","last_unstale":"2024-05-10T04:38:32.943959+0000","last_undegraded":"2024-05-10T04:38:32.943959+0000","last_fullsized":"2024-05-10T04:38:32.943959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:45:11.502258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.784342+0000","last_change":"2024-05-10T04:38:08.780354+0000","last_active":"2024-05-10T04:38:12.784342+0000","last_peered":"2024-05-10T04:38:12.784342+0000","last_clean":"2024-05-10T04:38:12.784342+0000","last_became_active":"2024-05-10T04:38:08.779777+0000","last_became_peered":"2024-05-10T04:38:08.779777+0000","last_unstale":"2024-05-10T04:38:12.784342+0000","last_undegraded":"2024-05-10T04:38:12.784342+0000","last_fullsized":"2024-05-10T04:38:12.784342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:28.243474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931246+0000","last_change":"2024-05-10T04:38:12.800759+0000","last_active":"2024-05-10T04:38:13.931246+0000","last_peered":"2024-05-10T04:38:13.931246+0000","last_clean":"2024-05-10T04:38:13.931246+0000","last_became_active":"2024-05-10T04:38:12.800417+0000","last_became_peered":"2024-05-10T04:38:12.800417+0000","last_unstale":"2024-05-10T04:38:13.931246+0000","last_undegraded":"2024-05-10T04:38:13.931246+0000","last_fullsized":"2024-05-10T04:38:13.931246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:29:04.734817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'33","reported_seq":116,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:01.988113+0000","last_change":"2024-05-10T04:38:08.343873+0000","last_active":"2024-05-10T04:39:01.988113+0000","last_peered":"2024-05-10T04:39:01.988113+0000","last_clean":"2024-05-10T04:39:01.988113+0000","last_became_active":"2024-05-10T04:38:08.343360+0000","last_became_peered":"2024-05-10T04:38:08.343360+0000","last_unstale":"2024-05-10T04:39:01.988113+0000","last_undegraded":"2024-05-10T04:39:01.988113+0000","last_fullsized":"2024-05-10T04:39:01.988113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:06:10.816343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61660,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":94,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156650+0000","last_change":"2024-05-10T04:38:08.783955+0000","last_active":"2024-05-10T04:38:13.156650+0000","last_peered":"2024-05-10T04:38:13.156650+0000","last_clean":"2024-05-10T04:38:13.156650+0000","last_became_active":"2024-05-10T04:38:08.779131+0000","last_became_peered":"2024-05-10T04:38:08.779131+0000","last_unstale":"2024-05-10T04:38:13.156650+0000","last_undegraded":"2024-05-10T04:38:13.156650+0000","last_fullsized":"2024-05-10T04:38:13.156650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:58.619571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930607+0000","last_change":"2024-05-10T04:38:12.795140+0000","last_active":"2024-05-10T04:38:13.930607+0000","last_peered":"2024-05-10T04:38:13.930607+0000","last_clean":"2024-05-10T04:38:13.930607+0000","last_became_active":"2024-05-10T04:38:12.794666+0000","last_became_peered":"2024-05-10T04:38:12.794666+0000","last_unstale":"2024-05-10T04:38:13.930607+0000","last_undegraded":"2024-05-10T04:38:13.930607+0000","last_fullsized":"2024-05-10T04:38:13.930607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:44.468321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.971051+0000","last_change":"2024-05-10T04:38:08.338866+0000","last_active":"2024-05-10T04:38:34.971051+0000","last_peered":"2024-05-10T04:38:34.971051+0000","last_clean":"2024-05-10T04:38:34.971051+0000","last_became_active":"2024-05-10T04:38:08.338434+0000","last_became_peered":"2024-05-10T04:38:08.338434+0000","last_unstale":"2024-05-10T04:38:34.971051+0000","last_undegraded":"2024-05-10T04:38:34.971051+0000","last_fullsized":"2024-05-10T04:38:34.971051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:49.858034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781395+0000","last_change":"2024-05-10T04:38:08.768998+0000","last_active":"2024-05-10T04:38:12.781395+0000","last_peered":"2024-05-10T04:38:12.781395+0000","last_clean":"2024-05-10T04:38:12.781395+0000","last_became_active":"2024-05-10T04:38:08.768687+0000","last_became_peered":"2024-05-10T04:38:08.768687+0000","last_unstale":"2024-05-10T04:38:12.781395+0000","last_undegraded":"2024-05-10T04:38:12.781395+0000","last_fullsized":"2024-05-10T04:38:12.781395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:19.390746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218211+0000","last_change":"2024-05-10T04:38:12.801223+0000","last_active":"2024-05-10T04:38:13.218211+0000","last_peered":"2024-05-10T04:38:13.218211+0000","last_clean":"2024-05-10T04:38:13.218211+0000","last_became_active":"2024-05-10T04:38:12.800750+0000","last_became_peered":"2024-05-10T04:38:12.800750+0000","last_unstale":"2024-05-10T04:38:13.218211+0000","last_undegraded":"2024-05-10T04:38:13.218211+0000","last_fullsized":"2024-05-10T04:38:13.218211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:33.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968494+0000","last_change":"2024-05-10T04:38:08.344579+0000","last_active":"2024-05-10T04:38:34.968494+0000","last_peered":"2024-05-10T04:38:34.968494+0000","last_clean":"2024-05-10T04:38:34.968494+0000","last_became_active":"2024-05-10T04:38:08.344250+0000","last_became_peered":"2024-05-10T04:38:08.344250+0000","last_unstale":"2024-05-10T04:38:34.968494+0000","last_undegraded":"2024-05-10T04:38:34.968494+0000","last_fullsized":"2024-05-10T04:38:34.968494+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:56.491732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786793+0000","last_change":"2024-05-10T04:38:08.784556+0000","last_active":"2024-05-10T04:38:12.786793+0000","last_peered":"2024-05-10T04:38:12.786793+0000","last_clean":"2024-05-10T04:38:12.786793+0000","last_became_active":"2024-05-10T04:38:08.784229+0000","last_became_peered":"2024-05-10T04:38:08.784229+0000","last_unstale":"2024-05-10T04:38:12.786793+0000","last_undegraded":"2024-05-10T04:38:12.786793+0000","last_fullsized":"2024-05-10T04:38:12.786793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:48:46.519837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930516+0000","last_change":"2024-05-10T04:38:12.805795+0000","last_active":"2024-05-10T04:38:13.930516+0000","last_peered":"2024-05-10T04:38:13.930516+0000","last_clean":"2024-05-10T04:38:13.930516+0000","last_became_active":"2024-05-10T04:38:12.805366+0000","last_became_peered":"2024-05-10T04:38:12.805366+0000","last_unstale":"2024-05-10T04:38:13.930516+0000","last_undegraded":"2024-05-10T04:38:13.930516+0000","last_fullsized":"2024-05-10T04:38:13.930516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:38:11.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:02.022159+0000","last_change":"2024-05-10T04:38:08.345396+0000","last_active":"2024-05-10T04:39:02.022159+0000","last_peered":"2024-05-10T04:39:02.022159+0000","last_clean":"2024-05-10T04:39:02.022159+0000","last_became_active":"2024-05-10T04:38:08.345076+0000","last_became_peered":"2024-05-10T04:38:08.345076+0000","last_unstale":"2024-05-10T04:39:02.022159+0000","last_undegraded":"2024-05-10T04:39:02.022159+0000","last_fullsized":"2024-05-10T04:39:02.022159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:38.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787293+0000","last_change":"2024-05-10T04:38:08.791089+0000","last_active":"2024-05-10T04:38:12.787293+0000","last_peered":"2024-05-10T04:38:12.787293+0000","last_clean":"2024-05-10T04:38:12.787293+0000","last_became_active":"2024-05-10T04:38:08.790813+0000","last_became_peered":"2024-05-10T04:38:08.790813+0000","last_unstale":"2024-05-10T04:38:12.787293+0000","last_undegraded":"2024-05-10T04:38:12.787293+0000","last_fullsized":"2024-05-10T04:38:12.787293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:52:45.753196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809384+0000","last_change":"2024-05-10T04:38:12.803896+0000","last_active":"2024-05-10T04:38:12.809384+0000","last_peered":"2024-05-10T04:38:12.809384+0000","last_clean":"2024-05-10T04:38:12.809384+0000","last_became_active":"2024-05-10T04:38:12.803512+0000","last_became_peered":"2024-05-10T04:38:12.803512+0000","last_unstale":"2024-05-10T04:38:12.809384+0000","last_undegraded":"2024-05-10T04:38:12.809384+0000","last_fullsized":"2024-05-10T04:38:12.809384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:01:23.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970681+0000","last_change":"2024-05-10T04:38:08.348187+0000","last_active":"2024-05-10T04:38:34.970681+0000","last_peered":"2024-05-10T04:38:34.970681+0000","last_clean":"2024-05-10T04:38:34.970681+0000","last_became_active":"2024-05-10T04:38:08.346986+0000","last_became_peered":"2024-05-10T04:38:08.346986+0000","last_unstale":"2024-05-10T04:38:34.970681+0000","last_undegraded":"2024-05-10T04:38:34.970681+0000","last_fullsized":"2024-05-10T04:38:34.970681+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:37:46.245042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.780738+0000","last_change":"2024-05-10T04:38:08.782905+0000","last_active":"2024-05-10T04:38:12.780738+0000","last_peered":"2024-05-10T04:38:12.780738+0000","last_clean":"2024-05-10T04:38:12.780738+0000","last_became_active":"2024-05-10T04:38:08.782508+0000","last_became_peered":"2024-05-10T04:38:08.782508+0000","last_unstale":"2024-05-10T04:38:12.780738+0000","last_undegraded":"2024-05-10T04:38:12.780738+0000","last_fullsized":"2024-05-10T04:38:12.780738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:15:29.026282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930246+0000","last_change":"2024-05-10T04:38:12.796046+0000","last_active":"2024-05-10T04:38:13.930246+0000","last_peered":"2024-05-10T04:38:13.930246+0000","last_clean":"2024-05-10T04:38:13.930246+0000","last_became_active":"2024-05-10T04:38:12.795635+0000","last_became_peered":"2024-05-10T04:38:12.795635+0000","last_unstale":"2024-05-10T04:38:13.930246+0000","last_undegraded":"2024-05-10T04:38:13.930246+0000","last_fullsized":"2024-05-10T04:38:13.930246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:50.822404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.941305+0000","last_change":"2024-05-10T04:38:08.348701+0000","last_active":"2024-05-10T04:38:32.941305+0000","last_peered":"2024-05-10T04:38:32.941305+0000","last_clean":"2024-05-10T04:38:32.941305+0000","last_became_active":"2024-05-10T04:38:08.348345+0000","last_became_peered":"2024-05-10T04:38:08.348345+0000","last_unstale":"2024-05-10T04:38:32.941305+0000","last_undegraded":"2024-05-10T04:38:32.941305+0000","last_fullsized":"2024-05-10T04:38:32.941305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:31.505996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787204+0000","last_change":"2024-05-10T04:38:08.793070+0000","last_active":"2024-05-10T04:38:12.787204+0000","last_peered":"2024-05-10T04:38:12.787204+0000","last_clean":"2024-05-10T04:38:12.787204+0000","last_became_active":"2024-05-10T04:38:08.792694+0000","last_became_peered":"2024-05-10T04:38:08.792694+0000","last_unstale":"2024-05-10T04:38:12.787204+0000","last_undegraded":"2024-05-10T04:38:12.787204+0000","last_fullsized":"2024-05-10T04:38:12.787204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:49:23.636222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156578+0000","last_change":"2024-05-10T04:38:12.809215+0000","last_active":"2024-05-10T04:38:13.156578+0000","last_peered":"2024-05-10T04:38:13.156578+0000","last_clean":"2024-05-10T04:38:13.156578+0000","last_became_active":"2024-05-10T04:38:12.808717+0000","last_became_peered":"2024-05-10T04:38:12.808717+0000","last_unstale":"2024-05-10T04:38:13.156578+0000","last_undegraded":"2024-05-10T04:38:13.156578+0000","last_fullsized":"2024-05-10T04:38:13.156578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:40.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931223+0000","last_change":"2024-05-10T04:38:08.347770+0000","last_active":"2024-05-10T04:38:13.931223+0000","last_peered":"2024-05-10T04:38:13.931223+0000","last_clean":"2024-05-10T04:38:13.931223+0000","last_became_active":"2024-05-10T04:38:08.347444+0000","last_became_peered":"2024-05-10T04:38:08.347444+0000","last_unstale":"2024-05-10T04:38:13.931223+0000","last_undegraded":"2024-05-10T04:38:13.931223+0000","last_fullsized":"2024-05-10T04:38:13.931223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:49.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809464+0000","last_change":"2024-05-10T04:38:08.765358+0000","last_active":"2024-05-10T04:38:12.809464+0000","last_peered":"2024-05-10T04:38:12.809464+0000","last_clean":"2024-05-10T04:38:12.809464+0000","last_became_active":"2024-05-10T04:38:08.765090+0000","last_became_peered":"2024-05-10T04:38:08.765090+0000","last_unstale":"2024-05-10T04:38:12.809464+0000","last_undegraded":"2024-05-10T04:38:12.809464+0000","last_fullsized":"2024-05-10T04:38:12.809464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:55:25.722331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.798587+0000","last_change":"2024-05-10T04:38:12.794765+0000","last_active":"2024-05-10T04:38:12.798587+0000","last_peered":"2024-05-10T04:38:12.798587+0000","last_clean":"2024-05-10T04:38:12.798587+0000","last_became_active":"2024-05-10T04:38:12.794400+0000","last_became_peered":"2024-05-10T04:38:12.794400+0000","last_unstale":"2024-05-10T04:38:12.798587+0000","last_undegraded":"2024-05-10T04:38:12.798587+0000","last_fullsized":"2024-05-10T04:38:12.798587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:15:16.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:03.023719+0000","last_change":"2024-05-10T04:38:08.348812+0000","last_active":"2024-05-10T04:39:03.023719+0000","last_peered":"2024-05-10T04:39:03.023719+0000","last_clean":"2024-05-10T04:39:03.023719+0000","last_became_active":"2024-05-10T04:38:08.348207+0000","last_became_peered":"2024-05-10T04:38:08.348207+0000","last_unstale":"2024-05-10T04:39:03.023719+0000","last_undegraded":"2024-05-10T04:39:03.023719+0000","last_fullsized":"2024-05-10T04:39:03.023719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:17.350094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.809431+0000","last_change":"2024-05-10T04:38:08.785502+0000","last_active":"2024-05-10T04:38:12.809431+0000","last_peered":"2024-05-10T04:38:12.809431+0000","last_clean":"2024-05-10T04:38:12.809431+0000","last_became_active":"2024-05-10T04:38:08.785073+0000","last_became_peered":"2024-05-10T04:38:08.785073+0000","last_unstale":"2024-05-10T04:38:12.809431+0000","last_undegraded":"2024-05-10T04:38:12.809431+0000","last_fullsized":"2024-05-10T04:38:12.809431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:08:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.810257+0000","last_change":"2024-05-10T04:38:12.809599+0000","last_active":"2024-05-10T04:38:12.810257+0000","last_peered":"2024-05-10T04:38:12.810257+0000","last_clean":"2024-05-10T04:38:12.810257+0000","last_became_active":"2024-05-10T04:38:12.809200+0000","last_became_peered":"2024-05-10T04:38:12.809200+0000","last_unstale":"2024-05-10T04:38:12.810257+0000","last_undegraded":"2024-05-10T04:38:12.810257+0000","last_fullsized":"2024-05-10T04:38:12.810257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:54.940517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781249+0000","last_change":"2024-05-10T04:38:08.779837+0000","last_active":"2024-05-10T04:38:12.781249+0000","last_peered":"2024-05-10T04:38:12.781249+0000","last_clean":"2024-05-10T04:38:12.781249+0000","last_became_active":"2024-05-10T04:38:08.779501+0000","last_became_peered":"2024-05-10T04:38:08.779501+0000","last_unstale":"2024-05-10T04:38:12.781249+0000","last_undegraded":"2024-05-10T04:38:12.781249+0000","last_fullsized":"2024-05-10T04:38:12.781249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:00.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:05.025155+0000","last_change":"2024-05-10T04:38:08.343649+0000","last_active":"2024-05-10T04:39:05.025155+0000","last_peered":"2024-05-10T04:39:05.025155+0000","last_clean":"2024-05-10T04:39:05.025155+0000","last_became_active":"2024-05-10T04:38:08.343292+0000","last_became_peered":"2024-05-10T04:38:08.343292+0000","last_unstale":"2024-05-10T04:39:05.025155+0000","last_undegraded":"2024-05-10T04:39:05.025155+0000","last_fullsized":"2024-05-10T04:39:05.025155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:53.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:14.390654+0000","last_change":"2024-05-10T04:38:12.796552+0000","last_active":"2024-05-10T04:38:14.390654+0000","last_peered":"2024-05-10T04:38:14.390654+0000","last_clean":"2024-05-10T04:38:14.390654+0000","last_became_active":"2024-05-10T04:38:12.796151+0000","last_became_peered":"2024-05-10T04:38:12.796151+0000","last_unstale":"2024-05-10T04:38:14.390654+0000","last_undegraded":"2024-05-10T04:38:14.390654+0000","last_fullsized":"2024-05-10T04:38:14.390654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:05.520859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787088+0000","last_change":"2024-05-10T04:38:08.770957+0000","last_active":"2024-05-10T04:38:12.787088+0000","last_peered":"2024-05-10T04:38:12.787088+0000","last_clean":"2024-05-10T04:38:12.787088+0000","last_became_active":"2024-05-10T04:38:08.770572+0000","last_became_peered":"2024-05-10T04:38:08.770572+0000","last_unstale":"2024-05-10T04:38:12.787088+0000","last_undegraded":"2024-05-10T04:38:12.787088+0000","last_fullsized":"2024-05-10T04:38:12.787088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:21.421033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.985828+0000","last_change":"2024-05-10T04:38:08.340788+0000","last_active":"2024-05-10T04:38:35.985828+0000","last_peered":"2024-05-10T04:38:35.985828+0000","last_clean":"2024-05-10T04:38:35.985828+0000","last_became_active":"2024-05-10T04:38:08.340407+0000","last_became_peered":"2024-05-10T04:38:08.340407+0000","last_unstale":"2024-05-10T04:38:35.985828+0000","last_undegraded":"2024-05-10T04:38:35.985828+0000","last_fullsized":"2024-05-10T04:38:35.985828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:30:05.276951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":327,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930055+0000","last_change":"2024-05-10T04:37:20.961282+0000","last_active":"2024-05-10T04:38:13.930055+0000","last_peered":"2024-05-10T04:38:13.930055+0000","last_clean":"2024-05-10T04:38:13.930055+0000","last_became_active":"2024-05-10T04:37:20.960828+0000","last_became_peered":"2024-05-10T04:37:20.960828+0000","last_unstale":"2024-05-10T04:38:13.930055+0000","last_undegraded":"2024-05-10T04:38:13.930055+0000","last_fullsized":"2024-05-10T04:38:13.930055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_clean_scrub_stamp":"2024-05-10T04:35:00.248380+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:08:04.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930338+0000","last_change":"2024-05-10T04:38:12.809899+0000","last_active":"2024-05-10T04:38:13.930338+0000","last_peered":"2024-05-10T04:38:13.930338+0000","last_clean":"2024-05-10T04:38:13.930338+0000","last_became_active":"2024-05-10T04:38:12.809441+0000","last_became_peered":"2024-05-10T04:38:12.809441+0000","last_unstale":"2024-05-10T04:38:13.930338+0000","last_undegraded":"2024-05-10T04:38:13.930338+0000","last_fullsized":"2024-05-10T04:38:13.930338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:48.378890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781228+0000","last_change":"2024-05-10T04:38:08.784838+0000","last_active":"2024-05-10T04:38:12.781228+0000","last_peered":"2024-05-10T04:38:12.781228+0000","last_clean":"2024-05-10T04:38:12.781228+0000","last_became_active":"2024-05-10T04:38:08.784464+0000","last_became_peered":"2024-05-10T04:38:08.784464+0000","last_unstale":"2024-05-10T04:38:12.781228+0000","last_undegraded":"2024-05-10T04:38:12.781228+0000","last_fullsized":"2024-05-10T04:38:12.781228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:00.335350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.953106+0000","last_change":"2024-05-10T04:38:08.348639+0000","last_active":"2024-05-10T04:38:32.953106+0000","last_peered":"2024-05-10T04:38:32.953106+0000","last_clean":"2024-05-10T04:38:32.953106+0000","last_became_active":"2024-05-10T04:38:08.348271+0000","last_became_peered":"2024-05-10T04:38:08.348271+0000","last_unstale":"2024-05-10T04:38:32.953106+0000","last_undegraded":"2024-05-10T04:38:32.953106+0000","last_fullsized":"2024-05-10T04:38:32.953106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:56:18.486150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.218189+0000","last_change":"2024-05-10T04:38:12.801079+0000","last_active":"2024-05-10T04:38:13.218189+0000","last_peered":"2024-05-10T04:38:13.218189+0000","last_clean":"2024-05-10T04:38:13.218189+0000","last_became_active":"2024-05-10T04:38:12.800630+0000","last_became_peered":"2024-05-10T04:38:12.800630+0000","last_unstale":"2024-05-10T04:38:13.218189+0000","last_undegraded":"2024-05-10T04:38:13.218189+0000","last_fullsized":"2024-05-10T04:38:13.218189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:05.213620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.964694+0000","last_change":"2024-05-10T04:38:08.348764+0000","last_active":"2024-05-10T04:38:33.964694+0000","last_peered":"2024-05-10T04:38:33.964694+0000","last_clean":"2024-05-10T04:38:33.964694+0000","last_became_active":"2024-05-10T04:38:08.347939+0000","last_became_peered":"2024-05-10T04:38:08.347939+0000","last_unstale":"2024-05-10T04:38:33.964694+0000","last_undegraded":"2024-05-10T04:38:33.964694+0000","last_fullsized":"2024-05-10T04:38:33.964694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:01.308255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782312+0000","last_change":"2024-05-10T04:38:08.771382+0000","last_active":"2024-05-10T04:38:12.782312+0000","last_peered":"2024-05-10T04:38:12.782312+0000","last_clean":"2024-05-10T04:38:12.782312+0000","last_became_active":"2024-05-10T04:38:08.771108+0000","last_became_peered":"2024-05-10T04:38:08.771108+0000","last_unstale":"2024-05-10T04:38:12.782312+0000","last_undegraded":"2024-05-10T04:38:12.782312+0000","last_fullsized":"2024-05-10T04:38:12.782312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:32.664640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.217051+0000","last_change":"2024-05-10T04:38:12.809582+0000","last_active":"2024-05-10T04:38:13.217051+0000","last_peered":"2024-05-10T04:38:13.217051+0000","last_clean":"2024-05-10T04:38:13.217051+0000","last_became_active":"2024-05-10T04:38:12.809134+0000","last_became_peered":"2024-05-10T04:38:12.809134+0000","last_unstale":"2024-05-10T04:38:13.217051+0000","last_undegraded":"2024-05-10T04:38:13.217051+0000","last_fullsized":"2024-05-10T04:38:13.217051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:31:07.694368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:37.782703+0000","last_change":"2024-05-10T04:38:08.345903+0000","last_active":"2024-05-10T04:38:37.782703+0000","last_peered":"2024-05-10T04:38:37.782703+0000","last_clean":"2024-05-10T04:38:37.782703+0000","last_became_active":"2024-05-10T04:38:08.345371+0000","last_became_peered":"2024-05-10T04:38:08.345371+0000","last_unstale":"2024-05-10T04:38:37.782703+0000","last_undegraded":"2024-05-10T04:38:37.782703+0000","last_fullsized":"2024-05-10T04:38:37.782703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:03.581112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789703+0000","last_change":"2024-05-10T04:38:08.779853+0000","last_active":"2024-05-10T04:38:12.789703+0000","last_peered":"2024-05-10T04:38:12.789703+0000","last_clean":"2024-05-10T04:38:12.789703+0000","last_became_active":"2024-05-10T04:38:08.779531+0000","last_became_peered":"2024-05-10T04:38:08.779531+0000","last_unstale":"2024-05-10T04:38:12.789703+0000","last_undegraded":"2024-05-10T04:38:12.789703+0000","last_fullsized":"2024-05-10T04:38:12.789703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:05.079540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.410643+0000","last_change":"2024-05-10T04:38:12.795970+0000","last_active":"2024-05-10T04:38:13.410643+0000","last_peered":"2024-05-10T04:38:13.410643+0000","last_clean":"2024-05-10T04:38:13.410643+0000","last_became_active":"2024-05-10T04:38:12.795497+0000","last_became_peered":"2024-05-10T04:38:12.795497+0000","last_unstale":"2024-05-10T04:38:13.410643+0000","last_undegraded":"2024-05-10T04:38:13.410643+0000","last_fullsized":"2024-05-10T04:38:13.410643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:11.174672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.970885+0000","last_change":"2024-05-10T04:38:08.343268+0000","last_active":"2024-05-10T04:38:34.970885+0000","last_peered":"2024-05-10T04:38:34.970885+0000","last_clean":"2024-05-10T04:38:34.970885+0000","last_became_active":"2024-05-10T04:38:08.342952+0000","last_became_peered":"2024-05-10T04:38:08.342952+0000","last_unstale":"2024-05-10T04:38:34.970885+0000","last_undegraded":"2024-05-10T04:38:34.970885+0000","last_fullsized":"2024-05-10T04:38:34.970885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:49.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788390+0000","last_change":"2024-05-10T04:38:08.784276+0000","last_active":"2024-05-10T04:38:12.788390+0000","last_peered":"2024-05-10T04:38:12.788390+0000","last_clean":"2024-05-10T04:38:12.788390+0000","last_became_active":"2024-05-10T04:38:08.783951+0000","last_became_peered":"2024-05-10T04:38:08.783951+0000","last_unstale":"2024-05-10T04:38:12.788390+0000","last_undegraded":"2024-05-10T04:38:12.788390+0000","last_fullsized":"2024-05-10T04:38:12.788390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:45:43.992567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220551+0000","last_change":"2024-05-10T04:38:12.811689+0000","last_active":"2024-05-10T04:38:13.220551+0000","last_peered":"2024-05-10T04:38:13.220551+0000","last_clean":"2024-05-10T04:38:13.220551+0000","last_became_active":"2024-05-10T04:38:12.811138+0000","last_became_peered":"2024-05-10T04:38:12.811138+0000","last_unstale":"2024-05-10T04:38:13.220551+0000","last_undegraded":"2024-05-10T04:38:13.220551+0000","last_fullsized":"2024-05-10T04:38:13.220551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:09.291137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410146+0000","last_change":"2024-05-10T04:38:08.348738+0000","last_active":"2024-05-10T04:38:32.410146+0000","last_peered":"2024-05-10T04:38:32.410146+0000","last_clean":"2024-05-10T04:38:32.410146+0000","last_became_active":"2024-05-10T04:38:08.348382+0000","last_became_peered":"2024-05-10T04:38:08.348382+0000","last_unstale":"2024-05-10T04:38:32.410146+0000","last_undegraded":"2024-05-10T04:38:32.410146+0000","last_fullsized":"2024-05-10T04:38:32.410146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:02.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.156551+0000","last_change":"2024-05-10T04:38:08.781483+0000","last_active":"2024-05-10T04:38:13.156551+0000","last_peered":"2024-05-10T04:38:13.156551+0000","last_clean":"2024-05-10T04:38:13.156551+0000","last_became_active":"2024-05-10T04:38:08.777713+0000","last_became_peered":"2024-05-10T04:38:08.777713+0000","last_unstale":"2024-05-10T04:38:13.156551+0000","last_undegraded":"2024-05-10T04:38:13.156551+0000","last_fullsized":"2024-05-10T04:38:13.156551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:41.050901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.930627+0000","last_change":"2024-05-10T04:38:12.800656+0000","last_active":"2024-05-10T04:38:13.930627+0000","last_peered":"2024-05-10T04:38:13.930627+0000","last_clean":"2024-05-10T04:38:13.930627+0000","last_became_active":"2024-05-10T04:38:12.800191+0000","last_became_peered":"2024-05-10T04:38:12.800191+0000","last_unstale":"2024-05-10T04:38:13.930627+0000","last_undegraded":"2024-05-10T04:38:13.930627+0000","last_fullsized":"2024-05-10T04:38:13.930627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:47.017642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.968707+0000","last_change":"2024-05-10T04:38:08.344017+0000","last_active":"2024-05-10T04:38:34.968707+0000","last_peered":"2024-05-10T04:38:34.968707+0000","last_clean":"2024-05-10T04:38:34.968707+0000","last_became_active":"2024-05-10T04:38:08.343646+0000","last_became_peered":"2024-05-10T04:38:08.343646+0000","last_unstale":"2024-05-10T04:38:34.968707+0000","last_undegraded":"2024-05-10T04:38:34.968707+0000","last_fullsized":"2024-05-10T04:38:34.968707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:02:30.455019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786221+0000","last_change":"2024-05-10T04:38:08.779065+0000","last_active":"2024-05-10T04:38:12.786221+0000","last_peered":"2024-05-10T04:38:12.786221+0000","last_clean":"2024-05-10T04:38:12.786221+0000","last_became_active":"2024-05-10T04:38:08.778685+0000","last_became_peered":"2024-05-10T04:38:08.778685+0000","last_unstale":"2024-05-10T04:38:12.786221+0000","last_undegraded":"2024-05-10T04:38:12.786221+0000","last_fullsized":"2024-05-10T04:38:12.786221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:25:58.780080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.353532+0000","last_change":"2024-05-10T04:38:12.801524+0000","last_active":"2024-05-10T04:38:13.353532+0000","last_peered":"2024-05-10T04:38:13.353532+0000","last_clean":"2024-05-10T04:38:13.353532+0000","last_became_active":"2024-05-10T04:38:12.800777+0000","last_became_peered":"2024-05-10T04:38:12.800777+0000","last_unstale":"2024-05-10T04:38:13.353532+0000","last_undegraded":"2024-05-10T04:38:13.353532+0000","last_fullsized":"2024-05-10T04:38:13.353532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:14.139236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:35.277600+0000","last_change":"2024-05-10T04:38:08.344785+0000","last_active":"2024-05-10T04:38:35.277600+0000","last_peered":"2024-05-10T04:38:35.277600+0000","last_clean":"2024-05-10T04:38:35.277600+0000","last_became_active":"2024-05-10T04:38:08.344337+0000","last_became_peered":"2024-05-10T04:38:08.344337+0000","last_unstale":"2024-05-10T04:38:35.277600+0000","last_undegraded":"2024-05-10T04:38:35.277600+0000","last_fullsized":"2024-05-10T04:38:35.277600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:47.682532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788214+0000","last_change":"2024-05-10T04:38:08.768686+0000","last_active":"2024-05-10T04:38:12.788214+0000","last_peered":"2024-05-10T04:38:12.788214+0000","last_clean":"2024-05-10T04:38:12.788214+0000","last_became_active":"2024-05-10T04:38:08.768317+0000","last_became_peered":"2024-05-10T04:38:08.768317+0000","last_unstale":"2024-05-10T04:38:12.788214+0000","last_undegraded":"2024-05-10T04:38:12.788214+0000","last_fullsized":"2024-05-10T04:38:12.788214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:42:33.670912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.220479+0000","last_change":"2024-05-10T04:38:12.809825+0000","last_active":"2024-05-10T04:38:13.220479+0000","last_peered":"2024-05-10T04:38:13.220479+0000","last_clean":"2024-05-10T04:38:13.220479+0000","last_became_active":"2024-05-10T04:38:12.809378+0000","last_became_peered":"2024-05-10T04:38:12.809378+0000","last_unstale":"2024-05-10T04:38:13.220479+0000","last_undegraded":"2024-05-10T04:38:13.220479+0000","last_fullsized":"2024-05-10T04:38:13.220479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:28.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:32.410164+0000","last_change":"2024-05-10T04:38:08.347871+0000","last_active":"2024-05-10T04:38:32.410164+0000","last_peered":"2024-05-10T04:38:32.410164+0000","last_clean":"2024-05-10T04:38:32.410164+0000","last_became_active":"2024-05-10T04:38:08.347618+0000","last_became_peered":"2024-05-10T04:38:08.347618+0000","last_unstale":"2024-05-10T04:38:32.410164+0000","last_undegraded":"2024-05-10T04:38:32.410164+0000","last_fullsized":"2024-05-10T04:38:32.410164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:09:42.060082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787115+0000","last_change":"2024-05-10T04:38:08.779191+0000","last_active":"2024-05-10T04:38:12.787115+0000","last_peered":"2024-05-10T04:38:12.787115+0000","last_clean":"2024-05-10T04:38:12.787115+0000","last_became_active":"2024-05-10T04:38:08.778829+0000","last_became_peered":"2024-05-10T04:38:08.778829+0000","last_unstale":"2024-05-10T04:38:12.787115+0000","last_undegraded":"2024-05-10T04:38:12.787115+0000","last_fullsized":"2024-05-10T04:38:12.787115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:07.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.931626+0000","last_change":"2024-05-10T04:38:12.788693+0000","last_active":"2024-05-10T04:38:13.931626+0000","last_peered":"2024-05-10T04:38:13.931626+0000","last_clean":"2024-05-10T04:38:13.931626+0000","last_became_active":"2024-05-10T04:38:12.788156+0000","last_became_peered":"2024-05-10T04:38:12.788156+0000","last_unstale":"2024-05-10T04:38:13.931626+0000","last_undegraded":"2024-05-10T04:38:13.931626+0000","last_fullsized":"2024-05-10T04:38:13.931626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:30.119523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:39:02.033680+0000","last_change":"2024-05-10T04:38:08.344211+0000","last_active":"2024-05-10T04:39:02.033680+0000","last_peered":"2024-05-10T04:39:02.033680+0000","last_clean":"2024-05-10T04:39:02.033680+0000","last_became_active":"2024-05-10T04:38:08.343443+0000","last_became_peered":"2024-05-10T04:38:08.343443+0000","last_unstale":"2024-05-10T04:39:02.033680+0000","last_undegraded":"2024-05-10T04:39:02.033680+0000","last_fullsized":"2024-05-10T04:39:02.033680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:33:06.960942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787186+0000","last_change":"2024-05-10T04:38:08.780352+0000","last_active":"2024-05-10T04:38:12.787186+0000","last_peered":"2024-05-10T04:38:12.787186+0000","last_clean":"2024-05-10T04:38:12.787186+0000","last_became_active":"2024-05-10T04:38:08.779779+0000","last_became_peered":"2024-05-10T04:38:08.779779+0000","last_unstale":"2024-05-10T04:38:12.787186+0000","last_undegraded":"2024-05-10T04:38:12.787186+0000","last_fullsized":"2024-05-10T04:38:12.787186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:54:31.973896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.340967+0000","last_change":"2024-05-10T04:38:12.805310+0000","last_active":"2024-05-10T04:38:13.340967+0000","last_peered":"2024-05-10T04:38:13.340967+0000","last_clean":"2024-05-10T04:38:13.340967+0000","last_became_active":"2024-05-10T04:38:12.804827+0000","last_became_peered":"2024-05-10T04:38:12.804827+0000","last_unstale":"2024-05-10T04:38:13.340967+0000","last_undegraded":"2024-05-10T04:38:13.340967+0000","last_fullsized":"2024-05-10T04:38:13.340967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:01.655788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:33.963212+0000","last_change":"2024-05-10T04:38:08.343813+0000","last_active":"2024-05-10T04:38:33.963212+0000","last_peered":"2024-05-10T04:38:33.963212+0000","last_clean":"2024-05-10T04:38:33.963212+0000","last_became_active":"2024-05-10T04:38:08.343309+0000","last_became_peered":"2024-05-10T04:38:08.343309+0000","last_unstale":"2024-05-10T04:38:33.963212+0000","last_undegraded":"2024-05-10T04:38:33.963212+0000","last_fullsized":"2024-05-10T04:38:33.963212+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:07.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786851+0000","last_change":"2024-05-10T04:38:08.780598+0000","last_active":"2024-05-10T04:38:12.786851+0000","last_peered":"2024-05-10T04:38:12.786851+0000","last_clean":"2024-05-10T04:38:12.786851+0000","last_became_active":"2024-05-10T04:38:08.775367+0000","last_became_peered":"2024-05-10T04:38:08.775367+0000","last_unstale":"2024-05-10T04:38:12.786851+0000","last_undegraded":"2024-05-10T04:38:12.786851+0000","last_fullsized":"2024-05-10T04:38:12.786851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:33.826116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:13.929645+0000","last_change":"2024-05-10T04:38:12.795975+0000","last_active":"2024-05-10T04:38:13.929645+0000","last_peered":"2024-05-10T04:38:13.929645+0000","last_clean":"2024-05-10T04:38:13.929645+0000","last_became_active":"2024-05-10T04:38:12.795582+0000","last_became_peered":"2024-05-10T04:38:12.795582+0000","last_unstale":"2024-05-10T04:38:13.929645+0000","last_undegraded":"2024-05-10T04:38:13.929645+0000","last_fullsized":"2024-05-10T04:38:13.929645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:42:30.734833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-10T04:38:34.969664+0000","last_change":"2024-05-10T04:38:08.346338+0000","last_active":"2024-05-10T04:38:34.969664+0000","last_peered":"2024-05-10T04:38:34.969664+0000","last_clean":"2024-05-10T04:38:34.969664+0000","last_became_active":"2024-05-10T04:38:08.345994+0000","last_became_peered":"2024-05-10T04:38:08.345994+0000","last_unstale":"2024-05-10T04:38:34.969664+0000","last_undegraded":"2024-05-10T04:38:34.969664+0000","last_fullsized":"2024-05-10T04:38:34.969664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:22.962514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787140+0000","last_change":"2024-05-10T04:38:08.780925+0000","last_active":"2024-05-10T04:38:12.787140+0000","last_peered":"2024-05-10T04:38:12.787140+0000","last_clean":"2024-05-10T04:38:12.787140+0000","last_became_active":"2024-05-10T04:38:08.778777+0000","last_became_peered":"2024-05-10T04:38:08.778777+0000","last_unstale":"2024-05-10T04:38:12.787140+0000","last_undegraded":"2024-05-10T04:38:12.787140+0000","last_fullsized":"2024-05-10T04:38:12.787140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:57.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786327+0000","last_change":"2024-05-10T04:38:08.780281+0000","last_active":"2024-05-10T04:38:12.786327+0000","last_peered":"2024-05-10T04:38:12.786327+0000","last_clean":"2024-05-10T04:38:12.786327+0000","last_became_active":"2024-05-10T04:38:08.779927+0000","last_became_peered":"2024-05-10T04:38:08.779927+0000","last_unstale":"2024-05-10T04:38:12.786327+0000","last_undegraded":"2024-05-10T04:38:12.786327+0000","last_fullsized":"2024-05-10T04:38:12.786327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:33.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789688+0000","last_change":"2024-05-10T04:38:08.784916+0000","last_active":"2024-05-10T04:38:12.789688+0000","last_peered":"2024-05-10T04:38:12.789688+0000","last_clean":"2024-05-10T04:38:12.789688+0000","last_became_active":"2024-05-10T04:38:08.784544+0000","last_became_peered":"2024-05-10T04:38:08.784544+0000","last_unstale":"2024-05-10T04:38:12.789688+0000","last_undegraded":"2024-05-10T04:38:12.789688+0000","last_fullsized":"2024-05-10T04:38:12.789688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:03.669897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786689+0000","last_change":"2024-05-10T04:38:08.769149+0000","last_active":"2024-05-10T04:38:12.786689+0000","last_peered":"2024-05-10T04:38:12.786689+0000","last_clean":"2024-05-10T04:38:12.786689+0000","last_became_active":"2024-05-10T04:38:08.768808+0000","last_became_peered":"2024-05-10T04:38:08.768808+0000","last_unstale":"2024-05-10T04:38:12.786689+0000","last_undegraded":"2024-05-10T04:38:12.786689+0000","last_fullsized":"2024-05-10T04:38:12.786689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:42:46.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.787125+0000","last_change":"2024-05-10T04:38:08.784411+0000","last_active":"2024-05-10T04:38:12.787125+0000","last_peered":"2024-05-10T04:38:12.787125+0000","last_clean":"2024-05-10T04:38:12.787125+0000","last_became_active":"2024-05-10T04:38:08.784064+0000","last_became_peered":"2024-05-10T04:38:08.784064+0000","last_unstale":"2024-05-10T04:38:12.787125+0000","last_undegraded":"2024-05-10T04:38:12.787125+0000","last_fullsized":"2024-05-10T04:38:12.787125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:32:46.493888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782409+0000","last_change":"2024-05-10T04:38:08.779643+0000","last_active":"2024-05-10T04:38:12.782409+0000","last_peered":"2024-05-10T04:38:12.782409+0000","last_clean":"2024-05-10T04:38:12.782409+0000","last_became_active":"2024-05-10T04:38:08.779291+0000","last_became_peered":"2024-05-10T04:38:08.779291+0000","last_unstale":"2024-05-10T04:38:12.782409+0000","last_undegraded":"2024-05-10T04:38:12.782409+0000","last_fullsized":"2024-05-10T04:38:12.782409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:24.535515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786192+0000","last_change":"2024-05-10T04:38:08.789382+0000","last_active":"2024-05-10T04:38:12.786192+0000","last_peered":"2024-05-10T04:38:12.786192+0000","last_clean":"2024-05-10T04:38:12.786192+0000","last_became_active":"2024-05-10T04:38:08.789003+0000","last_became_peered":"2024-05-10T04:38:08.789003+0000","last_unstale":"2024-05-10T04:38:12.786192+0000","last_undegraded":"2024-05-10T04:38:12.786192+0000","last_fullsized":"2024-05-10T04:38:12.786192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:03.031319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788582+0000","last_change":"2024-05-10T04:38:08.777290+0000","last_active":"2024-05-10T04:38:12.788582+0000","last_peered":"2024-05-10T04:38:12.788582+0000","last_clean":"2024-05-10T04:38:12.788582+0000","last_became_active":"2024-05-10T04:38:08.776331+0000","last_became_peered":"2024-05-10T04:38:08.776331+0000","last_unstale":"2024-05-10T04:38:12.788582+0000","last_undegraded":"2024-05-10T04:38:12.788582+0000","last_fullsized":"2024-05-10T04:38:12.788582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:51.074991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.789825+0000","last_change":"2024-05-10T04:38:08.780937+0000","last_active":"2024-05-10T04:38:12.789825+0000","last_peered":"2024-05-10T04:38:12.789825+0000","last_clean":"2024-05-10T04:38:12.789825+0000","last_became_active":"2024-05-10T04:38:08.780620+0000","last_became_peered":"2024-05-10T04:38:08.780620+0000","last_unstale":"2024-05-10T04:38:12.789825+0000","last_undegraded":"2024-05-10T04:38:12.789825+0000","last_fullsized":"2024-05-10T04:38:12.789825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:27.697571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.781208+0000","last_change":"2024-05-10T04:38:08.773314+0000","last_active":"2024-05-10T04:38:12.781208+0000","last_peered":"2024-05-10T04:38:12.781208+0000","last_clean":"2024-05-10T04:38:12.781208+0000","last_became_active":"2024-05-10T04:38:08.772973+0000","last_became_peered":"2024-05-10T04:38:08.772973+0000","last_unstale":"2024-05-10T04:38:12.781208+0000","last_undegraded":"2024-05-10T04:38:12.781208+0000","last_fullsized":"2024-05-10T04:38:12.781208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:53.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.788890+0000","last_change":"2024-05-10T04:38:08.775632+0000","last_active":"2024-05-10T04:38:12.788890+0000","last_peered":"2024-05-10T04:38:12.788890+0000","last_clean":"2024-05-10T04:38:12.788890+0000","last_became_active":"2024-05-10T04:38:08.775308+0000","last_became_peered":"2024-05-10T04:38:08.775308+0000","last_unstale":"2024-05-10T04:38:12.788890+0000","last_undegraded":"2024-05-10T04:38:12.788890+0000","last_fullsized":"2024-05-10T04:38:12.788890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:58.758863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.783887+0000","last_change":"2024-05-10T04:38:08.787354+0000","last_active":"2024-05-10T04:38:12.783887+0000","last_peered":"2024-05-10T04:38:12.783887+0000","last_clean":"2024-05-10T04:38:12.783887+0000","last_became_active":"2024-05-10T04:38:08.786645+0000","last_became_peered":"2024-05-10T04:38:08.786645+0000","last_unstale":"2024-05-10T04:38:12.783887+0000","last_undegraded":"2024-05-10T04:38:12.783887+0000","last_fullsized":"2024-05-10T04:38:12.783887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:35.060271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.782372+0000","last_change":"2024-05-10T04:38:08.769233+0000","last_active":"2024-05-10T04:38:12.782372+0000","last_peered":"2024-05-10T04:38:12.782372+0000","last_clean":"2024-05-10T04:38:12.782372+0000","last_became_active":"2024-05-10T04:38:08.768936+0000","last_became_peered":"2024-05-10T04:38:08.768936+0000","last_unstale":"2024-05-10T04:38:12.782372+0000","last_undegraded":"2024-05-10T04:38:12.782372+0000","last_fullsized":"2024-05-10T04:38:12.782372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:08.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-10T04:38:12.786960+0000","last_change":"2024-05-10T04:38:08.788189+0000","last_active":"2024-05-10T04:38:12.786960+0000","last_peered":"2024-05-10T04:38:12.786960+0000","last_clean":"2024-05-10T04:38:12.786960+0000","last_became_active":"2024-05-10T04:38:08.787806+0000","last_became_peered":"2024-05-10T04:38:08.787806+0000","last_unstale":"2024-05-10T04:38:12.786960+0000","last_undegraded":"2024-05-10T04:38:12.786960+0000","last_fullsized":"2024-05-10T04:38:12.786960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:00:51.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280050,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68283,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":333,"num_read_kb":242,"num_write":233,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":208153,"data_compressed":4791,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481814,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":942080,"data_stored":466157,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.39400000000000002}]},{"osd":1,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.63900000000000001}]},{"osd":2,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.70099999999999996}]},{"osd":5,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.82899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":9,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":1.012}]},{"osd":10,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.84299999999999997}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":1113909,"data_compressed":48903,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91500000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.89800000000000002}]},{"osd":9,"last update":"Fri May 10 04:38:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.84799999999999998}]},{"osd":11,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.88400000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":469838,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.44600000000000001,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55300000000000005,"5min":1.034,"15min":1.034},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64900000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.30099999999999999}]},{"osd":1,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90400000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.56599999999999995,"15min":0.58699999999999997},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.65900000000000003,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59699999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80500000000000005,"5min":1.101,"15min":1.101},"last":0.372},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.433}]},{"osd":5,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":1.411,"15min":1.411},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.29,"15min":1.29},"last":0.495}]},{"osd":8,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93999999999999995,"5min":1.53,"15min":1.53},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91500000000000004,"5min":1.502,"15min":1.502},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":962560,"data_stored":466120,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.497,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.66900000000000004,"15min":0.64400000000000002},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.59399999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.86399999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.70199999999999996},"min":{"1min":0.59099999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.67400000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.877}]},{"osd":7,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Fri May 10 04:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":0.96099999999999997}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":1.04}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364835,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":1015808,"data_stored":1062082,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.532,"5min":3.532,"15min":3.532},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69399999999999995,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.3759999999999999,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.67600000000000005,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.60799999999999998,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.4209999999999998,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.625,"15min":0.59899999999999998},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.9609999999999999,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.622,"15min":0.60299999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6179999999999999,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.755}]},{"osd":5,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.72899999999999998,"15min":0.71399999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4860000000000002,"5min":3.4860000000000002,"15min":3.4860000000000002},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri May 10 04:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.70699999999999996,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":3.996,"5min":3.996,"15min":3.996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.626}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":2.028,"15min":2.028},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91400000000000003},"min":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95299999999999996,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.90400000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":4.2670000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":0.95299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528359,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":461198,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40300000000000002,"15min":0.40699999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.40400000000000003}]},{"osd":1,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77800000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.627},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51900000000000002,"15min":0.52500000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48799999999999999}]},{"osd":4,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.63,"15min":0.63},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri May 10 04:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri May 10 04:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54500000000000004}]},{"osd":8,"last update":"Fri May 10 04:38:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.61699999999999999}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70599999999999996}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":1007616,"data_stored":525540,"data_compressed":13284,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53500000000000003,"15min":0.5},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.435,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.47899999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri May 10 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57899999999999996,"15min":0.56799999999999995},"min":{"1min":0.39400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.80700000000000005}]},{"osd":4,"last update":"Fri May 10 04:38:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46100000000000002,"15min":0.438},"min":{"1min":0.33200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.47999999999999998,"15min":0.44700000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.39100000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.58499999999999996,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":1.123,"15min":1.123},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.69199999999999995,"15min":0.64900000000000002},"min":{"1min":0.55100000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1100000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.629,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.74399999999999999}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80000000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.008,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.49299999999999999}]},{"osd":8,"last update":"Fri May 10 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.52800000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.626,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1040000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.60699999999999998}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.58399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78400000000000003,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0049999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.64600000000000002}]},{"osd":10,"last update":"Fri May 10 04:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.72499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":463047,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.673,"5min":2.673,"15min":2.673},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.55800000000000005,"15min":0.48699999999999999},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.621}]},{"osd":2,"last update":"Fri May 10 04:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.68500000000000005,"5min":1.123,"15min":1.123},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri May 10 04:37:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.41399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.37}]},{"osd":4,"last update":"Fri May 10 04:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0089999999999999,"5min":1.093,"15min":1.093},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri May 10 04:38:07 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52500000000000002,"15min":0.53100000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.41599999999999998}]},{"osd":7,"last update":"Fri May 10 04:37:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.61199999999999999}]},{"osd":8,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.58899999999999997}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69599999999999995,"15min":0.67300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.7190000000000003,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.56200000000000006}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77300000000000002}]},{"osd":11,"last update":"Fri May 10 04:38:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.68400000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":526105,"data_compressed":13284,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.502,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.86899999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.47999999999999998,"15min":0.45600000000000002},"min":{"1min":0.34499999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.67800000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44}]},{"osd":1,"last update":"Fri May 10 04:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54200000000000004,"15min":0.53000000000000003},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54500000000000004},"min":{"1min":0.39700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.40500000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80000000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78200000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":6,"last update":"Fri May 10 04:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri May 10 04:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.75700000000000001}]},{"osd":8,"last update":"Fri May 10 04:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58099999999999996}]},{"osd":9,"last update":"Fri May 10 04:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68899999999999995}]},{"osd":10,"last update":"Fri May 10 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149745,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":460954,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35799999999999998,"15min":0.34000000000000002},"min":{"1min":0.31900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.69799999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.48199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52700000000000002,"15min":0.52100000000000002},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.71899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.40500000000000003}]},{"osd":4,"last update":"Fri May 10 04:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.45500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81499999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.501}]},{"osd":5,"last update":"Fri May 10 04:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.379},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri May 10 04:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78300000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri May 10 04:38:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.69599999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96799999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60499999999999998}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91600000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.64800000000000002}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.621}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920814,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":465272,"data_compressed":11687,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":1.143,"15min":1.143},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.36099999999999999},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.441},{"interface":"front","average":{"1min":0.434,"5min":0.38200000000000001,"15min":0.36199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.627,"5min":0.748,"15min":0.748},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":1.03,"15min":1.03},"last":0.71899999999999997}]},{"osd":3,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88300000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90700000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri May 10 04:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67200000000000004,"5min":1.069,"15min":1.069},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.86499999999999999,"5min":1.079,"15min":1.079},"last":0.622}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.41399999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56599999999999995}]},{"osd":8,"last update":"Fri May 10 04:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.52800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88500000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.86399999999999999}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.80300000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73199999999999998}]},{"osd":10,"last update":"Fri May 10 04:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":1024000,"data_stored":1057760,"data_compressed":47306,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68999999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85499999999999998,"5min":1.008,"15min":1.008},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81599999999999995,"5min":1.089,"15min":1.089},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84499999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.59599999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89600000000000002,"5min":1.002,"15min":1.002},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri May 10 04:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.626},"min":{"1min":0.48399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.80600000000000005}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63700000000000001}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri May 10 04:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.745},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65016,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65581,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63348,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:39:08.217 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-10T04:39:08.217 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-10T04:39:08.217 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-10T04:39:08.217 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-10T04:39:08.443 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:08.697 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:08 smithi150 ceph-mon[42404]: pgmap v254: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:08.697 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.697 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.697 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:39:08.697 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:39:08.697 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:08 smithi161 ceph-mon[46912]: pgmap v254: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:39:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:39:08.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:08 smithi181 ceph-mon[47314]: pgmap v254: 129 pgs: 129 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-05-10T04:39:08.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:08.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:39:08.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:39:08.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 -- 172.21.15.150:0/1141093677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010106080 msgr2=0x7fa01010d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 --2- 172.21.15.150:0/1141093677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010106080 0x7fa01010d9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b0a0 tx=0x7fa00800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 -- 172.21.15.150:0/1141093677 shutdown_connections 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 --2- 172.21.15.150:0/1141093677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010106080 0x7fa01010d9e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 --2- 172.21.15.150:0/1141093677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa010100e00 0x7fa010101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 --2- 172.21.15.150:0/1141093677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa010105730 0x7fa010105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 -- 172.21.15.150:0/1141093677 >> 172.21.15.150:0/1141093677 conn(0x7fa0100fc7a0 msgr2=0x7fa0100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:09.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 -- 172.21.15.150:0/1141093677 shutdown_connections 2024-05-10T04:39:09.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.407+0000 7fa01868f700 1 -- 172.21.15.150:0/1141093677 wait complete. 2024-05-10T04:39:09.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 Processor -- start 2024-05-10T04:39:09.410 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 -- start start 2024-05-10T04:39:09.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa010100e00 0x7fa0101995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:09.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010105730 0x7fa010199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa010106080 0x7fa0101a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01019a280 con 0x7fa010105730 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0101a49a0 con 0x7fa010106080 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01868f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0101a4b60 con 0x7fa010100e00 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa015c2a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010105730 0x7fa010199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa015c2a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010105730 0x7fa010199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59168/0 (socket says 172.21.15.150:59168) 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa015c2a700 1 -- 172.21.15.150:0/1425611922 learned_addr learned my addr 172.21.15.150:0/1425611922 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.408+0000 7fa01642b700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa010100e00 0x7fa0101995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa016c2c700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa010106080 0x7fa0101a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa015c2a700 1 -- 172.21.15.150:0/1425611922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa010100e00 msgr2=0x7fa0101995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa015c2a700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa010100e00 0x7fa0101995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa015c2a700 1 -- 172.21.15.150:0/1425611922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa010106080 msgr2=0x7fa0101a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa015c2a700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa010106080 0x7fa0101a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa015c2a700 1 -- 172.21.15.150:0/1425611922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008009d00 con 0x7fa010105730 2024-05-10T04:39:09.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa015c2a700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010105730 0x7fa010199af0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa00000d980 tx=0x7fa00000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:09.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa0077fe700 1 -- 172.21.15.150:0/1425611922 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000009930 con 0x7fa010105730 2024-05-10T04:39:09.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0101a4e40 con 0x7fa010105730 2024-05-10T04:39:09.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa0077fe700 1 -- 172.21.15.150:0/1425611922 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa000010460 con 0x7fa010105730 2024-05-10T04:39:09.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa0077fe700 1 -- 172.21.15.150:0/1425611922 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00000f620 con 0x7fa010105730 2024-05-10T04:39:09.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.409+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0101a5280 con 0x7fa010105730 2024-05-10T04:39:09.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.410+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa010109130 con 0x7fa010105730 2024-05-10T04:39:09.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.412+0000 7fa0077fe700 1 -- 172.21.15.150:0/1425611922 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa0000108d0 con 0x7fa010105730 2024-05-10T04:39:09.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.412+0000 7fa0077fe700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9ffc0a11b0 0x7f9ffc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:09.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.412+0000 7fa0077fe700 1 -- 172.21.15.150:0/1425611922 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa0000cc680 con 0x7fa010105730 2024-05-10T04:39:09.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.412+0000 7fa01642b700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9ffc0a11b0 0x7f9ffc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:09.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.413+0000 7fa01642b700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9ffc0a11b0 0x7f9ffc0a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa00c005950 tx=0x7fa00c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:09.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.416+0000 7fa0077fe700 1 -- 172.21.15.150:0/1425611922 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa000095ac0 con 0x7fa010105730 2024-05-10T04:39:09.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.650+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa0101022b0 con 0x7fa010105730 2024-05-10T04:39:09.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.651+0000 7fa0077fe700 1 -- 172.21.15.150:0/1425611922 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa0101022b0 con 0x7fa010105730 2024-05-10T04:39:09.652 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:09.652 INFO:teuthology.orchestra.run.smithi150.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-10T04:39:09.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9ffc0a11b0 msgr2=0x7f9ffc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:09.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9ffc0a11b0 0x7f9ffc0a36c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa00c005950 tx=0x7fa00c00b410 comp rx=0 tx=0).stop 2024-05-10T04:39:09.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010105730 msgr2=0x7fa010199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:09.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010105730 0x7fa010199af0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa00000d980 tx=0x7fa00000dc90 comp rx=0 tx=0).stop 2024-05-10T04:39:09.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 shutdown_connections 2024-05-10T04:39:09.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa010106080 0x7fa0101a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9ffc0a11b0 0x7f9ffc0a36c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa010100e00 0x7fa0101995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 --2- 172.21.15.150:0/1425611922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa010105730 0x7fa010199af0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:09.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 >> 172.21.15.150:0/1425611922 conn(0x7fa0100fc7a0 msgr2=0x7fa010104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:09.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 shutdown_connections 2024-05-10T04:39:09.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:09.653+0000 7fa01868f700 1 -- 172.21.15.150:0/1425611922 wait complete. 2024-05-10T04:39:09.678 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:09 smithi150 ceph-mon[42404]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:09.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:09 smithi161 ceph-mon[46912]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:09.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:09 smithi181 ceph-mon[47314]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:10.038 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-10T04:39:10.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:10.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 -- 172.21.15.150:0/256920535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1031a0 msgr2=0x7ff10c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 --2- 172.21.15.150:0/256920535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1031a0 0x7ff10c1035b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009990 tx=0x7ff0fc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 -- 172.21.15.150:0/256920535 shutdown_connections 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 --2- 172.21.15.150:0/256920535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff10c100530 0x7ff10c10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 --2- 172.21.15.150:0/256920535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c1031a0 0x7ff10c1035b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 --2- 172.21.15.150:0/256920535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10c0ffbe0 0x7ff10c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 -- 172.21.15.150:0/256920535 >> 172.21.15.150:0/256920535 conn(0x7ff10c0fb8b0 msgr2=0x7ff10c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 -- 172.21.15.150:0/256920535 shutdown_connections 2024-05-10T04:39:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 -- 172.21.15.150:0/256920535 wait complete. 2024-05-10T04:39:10.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.174+0000 7ff111466700 1 Processor -- start 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff111466700 1 -- start start 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff111466700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff10c0ffbe0 0x7ff10c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff111466700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c100530 0x7ff10c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff111466700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10c1031a0 0x7ff10c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff111466700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c19fad0 con 0x7ff10c100530 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff111466700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c19fc40 con 0x7ff10c1031a0 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff111466700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c19fe00 con 0x7ff10c0ffbe0 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c100530 0x7ff10c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c100530 0x7ff10c198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59182/0 (socket says 172.21.15.150:59182) 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 -- 172.21.15.150:0/104627128 learned_addr learned my addr 172.21.15.150:0/104627128 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10affd700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff10c0ffbe0 0x7ff10c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10b7fe700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10c1031a0 0x7ff10c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 -- 172.21.15.150:0/104627128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff10c0ffbe0 msgr2=0x7ff10c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff10c0ffbe0 0x7ff10c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 -- 172.21.15.150:0/104627128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10c1031a0 msgr2=0x7ff10c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10c1031a0 0x7ff10c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.175+0000 7ff10a7fc700 1 -- 172.21.15.150:0/104627128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc0095f0 con 0x7ff10c100530 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.176+0000 7ff10a7fc700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c100530 0x7ff10c198530 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff10000d950 tx=0x7ff10000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.176+0000 7ff0f3fff700 1 -- 172.21.15.150:0/104627128 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100009940 con 0x7ff10c100530 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.176+0000 7ff0f3fff700 1 -- 172.21.15.150:0/104627128 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff100010460 con 0x7ff10c100530 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.176+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c1a00e0 con 0x7ff10c100530 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.176+0000 7ff0f3fff700 1 -- 172.21.15.150:0/104627128 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10000f620 con 0x7ff10c100530 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.176+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c1a05f0 con 0x7ff10c100530 2024-05-10T04:39:10.180 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.177+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10c108ff0 con 0x7ff10c100530 2024-05-10T04:39:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.178+0000 7ff0f3fff700 1 -- 172.21.15.150:0/104627128 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff1000108d0 con 0x7ff10c100530 2024-05-10T04:39:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.178+0000 7ff0f3fff700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0f40a11b0 0x7ff0f40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.179+0000 7ff10affd700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0f40a11b0 0x7ff0f40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.179+0000 7ff0f3fff700 1 -- 172.21.15.150:0/104627128 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff1000ccb60 con 0x7ff10c100530 2024-05-10T04:39:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.179+0000 7ff10affd700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0f40a11b0 0x7ff0f40a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b5c0 tx=0x7ff0fc011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:10.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.182+0000 7ff0f3fff700 1 -- 172.21.15.150:0/104627128 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff100095f40 con 0x7ff10c100530 2024-05-10T04:39:10.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.382+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff10c10d420 con 0x7ff10c100530 2024-05-10T04:39:10.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.383+0000 7ff0f3fff700 1 -- 172.21.15.150:0/104627128 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23993 (secure 0 0 0) 0x7ff100016090 con 0x7ff10c100530 2024-05-10T04:39:10.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:10.385 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":86,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:38:13.213405+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0f40a11b0 msgr2=0x7ff0f40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0f40a11b0 0x7ff0f40a36c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b5c0 tx=0x7ff0fc011040 comp rx=0 tx=0).stop 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c100530 msgr2=0x7ff10c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c100530 0x7ff10c198530 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff10000d950 tx=0x7ff10000dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 shutdown_connections 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10c1031a0 0x7ff10c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0f40a11b0 0x7ff0f40a36c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff10c0ffbe0 0x7ff10c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.385+0000 7ff111466700 1 --2- 172.21.15.150:0/104627128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff10c100530 0x7ff10c198530 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.386+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 >> 172.21.15.150:0/104627128 conn(0x7ff10c0fb8b0 msgr2=0x7ff10c106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.386+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 shutdown_connections 2024-05-10T04:39:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.386+0000 7ff111466700 1 -- 172.21.15.150:0/104627128 wait complete. 2024-05-10T04:39:10.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:39:10.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:39:10.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 -- 172.21.15.150:0/253181585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 msgr2=0x7fd994101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 --2- 172.21.15.150:0/253181585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994101ff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd988009990 tx=0x7fd988009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.545 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 -- 172.21.15.150:0/253181585 shutdown_connections 2024-05-10T04:39:10.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 --2- 172.21.15.150:0/253181585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd994102580 0x7fd994108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 --2- 172.21.15.150:0/253181585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd994100ac0 0x7fd994100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 --2- 172.21.15.150:0/253181585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994101ff0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 -- 172.21.15.150:0/253181585 >> 172.21.15.150:0/253181585 conn(0x7fd9940fc0d0 msgr2=0x7fd9940fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:10.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 -- 172.21.15.150:0/253181585 shutdown_connections 2024-05-10T04:39:10.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.538+0000 7fd99a843700 1 -- 172.21.15.150:0/253181585 wait complete. 2024-05-10T04:39:10.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 Processor -- start 2024-05-10T04:39:10.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 -- start start 2024-05-10T04:39:10.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd994100ac0 0x7fd994104880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd994102580 0x7fd994103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd994103b40 con 0x7fd994101be0 2024-05-10T04:39:10.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd994103cb0 con 0x7fd994100ac0 2024-05-10T04:39:10.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd99a843700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9941a1f80 con 0x7fd994102580 2024-05-10T04:39:10.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59208/0 (socket says 172.21.15.150:59208) 2024-05-10T04:39:10.548 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 -- 172.21.15.150:0/3603714248 learned_addr learned my addr 172.21.15.150:0/3603714248 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd993fff700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd994100ac0 0x7fd994104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd998de0700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd994102580 0x7fd994103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 -- 172.21.15.150:0/3603714248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd994102580 msgr2=0x7fd994103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd994102580 0x7fd994103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 -- 172.21.15.150:0/3603714248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd994100ac0 msgr2=0x7fd994104880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd994100ac0 0x7fd994104880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd9937fe700 1 -- 172.21.15.150:0/3603714248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880095f0 con 0x7fd994101be0 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.539+0000 7fd993fff700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd994100ac0 0x7fd994104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.540+0000 7fd9937fe700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994102f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098e0 tx=0x7fd988009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.540+0000 7fd9917fa700 1 -- 172.21.15.150:0/3603714248 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98801d070 con 0x7fd994101be0 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.540+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9941a2180 con 0x7fd994101be0 2024-05-10T04:39:10.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.540+0000 7fd9917fa700 1 -- 172.21.15.150:0/3603714248 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98800fbd0 con 0x7fd994101be0 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.540+0000 7fd9917fa700 1 -- 172.21.15.150:0/3603714248 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988017650 con 0x7fd994101be0 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.540+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9941a2600 con 0x7fd994101be0 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.541+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9940694d0 con 0x7fd994101be0 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.542+0000 7fd9917fa700 1 -- 172.21.15.150:0/3603714248 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fd98800fd40 con 0x7fd994101be0 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.542+0000 7fd9917fa700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd97c0a1260 0x7fd97c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.543+0000 7fd993fff700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd97c0a1260 0x7fd97c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.543+0000 7fd9917fa700 1 -- 172.21.15.150:0/3603714248 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd9880ce7e0 con 0x7fd994101be0 2024-05-10T04:39:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.543+0000 7fd993fff700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd97c0a1260 0x7fd97c0a3770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd984005950 tx=0x7fd9840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:10.552 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.546+0000 7fd9917fa700 1 -- 172.21.15.150:0/3603714248 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd988097cb0 con 0x7fd994101be0 2024-05-10T04:39:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:10 smithi150 ceph-mon[42404]: pgmap v255: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T04:39:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:10 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1425611922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T04:39:10.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:10 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/104627128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:10.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.745+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd99410c4c0 con 0x7fd994101be0 2024-05-10T04:39:10.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.746+0000 7fd9917fa700 1 -- 172.21.15.150:0/3603714248 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd98809b2d0 con 0x7fd994101be0 2024-05-10T04:39:10.748 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:39:10.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd97c0a1260 msgr2=0x7fd97c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd97c0a1260 0x7fd97c0a3770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd984005950 tx=0x7fd9840058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 msgr2=0x7fd994102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994102f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098e0 tx=0x7fd988009580 comp rx=0 tx=0).stop 2024-05-10T04:39:10.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 shutdown_connections 2024-05-10T04:39:10.750 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd97c0a1260 0x7fd97c0a3770 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd994102580 0x7fd994103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd994100ac0 0x7fd994104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 --2- 172.21.15.150:0/3603714248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd994101be0 0x7fd994102f20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 >> 172.21.15.150:0/3603714248 conn(0x7fd9940fc0d0 msgr2=0x7fd9940fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:10.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 shutdown_connections 2024-05-10T04:39:10.751 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.749+0000 7fd99a843700 1 -- 172.21.15.150:0/3603714248 wait complete. 2024-05-10T04:39:10.763 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:39:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:10 smithi161 ceph-mon[46912]: pgmap v255: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T04:39:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:10 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1425611922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T04:39:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:10 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/104627128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 -- 172.21.15.150:0/617748948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200104d50 msgr2=0x7f32001071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 --2- 172.21.15.150:0/617748948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200104d50 0x7f32001071f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009990 tx=0x7f31f4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 -- 172.21.15.150:0/617748948 shutdown_connections 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 --2- 172.21.15.150:0/617748948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3200107880 0x7f3200109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 --2- 172.21.15.150:0/617748948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3200069030 0x7f3200104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 --2- 172.21.15.150:0/617748948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200104d50 0x7f32001071f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 -- 172.21.15.150:0/617748948 >> 172.21.15.150:0/617748948 conn(0x7f32000fc0d0 msgr2=0x7f32000fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 -- 172.21.15.150:0/617748948 shutdown_connections 2024-05-10T04:39:10.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.898+0000 7f3205dd6700 1 -- 172.21.15.150:0/617748948 wait complete. 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.899+0000 7f3205dd6700 1 Processor -- start 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.899+0000 7f3205dd6700 1 -- start start 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f3205dd6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200069030 0x7f3200197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f3205dd6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3200104d50 0x7f3200198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f3205dd6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3200107880 0x7f320019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f3205dd6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320019f9f0 con 0x7f3200069030 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f3205dd6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320019fb60 con 0x7f3200107880 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f3205dd6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320019fd20 con 0x7f3200104d50 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200069030 0x7f3200197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200069030 0x7f3200197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59228/0 (socket says 172.21.15.150:59228) 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 -- 172.21.15.150:0/2862874033 learned_addr learned my addr 172.21.15.150:0/2862874033 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:10.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31feffd700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3200104d50 0x7f3200198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31fffff700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3200107880 0x7f320019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 -- 172.21.15.150:0/2862874033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3200104d50 msgr2=0x7f3200198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3200104d50 0x7f3200198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 -- 172.21.15.150:0/2862874033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3200107880 msgr2=0x7f320019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3200107880 0x7f320019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 -- 172.21.15.150:0/2862874033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f40095f0 con 0x7f3200069030 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31ff7fe700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200069030 0x7f3200197f10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e970 tx=0x7f31f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31fcff9700 1 -- 172.21.15.150:0/2862874033 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0009790 con 0x7f3200069030 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.900+0000 7f31fcff9700 1 -- 172.21.15.150:0/2862874033 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31f000eec0 con 0x7f3200069030 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.901+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320019ffa0 con 0x7f3200069030 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.901+0000 7f31fcff9700 1 -- 172.21.15.150:0/2862874033 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0005590 con 0x7f3200069030 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.901+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32001a0450 con 0x7f3200069030 2024-05-10T04:39:10.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.901+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32000fdc80 con 0x7f3200069030 2024-05-10T04:39:10.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:10 smithi181 ceph-mon[47314]: pgmap v255: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-10T04:39:10.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:10 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1425611922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-10T04:39:10.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:10 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/104627128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.903+0000 7f31fcff9700 1 -- 172.21.15.150:0/2862874033 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f31f000cbd0 con 0x7f3200069030 2024-05-10T04:39:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.904+0000 7f31fcff9700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f31e80a1260 0x7f31e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.904+0000 7f31fcff9700 1 -- 172.21.15.150:0/2862874033 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f31f0014070 con 0x7f3200069030 2024-05-10T04:39:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.904+0000 7f31feffd700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f31e80a1260 0x7f31e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.905+0000 7f31feffd700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f31e80a1260 0x7f31e80a3770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b2e0 tx=0x7f31f401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:10.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:10.908+0000 7f31fcff9700 1 -- 172.21.15.150:0/2862874033 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f31f0096510 con 0x7f3200069030 2024-05-10T04:39:11.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.107+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32000694d0 con 0x7f3200069030 2024-05-10T04:39:11.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.107+0000 7f31fcff9700 1 -- 172.21.15.150:0/2862874033 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31f0004ef0 con 0x7f3200069030 2024-05-10T04:39:11.110 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:11.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f31e80a1260 msgr2=0x7f31e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f31e80a1260 0x7f31e80a3770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b2e0 tx=0x7f31f401a040 comp rx=0 tx=0).stop 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200069030 msgr2=0x7f3200197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200069030 0x7f3200197f10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e970 tx=0x7f31f000ec80 comp rx=0 tx=0).stop 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 shutdown_connections 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3200107880 0x7f320019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3200069030 0x7f3200197f10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f31e80a1260 0x7f31e80a3770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 --2- 172.21.15.150:0/2862874033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3200104d50 0x7f3200198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 >> 172.21.15.150:0/2862874033 conn(0x7f32000fc0d0 msgr2=0x7f32001032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 shutdown_connections 2024-05-10T04:39:11.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.110+0000 7f3205dd6700 1 -- 172.21.15.150:0/2862874033 wait complete. 2024-05-10T04:39:11.125 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:39:11.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 -- 172.21.15.150:0/1668899701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa460105020 msgr2=0x7fa460107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 --2- 172.21.15.150:0/1668899701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa460105020 0x7fa460107440 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa450009990 tx=0x7fa450009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 -- 172.21.15.150:0/1668899701 shutdown_connections 2024-05-10T04:39:11.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 --2- 172.21.15.150:0/1668899701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460107980 0x7fa460109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 --2- 172.21.15.150:0/1668899701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4601026c0 0x7fa460104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 --2- 172.21.15.150:0/1668899701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa460105020 0x7fa460107440 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 -- 172.21.15.150:0/1668899701 >> 172.21.15.150:0/1668899701 conn(0x7fa4600fc0e0 msgr2=0x7fa4600fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:11.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 -- 172.21.15.150:0/1668899701 shutdown_connections 2024-05-10T04:39:11.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.259+0000 7fa46606a700 1 -- 172.21.15.150:0/1668899701 wait complete. 2024-05-10T04:39:11.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 Processor -- start 2024-05-10T04:39:11.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 -- start start 2024-05-10T04:39:11.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4601026c0 0x7fa460101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460105020 0x7fa4601005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa460107980 0x7fa460100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460101120 con 0x7fa4601026c0 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460101290 con 0x7fa460107980 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.260+0000 7fa46606a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460101450 con 0x7fa460105020 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4601026c0 0x7fa460101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4601026c0 0x7fa460101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59244/0 (socket says 172.21.15.150:59244) 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 -- 172.21.15.150:0/3713528357 learned_addr learned my addr 172.21.15.150:0/3713528357 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45effd700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460105020 0x7fa4601005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45ffff700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa460107980 0x7fa460100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 -- 172.21.15.150:0/3713528357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460105020 msgr2=0x7fa4601005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460105020 0x7fa4601005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 -- 172.21.15.150:0/3713528357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa460107980 msgr2=0x7fa460100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa460107980 0x7fa460100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 -- 172.21.15.150:0/3713528357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500095f0 con 0x7fa4601026c0 2024-05-10T04:39:11.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45ffff700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa460107980 0x7fa460100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:11.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45f7fe700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4601026c0 0x7fa460101ee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa45400e970 tx=0x7fa45400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:11.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45cff9700 1 -- 172.21.15.150:0/3713528357 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454009790 con 0x7fa4601026c0 2024-05-10T04:39:11.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45cff9700 1 -- 172.21.15.150:0/3713528357 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa454004cf0 con 0x7fa4601026c0 2024-05-10T04:39:11.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4601a21f0 con 0x7fa4601026c0 2024-05-10T04:39:11.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa45cff9700 1 -- 172.21.15.150:0/3713528357 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454005680 con 0x7fa4601026c0 2024-05-10T04:39:11.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.261+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4601a2700 con 0x7fa4601026c0 2024-05-10T04:39:11.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.262+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4600fdc90 con 0x7fa4601026c0 2024-05-10T04:39:11.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.264+0000 7fa45cff9700 1 -- 172.21.15.150:0/3713528357 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa454004750 con 0x7fa4601026c0 2024-05-10T04:39:11.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.264+0000 7fa45cff9700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4480a1260 0x7fa4480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.264+0000 7fa45cff9700 1 -- 172.21.15.150:0/3713528357 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa454014070 con 0x7fa4601026c0 2024-05-10T04:39:11.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.264+0000 7fa45effd700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4480a1260 0x7fa4480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.265+0000 7fa45effd700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4480a1260 0x7fa4480a3770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b2e0 tx=0x7fa45001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:11.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.268+0000 7fa45cff9700 1 -- 172.21.15.150:0/3713528357 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa454096600 con 0x7fa4601026c0 2024-05-10T04:39:11.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.480+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa46010d3f0 con 0x7fa4601026c0 2024-05-10T04:39:11.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.481+0000 7fa45cff9700 1 -- 172.21.15.150:0/3713528357 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa454099c20 con 0x7fa4601026c0 2024-05-10T04:39:11.482 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4480a1260 msgr2=0x7fa4480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4480a1260 0x7fa4480a3770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa45000b2e0 tx=0x7fa45001a040 comp rx=0 tx=0).stop 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4601026c0 msgr2=0x7fa460101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4601026c0 0x7fa460101ee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa45400e970 tx=0x7fa45400ed30 comp rx=0 tx=0).stop 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 shutdown_connections 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa460107980 0x7fa460100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4601026c0 0x7fa460101ee0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4480a1260 0x7fa4480a3770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 --2- 172.21.15.150:0/3713528357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460105020 0x7fa4601005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 >> 172.21.15.150:0/3713528357 conn(0x7fa4600fc0e0 msgr2=0x7fa460103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 shutdown_connections 2024-05-10T04:39:11.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.483+0000 7fa46606a700 1 -- 172.21.15.150:0/3713528357 wait complete. 2024-05-10T04:39:11.498 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:39:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 -- 172.21.15.150:0/1581180082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 msgr2=0x7f51c4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/1581180082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c4108e30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00b0a0 tx=0x7f51bc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 -- 172.21.15.150:0/1581180082 shutdown_connections 2024-05-10T04:39:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/1581180082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c4108e30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/1581180082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c4100ac0 0x7f51c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/1581180082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c4101be0 0x7f51c4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 -- 172.21.15.150:0/1581180082 >> 172.21.15.150:0/1581180082 conn(0x7f51c40fc0d0 msgr2=0x7f51c40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 -- 172.21.15.150:0/1581180082 shutdown_connections 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.632+0000 7f51c8cd2700 1 -- 172.21.15.150:0/1581180082 wait complete. 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.633+0000 7f51c8cd2700 1 Processor -- start 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.633+0000 7f51c8cd2700 1 -- start start 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.633+0000 7f51c8cd2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c4100ac0 0x7f51c4196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.633+0000 7f51c8cd2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c4101be0 0x7f51c4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.633+0000 7f51c8cd2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.633+0000 7f51c8cd2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c419e9d0 con 0x7f51c4102580 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.633+0000 7f51c8cd2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c419eb40 con 0x7f51c4100ac0 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c8cd2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c419ed00 con 0x7f51c4101be0 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c419e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59260/0 (socket says 172.21.15.150:59260) 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 -- 172.21.15.150:0/3858530797 learned_addr learned my addr 172.21.15.150:0/3858530797 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c1d9b700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c4101be0 0x7f51c4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c259c700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c4100ac0 0x7f51c4196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 -- 172.21.15.150:0/3858530797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c4101be0 msgr2=0x7f51c4197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c4101be0 0x7f51c4197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 -- 172.21.15.150:0/3858530797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c4100ac0 msgr2=0x7f51c4196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c4100ac0 0x7f51c4196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 -- 172.21.15.150:0/3858530797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc009d00 con 0x7f51c4102580 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c2d9d700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c419e3c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51bc006070 tx=0x7f51bc009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51bb7fe700 1 -- 172.21.15.150:0/3858530797 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00e040 con 0x7f51c4102580 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51bb7fe700 1 -- 172.21.15.150:0/3858530797 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51bc004d60 con 0x7f51c4102580 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c419ef80 con 0x7f51c4102580 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51bb7fe700 1 -- 172.21.15.150:0/3858530797 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc01d480 con 0x7f51c4102580 2024-05-10T04:39:11.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.634+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c40683c0 con 0x7f51c4102580 2024-05-10T04:39:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.635+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c410c2d0 con 0x7f51c4102580 2024-05-10T04:39:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.637+0000 7f51bb7fe700 1 -- 172.21.15.150:0/3858530797 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f51bc003a00 con 0x7f51c4102580 2024-05-10T04:39:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.637+0000 7f51bb7fe700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f51b00a3470 0x7f51b00a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.637+0000 7f51c259c700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f51b00a3470 0x7f51b00a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.637+0000 7f51bb7fe700 1 -- 172.21.15.150:0/3858530797 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f51bc0ce7a0 con 0x7f51c4102580 2024-05-10T04:39:11.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.638+0000 7f51c259c700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f51b00a3470 0x7f51b00a5980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51b4005950 tx=0x7f51b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:11.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.641+0000 7f51bb7fe700 1 -- 172.21.15.150:0/3858530797 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f51bc097b50 con 0x7f51c4102580 2024-05-10T04:39:11.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3603714248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2862874033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-10T04:39:11.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:11 smithi161 ceph-mon[46912]: pgmap v256: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:39:11.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.839+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51c41078c0 con 0x7f51c4102580 2024-05-10T04:39:11.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.840+0000 7f51bb7fe700 1 -- 172.21.15.150:0/3858530797 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51bc09b170 con 0x7f51c4102580 2024-05-10T04:39:11.841 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f51b00a3470 msgr2=0x7f51b00a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f51b00a3470 0x7f51b00a5980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51b4005950 tx=0x7f51b40058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 msgr2=0x7f51c419e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c419e3c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51bc006070 tx=0x7f51bc009440 comp rx=0 tx=0).stop 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 shutdown_connections 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4102580 0x7f51c419e3c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51c4100ac0 0x7f51c4196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f51b00a3470 0x7f51b00a5980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 --2- 172.21.15.150:0/3858530797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51c4101be0 0x7f51c4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 >> 172.21.15.150:0/3858530797 conn(0x7f51c40fc0d0 msgr2=0x7f51c4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 shutdown_connections 2024-05-10T04:39:11.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.842+0000 7f51c8cd2700 1 -- 172.21.15.150:0/3858530797 wait complete. 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3603714248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2862874033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-10T04:39:11.856 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-05-10T04:39:11.857 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-10T04:39:11.857 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-10T04:39:11.857 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-10T04:39:11.857 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-10T04:39:11.857 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:11 smithi150 ceph-mon[42404]: pgmap v256: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:39:11.858 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:39:11.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3603714248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2862874033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-10T04:39:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:11 smithi181 ceph-mon[47314]: pgmap v256: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 -- 172.21.15.150:0/57324727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 msgr2=0x7f93081024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 --2- 172.21.15.150:0/57324727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f93081024b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009990 tx=0x7f92f8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 -- 172.21.15.150:0/57324727 shutdown_connections 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 --2- 172.21.15.150:0/57324727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9308101700 0x7f9308109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 --2- 172.21.15.150:0/57324727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f93081024b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 --2- 172.21.15.150:0/57324727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9308100db0 0x7f93081011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 -- 172.21.15.150:0/57324727 >> 172.21.15.150:0/57324727 conn(0x7f93080fb8f0 msgr2=0x7f93080fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 -- 172.21.15.150:0/57324727 shutdown_connections 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.990+0000 7f930fd4e700 1 -- 172.21.15.150:0/57324727 wait complete. 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 Processor -- start 2024-05-10T04:39:11.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 -- start start 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9308100db0 0x7f9308197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9308101700 0x7f93081984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f930819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930819fa90 con 0x7f93081020a0 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930819fc00 con 0x7f9308100db0 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930fd4e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930819fdc0 con 0x7f9308101700 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930e2eb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f930819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930e2eb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f930819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59280/0 (socket says 172.21.15.150:59280) 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.991+0000 7f930e2eb700 1 -- 172.21.15.150:0/1665074223 learned_addr learned my addr 172.21.15.150:0/1665074223 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930daea700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9308100db0 0x7f9308197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930d2e9700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9308101700 0x7f93081984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930e2eb700 1 -- 172.21.15.150:0/1665074223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9308101700 msgr2=0x7f93081984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930e2eb700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9308101700 0x7f93081984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930e2eb700 1 -- 172.21.15.150:0/1665074223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9308100db0 msgr2=0x7f9308197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:11.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930e2eb700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9308100db0 0x7f9308197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930e2eb700 1 -- 172.21.15.150:0/1665074223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80095f0 con 0x7f93081020a0 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930d2e9700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9308101700 0x7f93081984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930e2eb700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f930819f480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f930000ef30 tx=0x7f930000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f92feffd700 1 -- 172.21.15.150:0/1665074223 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300018070 con 0x7f93081020a0 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93081a0040 con 0x7f93081020a0 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f92feffd700 1 -- 172.21.15.150:0/1665074223 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f930000f040 con 0x7f93081020a0 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f92feffd700 1 -- 172.21.15.150:0/1665074223 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300003dc0 con 0x7f93081020a0 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.992+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93081a0550 con 0x7f93081020a0 2024-05-10T04:39:11.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.993+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93080678b0 con 0x7f93081020a0 2024-05-10T04:39:11.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.995+0000 7f92feffd700 1 -- 172.21.15.150:0/1665074223 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f930000caa0 con 0x7f93081020a0 2024-05-10T04:39:11.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.995+0000 7f92feffd700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92f40a11b0 0x7f92f40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:11.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.996+0000 7f92feffd700 1 -- 172.21.15.150:0/1665074223 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f93000cd410 con 0x7f93081020a0 2024-05-10T04:39:11.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.996+0000 7f930daea700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92f40a11b0 0x7f92f40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:11.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.996+0000 7f930daea700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92f40a11b0 0x7f92f40a36c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b2e0 tx=0x7f92f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:12.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:11.999+0000 7f92feffd700 1 -- 172.21.15.150:0/1665074223 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9300096760 con 0x7f93081020a0 2024-05-10T04:39:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.241+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f930810c290 con 0x7f93081020a0 2024-05-10T04:39:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.242+0000 7f92feffd700 1 -- 172.21.15.150:0/1665074223 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f93000a0070 con 0x7f93081020a0 2024-05-10T04:39:12.243 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:12.244 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:35.978906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92f40a11b0 msgr2=0x7f92f40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92f40a11b0 0x7f92f40a36c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b2e0 tx=0x7f92f801a040 comp rx=0 tx=0).stop 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 msgr2=0x7f930819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f930819f480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f930000ef30 tx=0x7f930000c450 comp rx=0 tx=0).stop 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 shutdown_connections 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93081020a0 0x7f930819f480 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9308100db0 0x7f9308197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92f40a11b0 0x7f92f40a36c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 --2- 172.21.15.150:0/1665074223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9308101700 0x7f93081984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 >> 172.21.15.150:0/1665074223 conn(0x7f93080fb8f0 msgr2=0x7f93080fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 shutdown_connections 2024-05-10T04:39:12.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.244+0000 7f930fd4e700 1 -- 172.21.15.150:0/1665074223 wait complete. 2024-05-10T04:39:12.247 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 28 2024-05-10T04:39:12.259 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 -- 172.21.15.150:0/1459508331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f01013e0 msgr2=0x7fb8f01017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/1459508331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f01013e0 0x7fb8f01017f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009990 tx=0x7fb8ec009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 -- 172.21.15.150:0/1459508331 shutdown_connections 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/1459508331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f0101d80 0x7fb8f0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/1459508331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f01002c0 0x7fb8f01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/1459508331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f01013e0 0x7fb8f01017f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 -- 172.21.15.150:0/1459508331 >> 172.21.15.150:0/1459508331 conn(0x7fb8f00fb8f0 msgr2=0x7fb8f00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 -- 172.21.15.150:0/1459508331 shutdown_connections 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.392+0000 7fb8f83c2700 1 -- 172.21.15.150:0/1459508331 wait complete. 2024-05-10T04:39:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 Processor -- start 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 -- start start 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f01002c0 0x7fb8f0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f01013e0 0x7fb8f01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f0101d80 0x7fb8f019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019fa70 con 0x7fb8f0101d80 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019fbe0 con 0x7fb8f01002c0 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f83c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019fda0 con 0x7fb8f01013e0 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f0101d80 0x7fb8f019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f0101d80 0x7fb8f019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59296/0 (socket says 172.21.15.150:59296) 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 -- 172.21.15.150:0/2010012229 learned_addr learned my addr 172.21.15.150:0/2010012229 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f595d700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f01013e0 0x7fb8f01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f615e700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f01002c0 0x7fb8f0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 -- 172.21.15.150:0/2010012229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f01013e0 msgr2=0x7fb8f01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f01013e0 0x7fb8f01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 -- 172.21.15.150:0/2010012229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f01002c0 msgr2=0x7fb8f0197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f01002c0 0x7fb8f0197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.393+0000 7fb8f695f700 1 -- 172.21.15.150:0/2010012229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0095f0 con 0x7fb8f0101d80 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.394+0000 7fb8f695f700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f0101d80 0x7fb8f019f460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8007ee0 tx=0x7fb8e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.394+0000 7fb8e77fe700 1 -- 172.21.15.150:0/2010012229 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8017070 con 0x7fb8f0101d80 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.394+0000 7fb8e77fe700 1 -- 172.21.15.150:0/2010012229 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8e800f040 con 0x7fb8f0101d80 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.394+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f01a0080 con 0x7fb8f0101d80 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.394+0000 7fb8e77fe700 1 -- 172.21.15.150:0/2010012229 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8005740 con 0x7fb8f0101d80 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.394+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f01a0540 con 0x7fb8f0101d80 2024-05-10T04:39:12.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.395+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f01074d0 con 0x7fb8f0101d80 2024-05-10T04:39:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.396+0000 7fb8e77fe700 1 -- 172.21.15.150:0/2010012229 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb8e800dc60 con 0x7fb8f0101d80 2024-05-10T04:39:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.397+0000 7fb8e77fe700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb8dc0a11b0 0x7fb8dc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.397+0000 7fb8f615e700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb8dc0a11b0 0x7fb8dc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.397+0000 7fb8e77fe700 1 -- 172.21.15.150:0/2010012229 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb8e80ccbe0 con 0x7fb8f0101d80 2024-05-10T04:39:12.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.397+0000 7fb8f615e700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb8dc0a11b0 0x7fb8dc0a36c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005950 tx=0x7fb8e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:12.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.401+0000 7fb8e77fe700 1 -- 172.21.15.150:0/2010012229 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb8e8095ea0 con 0x7fb8f0101d80 2024-05-10T04:39:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.606+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8f00678b0 con 0x7fb8f0101d80 2024-05-10T04:39:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.608+0000 7fb8e77fe700 1 -- 172.21.15.150:0/2010012229 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7fb8e80994c0 con 0x7fb8f0101d80 2024-05-10T04:39:12.609 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:12.610 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb8dc0a11b0 msgr2=0x7fb8dc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb8dc0a11b0 0x7fb8dc0a36c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005950 tx=0x7fb8e00058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f0101d80 msgr2=0x7fb8f019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f0101d80 0x7fb8f019f460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8007ee0 tx=0x7fb8e800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 shutdown_connections 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8f0101d80 0x7fb8f019f460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8f01002c0 0x7fb8f0197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb8dc0a11b0 0x7fb8dc0a36c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 --2- 172.21.15.150:0/2010012229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f01013e0 0x7fb8f01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 >> 172.21.15.150:0/2010012229 conn(0x7fb8f00fb8f0 msgr2=0x7fb8f0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 shutdown_connections 2024-05-10T04:39:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.610+0000 7fb8f83c2700 1 -- 172.21.15.150:0/2010012229 wait complete. 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T04:39:12.624 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3713528357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:12.625 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3858530797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:12.625 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:12 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1665074223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:12.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:39:12.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 -- 172.21.15.150:0/2717127756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b0101c00 msgr2=0x7f48b0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 --2- 172.21.15.150:0/2717127756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b0101c00 0x7f48b0102010 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f48a00099c0 tx=0x7f48a0009cd0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 -- 172.21.15.150:0/2717127756 shutdown_connections 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 --2- 172.21.15.150:0/2717127756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b01036c0 0x7f48b0109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 --2- 172.21.15.150:0/2717127756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b0101c00 0x7f48b0102010 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 --2- 172.21.15.150:0/2717127756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48b0102d20 0x7f48b0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 -- 172.21.15.150:0/2717127756 >> 172.21.15.150:0/2717127756 conn(0x7f48b00fd210 msgr2=0x7f48b00ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 -- 172.21.15.150:0/2717127756 shutdown_connections 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 -- 172.21.15.150:0/2717127756 wait complete. 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.763+0000 7f48b605d700 1 Processor -- start 2024-05-10T04:39:12.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48b605d700 1 -- start start 2024-05-10T04:39:12.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48b605d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b0101c00 0x7f48b0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48b605d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48b0102d20 0x7f48b0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48b605d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b01036c0 0x7f48b019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48b605d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b019fa20 con 0x7f48b0102d20 2024-05-10T04:39:12.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48b605d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b019fb90 con 0x7f48b01036c0 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48b605d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b019fd50 con 0x7f48b0101c00 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b0101c00 0x7f48b0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48aeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48b0102d20 0x7f48b0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b0101c00 0x7f48b0197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:45438/0 (socket says 172.21.15.150:45438) 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 -- 172.21.15.150:0/2685711041 learned_addr learned my addr 172.21.15.150:0/2685711041 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48affff700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b01036c0 0x7f48b019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 -- 172.21.15.150:0/2685711041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b01036c0 msgr2=0x7f48b019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b01036c0 0x7f48b019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 -- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48b0102d20 msgr2=0x7f48b0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48b0102d20 0x7f48b0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48af7fe700 1 -- 172.21.15.150:0/2685711041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00095f0 con 0x7f48b0101c00 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.764+0000 7f48aeffd700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48b0102d20 0x7f48b0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.765+0000 7f48af7fe700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b0101c00 0x7f48b0197f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009990 tx=0x7f48a000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.765+0000 7f48acff9700 1 -- 172.21.15.150:0/2685711041 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a001d070 con 0x7f48b0101c00 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.765+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b019ffd0 con 0x7f48b0101c00 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.765+0000 7f48acff9700 1 -- 172.21.15.150:0/2685711041 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48a0005030 con 0x7f48b0101c00 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.765+0000 7f48acff9700 1 -- 172.21.15.150:0/2685711041 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0004950 con 0x7f48b0101c00 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.765+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b01a0340 con 0x7f48b0101c00 2024-05-10T04:39:12.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.766+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0108e10 con 0x7f48b0101c00 2024-05-10T04:39:12.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.768+0000 7f48acff9700 1 -- 172.21.15.150:0/2685711041 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f48a0005410 con 0x7f48b0101c00 2024-05-10T04:39:12.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.768+0000 7f48acff9700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f48980a1260 0x7f48980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:12.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.768+0000 7f48acff9700 1 -- 172.21.15.150:0/2685711041 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f48a00cdee0 con 0x7f48b0101c00 2024-05-10T04:39:12.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.769+0000 7f48aeffd700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f48980a1260 0x7f48980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:12.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.769+0000 7f48aeffd700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f48980a1260 0x7f48980a3770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48b0198fe0 tx=0x7f48a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:12.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.773+0000 7f48acff9700 1 -- 172.21.15.150:0/2685711041 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48a0097230 con 0x7f48b0101c00 2024-05-10T04:39:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-05-10T04:39:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3713528357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3858530797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:12 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1665074223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:12.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: osdmap e87: 12 total, 12 up, 12 in 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3713528357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3858530797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:12.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:12 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1665074223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:12.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.973+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48b010d430 con 0x7f48b0101c00 2024-05-10T04:39:12.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.974+0000 7f48acff9700 1 -- 172.21.15.150:0/2685711041 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48a009a850 con 0x7f48b0101c00 2024-05-10T04:39:12.976 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:39:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.976+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f48980a1260 msgr2=0x7f48980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.976+0000 7f48b605d700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f48980a1260 0x7f48980a3770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48b0198fe0 tx=0x7f48a4009260 comp rx=0 tx=0).stop 2024-05-10T04:39:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.976+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b0101c00 msgr2=0x7f48b0197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.976+0000 7f48b605d700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b0101c00 0x7f48b0197f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009990 tx=0x7f48a000f690 comp rx=0 tx=0).stop 2024-05-10T04:39:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.976+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 shutdown_connections 2024-05-10T04:39:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.976+0000 7f48b605d700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48b01036c0 0x7f48b019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.977+0000 7f48b605d700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f48980a1260 0x7f48980a3770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.977+0000 7f48b605d700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b0101c00 0x7f48b0197f40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.977+0000 7f48b605d700 1 --2- 172.21.15.150:0/2685711041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48b0102d20 0x7f48b0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:12.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.977+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 >> 172.21.15.150:0/2685711041 conn(0x7f48b00fd210 msgr2=0x7f48b0105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:12.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.977+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 shutdown_connections 2024-05-10T04:39:12.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:12.977+0000 7f48b605d700 1 -- 172.21.15.150:0/2685711041 wait complete. 2024-05-10T04:39:12.991 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:39:13.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.125+0000 7f94b3eea700 1 -- 172.21.15.150:0/3464804575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac101c10 msgr2=0x7f94ac102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.125+0000 7f94b3eea700 1 --2- 172.21.15.150:0/3464804575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac101c10 0x7f94ac102020 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f949c009990 tx=0x7f949c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 -- 172.21.15.150:0/3464804575 shutdown_connections 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 --2- 172.21.15.150:0/3464804575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac1025b0 0x7f94ac108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 --2- 172.21.15.150:0/3464804575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac100af0 0x7f94ac100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 --2- 172.21.15.150:0/3464804575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac101c10 0x7f94ac102020 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 -- 172.21.15.150:0/3464804575 >> 172.21.15.150:0/3464804575 conn(0x7f94ac0fc0e0 msgr2=0x7f94ac0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 -- 172.21.15.150:0/3464804575 shutdown_connections 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 -- 172.21.15.150:0/3464804575 wait complete. 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 Processor -- start 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 -- start start 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac100af0 0x7f94ac073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac101c10 0x7f94ac19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.126+0000 7f94b3eea700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac1025b0 0x7f94ac19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b3eea700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19f9f0 con 0x7f94ac1025b0 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b3eea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19fb60 con 0x7f94ac101c10 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b3eea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19fd20 con 0x7f94ac100af0 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac1025b0 0x7f94ac19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac1025b0 0x7f94ac19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59352/0 (socket says 172.21.15.150:59352) 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 -- 172.21.15.150:0/2509197853 learned_addr learned my addr 172.21.15.150:0/2509197853 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b1485700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac101c10 0x7f94ac19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b1c86700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac100af0 0x7f94ac073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 -- 172.21.15.150:0/2509197853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac100af0 msgr2=0x7f94ac073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac100af0 0x7f94ac073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 -- 172.21.15.150:0/2509197853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac101c10 msgr2=0x7f94ac19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac101c10 0x7f94ac19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 -- 172.21.15.150:0/2509197853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0095f0 con 0x7f94ac1025b0 2024-05-10T04:39:13.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b1c86700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac100af0 0x7f94ac073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b1485700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac101c10 0x7f94ac19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b2487700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac1025b0 0x7f94ac19f2a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007ee0 tx=0x7f94a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94a2ffd700 1 -- 172.21.15.150:0/2509197853 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4017070 con 0x7f94ac1025b0 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94a2ffd700 1 -- 172.21.15.150:0/2509197853 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94a400f040 con 0x7f94ac1025b0 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac19ffa0 con 0x7f94ac1025b0 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94a2ffd700 1 -- 172.21.15.150:0/2509197853 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4005740 con 0x7f94ac1025b0 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.127+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac1a03d0 con 0x7f94ac1025b0 2024-05-10T04:39:13.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.128+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac107d00 con 0x7f94ac1025b0 2024-05-10T04:39:13.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.129+0000 7f94a2ffd700 1 -- 172.21.15.150:0/2509197853 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f94a40058a0 con 0x7f94ac1025b0 2024-05-10T04:39:13.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.130+0000 7f94a2ffd700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94980a1260 0x7f94980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.130+0000 7f94a2ffd700 1 -- 172.21.15.150:0/2509197853 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f94a40ccc10 con 0x7f94ac1025b0 2024-05-10T04:39:13.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.130+0000 7f94b1c86700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94980a1260 0x7f94980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.131+0000 7f94b1c86700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94980a1260 0x7f94980a3770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94a8005fd0 tx=0x7f94a8005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:13.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.134+0000 7f94a2ffd700 1 -- 172.21.15.150:0/2509197853 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f94a4095f30 con 0x7f94ac1025b0 2024-05-10T04:39:13.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.330+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94ac10c2f0 con 0x7f94ac1025b0 2024-05-10T04:39:13.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.330+0000 7f94a2ffd700 1 -- 172.21.15.150:0/2509197853 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94a4099550 con 0x7f94ac1025b0 2024-05-10T04:39:13.332 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:13.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94980a1260 msgr2=0x7f94980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94980a1260 0x7f94980a3770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94a8005fd0 tx=0x7f94a8005f00 comp rx=0 tx=0).stop 2024-05-10T04:39:13.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac1025b0 msgr2=0x7f94ac19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac1025b0 0x7f94ac19f2a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007ee0 tx=0x7f94a400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 shutdown_connections 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac1025b0 0x7f94ac19f2a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f94980a1260 0x7f94980a3770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac100af0 0x7f94ac073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 --2- 172.21.15.150:0/2509197853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94ac101c10 0x7f94ac19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 >> 172.21.15.150:0/2509197853 conn(0x7f94ac0fc0e0 msgr2=0x7f94ac104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 shutdown_connections 2024-05-10T04:39:13.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.333+0000 7f94b3eea700 1 -- 172.21.15.150:0/2509197853 wait complete. 2024-05-10T04:39:13.347 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:39:13.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 -- 172.21.15.150:0/2479376118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720101be0 msgr2=0x7f6720101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 --2- 172.21.15.150:0/2479376118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720101be0 0x7f6720101ff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6714009990 tx=0x7f6714009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 -- 172.21.15.150:0/2479376118 shutdown_connections 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 --2- 172.21.15.150:0/2479376118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6720102580 0x7f6720108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 --2- 172.21.15.150:0/2479376118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6720100ac0 0x7f6720100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 --2- 172.21.15.150:0/2479376118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720101be0 0x7f6720101ff0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 -- 172.21.15.150:0/2479376118 >> 172.21.15.150:0/2479376118 conn(0x7f67200fc0d0 msgr2=0x7f67200fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 -- 172.21.15.150:0/2479376118 shutdown_connections 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.484+0000 7f6724bc1700 1 -- 172.21.15.150:0/2479376118 wait complete. 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 Processor -- start 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 -- start start 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6720100ac0 0x7f6720068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6720101be0 0x7f672019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720102580 0x7f672019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672019f9f0 con 0x7f6720102580 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672019fb60 con 0x7f6720100ac0 2024-05-10T04:39:13.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f6724bc1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672019fd20 con 0x7f6720101be0 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720102580 0x7f672019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720102580 0x7f672019f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59370/0 (socket says 172.21.15.150:59370) 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 -- 172.21.15.150:0/895064445 learned_addr learned my addr 172.21.15.150:0/895064445 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671dd9b700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6720101be0 0x7f672019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671e59c700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6720100ac0 0x7f6720068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 -- 172.21.15.150:0/895064445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6720101be0 msgr2=0x7f672019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6720101be0 0x7f672019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 -- 172.21.15.150:0/895064445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6720100ac0 msgr2=0x7f6720068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6720100ac0 0x7f6720068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 -- 172.21.15.150:0/895064445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140095f0 con 0x7f6720102580 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.485+0000 7f671ed9d700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720102580 0x7f672019f2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f671800ada0 tx=0x7f671800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.486+0000 7f670f7fe700 1 -- 172.21.15.150:0/895064445 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671800f070 con 0x7f6720102580 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.486+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672019ffa0 con 0x7f6720102580 2024-05-10T04:39:13.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.486+0000 7f670f7fe700 1 -- 172.21.15.150:0/895064445 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6718007b30 con 0x7f6720102580 2024-05-10T04:39:13.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.486+0000 7f670f7fe700 1 -- 172.21.15.150:0/895064445 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718005280 con 0x7f6720102580 2024-05-10T04:39:13.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.486+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67201a03d0 con 0x7f6720102580 2024-05-10T04:39:13.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.487+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6720107cd0 con 0x7f6720102580 2024-05-10T04:39:13.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.489+0000 7f670f7fe700 1 -- 172.21.15.150:0/895064445 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f671800a090 con 0x7f6720102580 2024-05-10T04:39:13.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.489+0000 7f670f7fe700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67080a1260 0x7f67080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.489+0000 7f670f7fe700 1 -- 172.21.15.150:0/895064445 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f67180cc7f0 con 0x7f6720102580 2024-05-10T04:39:13.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.489+0000 7f671e59c700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67080a1260 0x7f67080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.490+0000 7f671e59c700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67080a1260 0x7f67080a3770 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6720101ae0 tx=0x7f671000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:13.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.493+0000 7f670f7fe700 1 -- 172.21.15.150:0/895064445 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6718099630 con 0x7f6720102580 2024-05-10T04:39:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:13 smithi150 ceph-mon[42404]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T04:39:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:13 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2010012229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:13 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2685711041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:13 smithi150 ceph-mon[42404]: pgmap v259: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 10 op/s 2024-05-10T04:39:13.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:13 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2509197853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:13.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.688+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f672010c2c0 con 0x7f6720102580 2024-05-10T04:39:13.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.688+0000 7f670f7fe700 1 -- 172.21.15.150:0/895064445 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f67180991c0 con 0x7f6720102580 2024-05-10T04:39:13.690 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:39:13.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.690+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67080a1260 msgr2=0x7f67080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.690+0000 7f6724bc1700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67080a1260 0x7f67080a3770 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6720101ae0 tx=0x7f671000b410 comp rx=0 tx=0).stop 2024-05-10T04:39:13.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.690+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720102580 msgr2=0x7f672019f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.690+0000 7f6724bc1700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720102580 0x7f672019f2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f671800ada0 tx=0x7f671800c330 comp rx=0 tx=0).stop 2024-05-10T04:39:13.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 shutdown_connections 2024-05-10T04:39:13.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6720102580 0x7f672019f2a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6720100ac0 0x7f6720068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67080a1260 0x7f67080a3770 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 --2- 172.21.15.150:0/895064445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6720101be0 0x7f672019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 >> 172.21.15.150:0/895064445 conn(0x7f67200fc0d0 msgr2=0x7f6720104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:13.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 shutdown_connections 2024-05-10T04:39:13.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.691+0000 7f6724bc1700 1 -- 172.21.15.150:0/895064445 wait complete. 2024-05-10T04:39:13.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:39:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:13 smithi161 ceph-mon[46912]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T04:39:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:13 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2010012229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:13.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:13 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2685711041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:13.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:13 smithi161 ceph-mon[46912]: pgmap v259: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 10 op/s 2024-05-10T04:39:13.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:13 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2509197853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:13.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 -- 172.21.15.150:0/215932279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380101bf0 msgr2=0x7f7380102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 --2- 172.21.15.150:0/215932279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380101bf0 0x7f7380102000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7370009990 tx=0x7f7370009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 -- 172.21.15.150:0/215932279 shutdown_connections 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 --2- 172.21.15.150:0/215932279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7380102590 0x7f7380108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 --2- 172.21.15.150:0/215932279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7380100ad0 0x7f7380100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 --2- 172.21.15.150:0/215932279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380101bf0 0x7f7380102000 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 -- 172.21.15.150:0/215932279 >> 172.21.15.150:0/215932279 conn(0x7f73800fc0e0 msgr2=0x7f73800fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 -- 172.21.15.150:0/215932279 shutdown_connections 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.839+0000 7f7386259700 1 -- 172.21.15.150:0/215932279 wait complete. 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 Processor -- start 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 -- start start 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380100ad0 0x7f7380196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7380101bf0 0x7f7380197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7380102590 0x7f738019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019e900 con 0x7f7380100ad0 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019ea70 con 0x7f7380101bf0 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f7386259700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738019ec30 con 0x7f7380102590 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f737f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380100ad0 0x7f7380196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f737f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380100ad0 0x7f7380196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59382/0 (socket says 172.21.15.150:59382) 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f737f7fe700 1 -- 172.21.15.150:0/3056370936 learned_addr learned my addr 172.21.15.150:0/3056370936 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.840+0000 7f737effd700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7380101bf0 0x7f7380197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737ffff700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7380102590 0x7f738019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737f7fe700 1 -- 172.21.15.150:0/3056370936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7380102590 msgr2=0x7f738019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737f7fe700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7380102590 0x7f738019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737f7fe700 1 -- 172.21.15.150:0/3056370936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7380101bf0 msgr2=0x7f7380197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737f7fe700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7380101bf0 0x7f7380197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737f7fe700 1 -- 172.21.15.150:0/3056370936 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700095f0 con 0x7f7380100ad0 2024-05-10T04:39:13.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737f7fe700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380100ad0 0x7f7380196e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f737400e940 tx=0x7f737400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:13.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737cff9700 1 -- 172.21.15.150:0/3056370936 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737400cbd0 con 0x7f7380100ad0 2024-05-10T04:39:13.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737cff9700 1 -- 172.21.15.150:0/3056370936 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737400ee90 con 0x7f7380100ad0 2024-05-10T04:39:13.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f737cff9700 1 -- 172.21.15.150:0/3056370936 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374003cb0 con 0x7f7380100ad0 2024-05-10T04:39:13.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738019ef10 con 0x7f7380100ad0 2024-05-10T04:39:13.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.841+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738019f2e0 con 0x7f7380100ad0 2024-05-10T04:39:13.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.842+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7380107ce0 con 0x7f7380100ad0 2024-05-10T04:39:13.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.843+0000 7f737cff9700 1 -- 172.21.15.150:0/3056370936 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f737400cd30 con 0x7f7380100ad0 2024-05-10T04:39:13.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.844+0000 7f737cff9700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73680a1260 0x7f73680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:13.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.844+0000 7f737cff9700 1 -- 172.21.15.150:0/3056370936 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7374014070 con 0x7f7380100ad0 2024-05-10T04:39:13.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.844+0000 7f737effd700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73680a1260 0x7f73680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:13.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.845+0000 7f737effd700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73680a1260 0x7f73680a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f737000b2e0 tx=0x7f737001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:13.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:13.848+0000 7f737cff9700 1 -- 172.21.15.150:0/3056370936 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f73740963e0 con 0x7f7380100ad0 2024-05-10T04:39:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:13 smithi181 ceph-mon[47314]: osdmap e88: 12 total, 12 up, 12 in 2024-05-10T04:39:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:13 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2010012229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:13 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2685711041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:13 smithi181 ceph-mon[47314]: pgmap v259: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 10 op/s 2024-05-10T04:39:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:13 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2509197853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:14.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.044+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7380068730 con 0x7f7380100ad0 2024-05-10T04:39:14.046 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.045+0000 7f737cff9700 1 -- 172.21.15.150:0/3056370936 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7374099a00 con 0x7f7380100ad0 2024-05-10T04:39:14.046 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73680a1260 msgr2=0x7f73680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73680a1260 0x7f73680a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f737000b2e0 tx=0x7f737001a040 comp rx=0 tx=0).stop 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380100ad0 msgr2=0x7f7380196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380100ad0 0x7f7380196e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f737400e940 tx=0x7f737400ec50 comp rx=0 tx=0).stop 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 shutdown_connections 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f73680a1260 0x7f73680a3770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7380102590 0x7f738019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7380100ad0 0x7f7380196e20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 --2- 172.21.15.150:0/3056370936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7380101bf0 0x7f7380197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 >> 172.21.15.150:0/3056370936 conn(0x7f73800fc0e0 msgr2=0x7f7380104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 shutdown_connections 2024-05-10T04:39:14.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.047+0000 7f7386259700 1 -- 172.21.15.150:0/3056370936 wait complete. 2024-05-10T04:39:14.062 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:39:14.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.193+0000 7ff01a7ba700 1 -- 172.21.15.150:0/352396300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014107980 msgr2=0x7ff014109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.193+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/352396300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014107980 0x7ff014109da0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00b0a0 tx=0x7ff00c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.193+0000 7ff01a7ba700 1 -- 172.21.15.150:0/352396300 shutdown_connections 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.193+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/352396300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014107980 0x7ff014109da0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.193+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/352396300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0141026c0 0x7ff014104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.193+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/352396300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff014105020 0x7ff014107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.194+0000 7ff01a7ba700 1 -- 172.21.15.150:0/352396300 >> 172.21.15.150:0/352396300 conn(0x7ff0140fc0e0 msgr2=0x7ff0140fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.194+0000 7ff01a7ba700 1 -- 172.21.15.150:0/352396300 shutdown_connections 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.194+0000 7ff01a7ba700 1 -- 172.21.15.150:0/352396300 wait complete. 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.194+0000 7ff01a7ba700 1 Processor -- start 2024-05-10T04:39:14.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.194+0000 7ff01a7ba700 1 -- start start 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0141026c0 0x7ff014198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014105020 0x7ff014198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff014107980 0x7ff01419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419fc10 con 0x7ff014105020 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419fd80 con 0x7ff0141026c0 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419ff40 con 0x7ff014107980 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014105020 0x7ff014198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014105020 0x7ff014198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59396/0 (socket says 172.21.15.150:59396) 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 -- 172.21.15.150:0/2771600261 learned_addr learned my addr 172.21.15.150:0/2771600261 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff013fff700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0141026c0 0x7ff014198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff018d57700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff014107980 0x7ff01419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 -- 172.21.15.150:0/2771600261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff014107980 msgr2=0x7ff01419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff014107980 0x7ff01419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 -- 172.21.15.150:0/2771600261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0141026c0 msgr2=0x7ff014198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0141026c0 0x7ff014198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 -- 172.21.15.150:0/2771600261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c009d00 con 0x7ff014105020 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0137fe700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014105020 0x7ff014198670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff00800e9a0 tx=0x7ff00800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0117fa700 1 -- 172.21.15.150:0/2771600261 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008009790 con 0x7ff014105020 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0141a0220 con 0x7ff014105020 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0117fa700 1 -- 172.21.15.150:0/2771600261 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00800cbd0 con 0x7ff014105020 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff0117fa700 1 -- 172.21.15.150:0/2771600261 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008018680 con 0x7ff014105020 2024-05-10T04:39:14.199 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.195+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0141a0730 con 0x7ff014105020 2024-05-10T04:39:14.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.197+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0140fdc90 con 0x7ff014105020 2024-05-10T04:39:14.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.198+0000 7ff0117fa700 1 -- 172.21.15.150:0/2771600261 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff00800cd40 con 0x7ff014105020 2024-05-10T04:39:14.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.198+0000 7ff0117fa700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feffc0a11b0 0x7feffc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.199+0000 7ff013fff700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feffc0a11b0 0x7feffc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.199+0000 7ff0117fa700 1 -- 172.21.15.150:0/2771600261 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff008014070 con 0x7ff014105020 2024-05-10T04:39:14.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.199+0000 7ff013fff700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feffc0a11b0 0x7feffc0a36c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff004005950 tx=0x7ff00400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:14.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.202+0000 7ff0117fa700 1 -- 172.21.15.150:0/2771600261 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff008096620 con 0x7ff014105020 2024-05-10T04:39:14.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.445+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff014068730 con 0x7ff014105020 2024-05-10T04:39:14.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.446+0000 7ff0117fa700 1 -- 172.21.15.150:0/2771600261 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10933 (secure 0 0 0) 0x7ff008099c40 con 0x7ff014105020 2024-05-10T04:39:14.447 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:14.448 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:35.978906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feffc0a11b0 msgr2=0x7feffc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feffc0a11b0 0x7feffc0a36c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff004005950 tx=0x7ff00400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014105020 msgr2=0x7ff014198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014105020 0x7ff014198670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff00800e9a0 tx=0x7ff00800ecb0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 shutdown_connections 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feffc0a11b0 0x7feffc0a36c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff014107980 0x7ff01419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0141026c0 0x7ff014198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 --2- 172.21.15.150:0/2771600261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff014105020 0x7ff014198670 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 >> 172.21.15.150:0/2771600261 conn(0x7ff0140fc0e0 msgr2=0x7ff014103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 shutdown_connections 2024-05-10T04:39:14.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.448+0000 7ff01a7ba700 1 -- 172.21.15.150:0/2771600261 wait complete. 2024-05-10T04:39:14.451 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 28 2024-05-10T04:39:14.464 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:14.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 -- 172.21.15.150:0/1985016475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101640 msgr2=0x7fc278108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 --2- 172.21.15.150:0/1985016475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101640 0x7fc278108f50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc27000b0a0 tx=0x7fc27000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 -- 172.21.15.150:0/1985016475 shutdown_connections 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 --2- 172.21.15.150:0/1985016475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101640 0x7fc278108f50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 --2- 172.21.15.150:0/1985016475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc278101ff0 0x7fc278102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 --2- 172.21.15.150:0/1985016475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc278100cf0 0x7fc278101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 -- 172.21.15.150:0/1985016475 >> 172.21.15.150:0/1985016475 conn(0x7fc2780fb8b0 msgr2=0x7fc2780fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 -- 172.21.15.150:0/1985016475 shutdown_connections 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 -- 172.21.15.150:0/1985016475 wait complete. 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.596+0000 7fc27d447700 1 Processor -- start 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc27d447700 1 -- start start 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc27d447700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc278100cf0 0x7fc278196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc27d447700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc278101640 0x7fc2781974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc27d447700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101ff0 0x7fc27819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc27d447700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27819ea90 con 0x7fc278101ff0 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc27d447700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27819ec00 con 0x7fc278100cf0 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc27d447700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27819edc0 con 0x7fc278101640 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101ff0 0x7fc27819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101ff0 0x7fc27819e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59406/0 (socket says 172.21.15.150:59406) 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 -- 172.21.15.150:0/2069185111 learned_addr learned my addr 172.21.15.150:0/2069185111 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc276ffd700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc278100cf0 0x7fc278196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2767fc700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc278101640 0x7fc2781974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 -- 172.21.15.150:0/2069185111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc278101640 msgr2=0x7fc2781974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc278101640 0x7fc2781974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 -- 172.21.15.150:0/2069185111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc278100cf0 msgr2=0x7fc278196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc278100cf0 0x7fc278196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 -- 172.21.15.150:0/2069185111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009d00 con 0x7fc278101ff0 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc2777fe700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101ff0 0x7fc27819e480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc270007840 tx=0x7fc270007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:14.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.597+0000 7fc26ffff700 1 -- 172.21.15.150:0/2069185111 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000e040 con 0x7fc278101ff0 2024-05-10T04:39:14.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.598+0000 7fc26ffff700 1 -- 172.21.15.150:0/2069185111 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27000be80 con 0x7fc278101ff0 2024-05-10T04:39:14.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.598+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27819f040 con 0x7fc278101ff0 2024-05-10T04:39:14.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.598+0000 7fc26ffff700 1 -- 172.21.15.150:0/2069185111 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700052a0 con 0x7fc278101ff0 2024-05-10T04:39:14.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.598+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27819f4c0 con 0x7fc278101ff0 2024-05-10T04:39:14.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.598+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc278107df0 con 0x7fc278101ff0 2024-05-10T04:39:14.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.600+0000 7fc26ffff700 1 -- 172.21.15.150:0/2069185111 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc27001d470 con 0x7fc278101ff0 2024-05-10T04:39:14.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.601+0000 7fc26ffff700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2640a11b0 0x7fc2640a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.601+0000 7fc276ffd700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2640a11b0 0x7fc2640a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.601+0000 7fc26ffff700 1 -- 172.21.15.150:0/2069185111 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc2700ce760 con 0x7fc278101ff0 2024-05-10T04:39:14.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.601+0000 7fc276ffd700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2640a11b0 0x7fc2640a36c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc278069980 tx=0x7fc260009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:14.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.605+0000 7fc26ffff700 1 -- 172.21.15.150:0/2069185111 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc270097b10 con 0x7fc278101ff0 2024-05-10T04:39:14.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.801+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc27810c340 con 0x7fc278101ff0 2024-05-10T04:39:14.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.803+0000 7fc26ffff700 1 -- 172.21.15.150:0/2069185111 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7fc27009b130 con 0x7fc278101ff0 2024-05-10T04:39:14.805 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:14.805 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2640a11b0 msgr2=0x7fc2640a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2640a11b0 0x7fc2640a36c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc278069980 tx=0x7fc260009260 comp rx=0 tx=0).stop 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101ff0 msgr2=0x7fc27819e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101ff0 0x7fc27819e480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc270007840 tx=0x7fc270007870 comp rx=0 tx=0).stop 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 shutdown_connections 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc278101ff0 0x7fc27819e480 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc278100cf0 0x7fc278196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc2640a11b0 0x7fc2640a36c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 --2- 172.21.15.150:0/2069185111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc278101640 0x7fc2781974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.805+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 >> 172.21.15.150:0/2069185111 conn(0x7fc2780fb8b0 msgr2=0x7fc278104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:14.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.806+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 shutdown_connections 2024-05-10T04:39:14.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.806+0000 7fc27d447700 1 -- 172.21.15.150:0/2069185111 wait complete. 2024-05-10T04:39:14.819 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:14 smithi150 ceph-mon[42404]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-10T04:39:14.819 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:14 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/895064445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:14.819 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:14 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3056370936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:14.819 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:14 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2771600261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:14.820 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:39:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:14 smithi161 ceph-mon[46912]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-10T04:39:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:14 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/895064445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:14 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3056370936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:14 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2771600261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:14.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:14 smithi181 ceph-mon[47314]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-05-10T04:39:14.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:14 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/895064445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:14.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:14 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3056370936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:14.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:14 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2771600261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:14.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 -- 172.21.15.150:0/4214400183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4100ac0 msgr2=0x7f70f4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 --2- 172.21.15.150:0/4214400183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4100ac0 0x7f70f4100ed0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009990 tx=0x7f70f0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 -- 172.21.15.150:0/4214400183 shutdown_connections 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 --2- 172.21.15.150:0/4214400183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f4102580 0x7f70f4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 --2- 172.21.15.150:0/4214400183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4100ac0 0x7f70f4100ed0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 --2- 172.21.15.150:0/4214400183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f4101be0 0x7f70f4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 -- 172.21.15.150:0/4214400183 >> 172.21.15.150:0/4214400183 conn(0x7f70f40fc0d0 msgr2=0x7f70f40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 -- 172.21.15.150:0/4214400183 shutdown_connections 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 -- 172.21.15.150:0/4214400183 wait complete. 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.954+0000 7f70fc10d700 1 Processor -- start 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fc10d700 1 -- start start 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fc10d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f4100ac0 0x7f70f4068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fc10d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4101be0 0x7f70f419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fc10d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f4102580 0x7f70f419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fc10d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f419fa20 con 0x7f70f4101be0 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fc10d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f419fb90 con 0x7f70f4100ac0 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fc10d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f419fd50 con 0x7f70f4102580 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4101be0 0x7f70f419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4101be0 0x7f70f419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59416/0 (socket says 172.21.15.150:59416) 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 -- 172.21.15.150:0/2388218086 learned_addr learned my addr 172.21.15.150:0/2388218086 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f9ea9700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f4100ac0 0x7f70f4068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70fa6aa700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f4102580 0x7f70f419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 -- 172.21.15.150:0/2388218086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f4102580 msgr2=0x7f70f419f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f4102580 0x7f70f419f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 -- 172.21.15.150:0/2388218086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f4100ac0 msgr2=0x7f70f4068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f4100ac0 0x7f70f4068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 -- 172.21.15.150:0/2388218086 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00095f0 con 0x7f70f4101be0 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.955+0000 7f70f96a8700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4101be0 0x7f70f419cc90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f70e400e9a0 tx=0x7f70e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:14.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.956+0000 7f70eaffd700 1 -- 172.21.15.150:0/2388218086 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4009790 con 0x7f70f4101be0 2024-05-10T04:39:14.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.956+0000 7f70eaffd700 1 -- 172.21.15.150:0/2388218086 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e4010430 con 0x7f70f4101be0 2024-05-10T04:39:14.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.956+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f419ffd0 con 0x7f70f4101be0 2024-05-10T04:39:14.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.956+0000 7f70eaffd700 1 -- 172.21.15.150:0/2388218086 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4018680 con 0x7f70f4101be0 2024-05-10T04:39:14.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.956+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f41a03f0 con 0x7f70f4101be0 2024-05-10T04:39:14.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.957+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f4107cd0 con 0x7f70f4101be0 2024-05-10T04:39:14.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.958+0000 7f70eaffd700 1 -- 172.21.15.150:0/2388218086 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f70e40105a0 con 0x7f70f4101be0 2024-05-10T04:39:14.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.959+0000 7f70eaffd700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70e00a11b0 0x7f70e00a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:14.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.959+0000 7f70eaffd700 1 -- 172.21.15.150:0/2388218086 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f70e4014070 con 0x7f70f4101be0 2024-05-10T04:39:14.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.959+0000 7f70f9ea9700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70e00a11b0 0x7f70e00a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:14.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.959+0000 7f70f9ea9700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70e00a11b0 0x7f70e00a36c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70f0006010 tx=0x7f70f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:14.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:14.962+0000 7f70eaffd700 1 -- 172.21.15.150:0/2388218086 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f70e4096490 con 0x7f70f4101be0 2024-05-10T04:39:15.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.207+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70f4069470 con 0x7f70f4101be0 2024-05-10T04:39:15.209 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.208+0000 7f70eaffd700 1 -- 172.21.15.150:0/2388218086 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f70e4026090 con 0x7f70f4101be0 2024-05-10T04:39:15.209 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:15.210 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:35.978906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70e00a11b0 msgr2=0x7f70e00a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70e00a11b0 0x7f70e00a36c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70f0006010 tx=0x7f70f001a040 comp rx=0 tx=0).stop 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4101be0 msgr2=0x7f70f419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4101be0 0x7f70f419cc90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f70e400e9a0 tx=0x7f70e400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 shutdown_connections 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f70e00a11b0 0x7f70e00a36c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70f4102580 0x7f70f419f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f4100ac0 0x7f70f4068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 --2- 172.21.15.150:0/2388218086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70f4101be0 0x7f70f419cc90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.210+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 >> 172.21.15.150:0/2388218086 conn(0x7f70f40fc0d0 msgr2=0x7f70f4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.211+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 shutdown_connections 2024-05-10T04:39:15.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.211+0000 7f70fc10d700 1 -- 172.21.15.150:0/2388218086 wait complete. 2024-05-10T04:39:15.213 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 28 2024-05-10T04:39:15.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T04:38:08.750329+0000', 'modified': '2024-05-10T04:38:35.978906+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24548, 'mds_1': 24520, 'mds_2': 24484, 'mds_3': 24455, 'mds_4': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'e', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/2306790268', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 2306790268}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 2306790268}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14664': {'gid': 14664, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6839/308347873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 308347873}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 308347873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14676': {'gid': 14676, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6841/3789106275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6840', 'nonce': 3789106275}, {'type': 'v1', 'addr': '172.21.15.150:6841', 'nonce': 3789106275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.181:6833/1504167586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 1504167586}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 1504167586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.161:6833/825127258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 825127258}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 825127258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.181:6835/3090072990', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3090072990}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3090072990}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/1032605592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1032605592}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1032605592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/55043330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 55043330}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 55043330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.181:6839/4287682926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 4287682926}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 4287682926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/3001288873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 3001288873}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 3001288873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T04:39:15.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-10T04:39:15.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-10T04:39:15.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 -- 172.21.15.150:0/13169195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 msgr2=0x7f6790108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 --2- 172.21.15.150:0/13169195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f6790108e60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f678800b0a0 tx=0x7f678800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 -- 172.21.15.150:0/13169195 shutdown_connections 2024-05-10T04:39:15.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 --2- 172.21.15.150:0/13169195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f6790108e60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.360 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 --2- 172.21.15.150:0/13169195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6790100af0 0x7f6790100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 --2- 172.21.15.150:0/13169195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6790101c10 0x7f6790102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 -- 172.21.15.150:0/13169195 >> 172.21.15.150:0/13169195 conn(0x7f67900fc0e0 msgr2=0x7f67900fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 -- 172.21.15.150:0/13169195 shutdown_connections 2024-05-10T04:39:15.361 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 -- 172.21.15.150:0/13169195 wait complete. 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.359+0000 7f6795244700 1 Processor -- start 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f6795244700 1 -- start start 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f6795244700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6790100af0 0x7f6790073f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f6795244700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6790101c10 0x7f679019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f6795244700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f679019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f6795244700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679019fb30 con 0x7f67901025b0 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f6795244700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679019fca0 con 0x7f6790101c10 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f6795244700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679019fe60 con 0x7f6790100af0 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f679019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f679019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59434/0 (socket says 172.21.15.150:59434) 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 -- 172.21.15.150:0/2737979384 learned_addr learned my addr 172.21.15.150:0/2737979384 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678e59c700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6790101c10 0x7f679019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6790100af0 0x7f6790073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 -- 172.21.15.150:0/2737979384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6790100af0 msgr2=0x7f6790073f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6790100af0 0x7f6790073f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 -- 172.21.15.150:0/2737979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6790101c10 msgr2=0x7f679019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6790101c10 0x7f679019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.360+0000 7f678f59e700 1 -- 172.21.15.150:0/2737979384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788009d00 con 0x7f67901025b0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.361+0000 7f678f59e700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f679019f430 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6788007930 tx=0x7f6788007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.361+0000 7f6777fff700 1 -- 172.21.15.150:0/2737979384 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800e040 con 0x7f67901025b0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.361+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67901a00e0 con 0x7f67901025b0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.361+0000 7f6777fff700 1 -- 172.21.15.150:0/2737979384 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f678800be80 con 0x7f67901025b0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.361+0000 7f6777fff700 1 -- 172.21.15.150:0/2737979384 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880052a0 con 0x7f67901025b0 2024-05-10T04:39:15.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.361+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67901a05b0 con 0x7f67901025b0 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.363+0000 7f6777fff700 1 -- 172.21.15.150:0/2737979384 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6788005400 con 0x7f67901025b0 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.363+0000 7f6777fff700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67780a1260 0x7f67780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.364+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67780a1260 0x7f67780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.364+0000 7f6777fff700 1 -- 172.21.15.150:0/2737979384 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f67880ce220 con 0x7f67901025b0 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.364+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67780a1260 0x7f67780a3770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6790101b10 tx=0x7f6780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.364+0000 7f67a5588b80 1 Processor -- start 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.364+0000 7f67a5588b80 1 -- start start 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59aba60 0x5584d59abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59ac930 0x5584d59acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67a5588b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d58ce100 con 0x5584d59ac930 2024-05-10T04:39:15.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67a5588b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d58cbac0 con 0x5584d59ad280 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67a5588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d58c73b0 con 0x5584d59aba60 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59ac930 0x5584d59acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59ac930 0x5584d59acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59440/0 (socket says 172.21.15.150:59440) 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 -- 172.21.15.150:0/3099607973 learned_addr learned my addr 172.21.15.150:0/3099607973 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678ed9d700 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59aba60 0x5584d59abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678f59e700 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 -- 172.21.15.150:0/3099607973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59aba60 msgr2=0x5584d59abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59aba60 0x5584d59abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 -- 172.21.15.150:0/3099607973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 msgr2=0x5584d59b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 -- 172.21.15.150:0/3099607973 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d5748360 con 0x5584d59ac930 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678f59e700 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f678e59c700 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59ac930 0x5584d59acd40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f678400bd30 tx=0x7f67840106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c7c87f9f5af04f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67757fa700 1 -- 172.21.15.150:0/3099607973 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400be80 con 0x5584d59ac930 2024-05-10T04:39:15.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67757fa700 1 -- 172.21.15.150:0/3099607973 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6784015070 con 0x5584d59ac930 2024-05-10T04:39:15.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.365+0000 7f67757fa700 1 -- 172.21.15.150:0/3099607973 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840116e0 con 0x5584d59ac930 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 -- 172.21.15.150:0/3099607973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59ac930 msgr2=0x5584d59acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59ac930 0x5584d59acd40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f678400bd30 tx=0x7f67840106b0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 -- 172.21.15.150:0/3099607973 shutdown_connections 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59aba60 0x5584d59abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 --2- 172.21.15.150:0/3099607973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59ac930 0x5584d59acd40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 -- 172.21.15.150:0/3099607973 >> 172.21.15.150:0/3099607973 conn(0x5584d5909a10 msgr2=0x5584d59a1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 -- 172.21.15.150:0/3099607973 shutdown_connections 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.366+0000 7f67a5588b80 1 -- 172.21.15.150:0/3099607973 wait complete. 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 -- start start 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 Processor -- start 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5863510 con 0x5584d59aba60 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d58631b0 con 0x5584d59ac930 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d574a650 con 0x5584d59ad280 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f67a5588b80 10 client.0 _ll_register_callbacks cb 0x5584d586cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59446/0 (socket says 172.21.15.150:59446) 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f678ed9d700 1 -- 172.21.15.150:0/2376347311 learned_addr learned my addr 172.21.15.150:0/2376347311 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f678e59c700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f678ed9d700 1 -- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 msgr2=0x5584d59bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f678f59e700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.367+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59bcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f678ed9d700 1 -- 172.21.15.150:0/2376347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 msgr2=0x5584d59b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f678ed9d700 1 -- 172.21.15.150:0/2376347311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678400b960 con 0x5584d59aba60 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f678e59c700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b5aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f678001e8a0 tx=0x7f67800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eead17d191a4fc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f6772ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f6772ff5700 1 -- 172.21.15.150:0/2376347311 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000e940 con 0x5584d59aba60 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f6772ff5700 1 -- 172.21.15.150:0/2376347311 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f678000eaa0 con 0x5584d59aba60 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584d5820220 con 0x5584d59aba60 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f6772ff5700 1 -- 172.21.15.150:0/2376347311 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f678000ec60 con 0x5584d59aba60 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 10 client.15261 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5584d56f4b10 con 0x5584d59aba60 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f6772ff5700 1 -- 172.21.15.150:0/2376347311 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584d5820220 con 0x5584d59aba60 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f6772ff5700 1 -- 172.21.15.150:0/2376347311 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f678001c410 con 0x5584d59aba60 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 10 client.15261 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 20 client.15261 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d59c0030 0x5584d59c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 4 client.15261 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.368+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584d5976730 con 0x5584d59c0030 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.369+0000 7f678e59c700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d59c0030 0x5584d59c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.369+0000 7f6772ff5700 10 client.15261 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.369+0000 7f678e59c700 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d59c0030 0x5584d59c5ec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7f769d388af3fa7 server_cookie=9cb893d5c103aa29 in_seq=0 out_seq=0 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f6772ff5700 1 -- 172.21.15.150:0/2376347311 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584d5976730 con 0x5584d59c0030 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f6772ff5700 10 client.15261 handle_command_reply: tid=0 2024-05-10T04:39:15.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 client.15261 shutdown 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 msgr2=0x5584d59b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b5aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f678001e8a0 tx=0x7f67800047c0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 shutdown_connections 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d59c0030 0x5584d59c5ec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b5aa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2376347311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f6772ff5700 0 client.15261 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 >> 172.21.15.150:0/2376347311 conn(0x5584d5909a10 msgr2=0x5584d59a1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 shutdown_connections 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 1 -- 172.21.15.150:0/2376347311 wait complete. 2024-05-10T04:39:15.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.370+0000 7f67a5588b80 20 client.15261 trim_cache size 0 max 16384 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.388+0000 7f67a5588b80 1 Processor -- start 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.388+0000 7f67a5588b80 1 -- start start 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ac930 0x5584d59ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f67a5588b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5976760 con 0x5584d59aba60 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f67a5588b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5820250 con 0x5584d59ad280 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f67a5588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d58c0150 con 0x5584d59ac930 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59454/0 (socket says 172.21.15.150:59454) 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 -- 172.21.15.150:0/2665834654 learned_addr learned my addr 172.21.15.150:0/2665834654 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678f59e700 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 -- 172.21.15.150:0/2665834654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ac930 msgr2=0x5584d59ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ac930 0x5584d59ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 -- 172.21.15.150:0/2665834654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 msgr2=0x5584d59ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 -- 172.21.15.150:0/2665834654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d588d510 con 0x5584d59aba60 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678f59e700 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f678ed9d700 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59acd40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6780000c00 tx=0x7f6780004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825e43d3ebf726e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f6775ffb700 1 -- 172.21.15.150:0/2665834654 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000e3b0 con 0x5584d59aba60 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f6775ffb700 1 -- 172.21.15.150:0/2665834654 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67800044e0 con 0x5584d59aba60 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.389+0000 7f6775ffb700 1 -- 172.21.15.150:0/2665834654 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780003680 con 0x5584d59aba60 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 -- 172.21.15.150:0/2665834654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 msgr2=0x5584d59acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59acd40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6780000c00 tx=0x7f6780004750 comp rx=0 tx=0).stop 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 -- 172.21.15.150:0/2665834654 shutdown_connections 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ad280 0x5584d59ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59acd40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 --2- 172.21.15.150:0/2665834654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ac930 0x5584d59ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 -- 172.21.15.150:0/2665834654 >> 172.21.15.150:0/2665834654 conn(0x5584d5909a10 msgr2=0x5584d59afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 -- 172.21.15.150:0/2665834654 shutdown_connections 2024-05-10T04:39:15.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 -- 172.21.15.150:0/2665834654 wait complete. 2024-05-10T04:39:15.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.390+0000 7f67a5588b80 1 -- start start 2024-05-10T04:39:15.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 1 Processor -- start 2024-05-10T04:39:15.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d59ba700 con 0x5584d59aba60 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d59ba870 con 0x5584d59ac930 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d59baa30 con 0x5584d59ad280 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f67a5588b80 10 client.0 _ll_register_callbacks cb 0x5584d586cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59466/0 (socket says 172.21.15.150:59466) 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 -- 172.21.15.150:0/784084228 learned_addr learned my addr 172.21.15.150:0/784084228 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678e59c700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678f59e700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 -- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 msgr2=0x5584d59ba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59ba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 -- 172.21.15.150:0/784084228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 msgr2=0x5584d59b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 -- 172.21.15.150:0/784084228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584d56f4b10 con 0x5584d59aba60 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678f59e700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f678ed9d700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b8d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f678000ea00 tx=0x7f678000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbd7ab29a92930b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.391+0000 7f6772ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f6772ff5700 1 -- 172.21.15.150:0/784084228 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678001c960 con 0x5584d59aba60 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f6772ff5700 1 -- 172.21.15.150:0/784084228 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f678001cac0 con 0x5584d59aba60 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6784015070 con 0x5584d59aba60 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f6772ff5700 1 -- 172.21.15.150:0/784084228 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f678001cc80 con 0x5584d59aba60 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 10 client.15267 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5584d574cc80 con 0x5584d59aba60 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f6772ff5700 1 -- 172.21.15.150:0/784084228 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6784015070 con 0x5584d59aba60 2024-05-10T04:39:15.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f6772ff5700 1 -- 172.21.15.150:0/784084228 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67800049f0 con 0x5584d59aba60 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 10 client.15267 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 10 client.15267 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 20 client.15267 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d5a19ed0 0x5584d5a17e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 4 client.15267 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.392+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5584d58bfe70 con 0x5584d5a19ed0 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f678e59c700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d5a19ed0 0x5584d5a17e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f6772ff5700 10 client.15267 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f678e59c700 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d5a19ed0 0x5584d5a17e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b67acde796dfc3c9 server_cookie=92b147c21ef96617 in_seq=0 out_seq=0 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f6772ff5700 1 -- 172.21.15.150:0/784084228 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x5584d58bfe70 con 0x5584d5a19ed0 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f6772ff5700 10 client.15267 handle_command_reply: tid=0 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f67a5588b80 1 client.15267 shutdown 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 msgr2=0x5584d59b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.393+0000 7f67a5588b80 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b8d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f678000ea00 tx=0x7f678000ee30 comp rx=0 tx=0).stop 2024-05-10T04:39:15.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 shutdown_connections 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5584d5a19ed0 0x5584d5a17e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5584d59ad280 0x5584d59ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6772ff5700 0 client.15267 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584d59aba60 0x5584d59b8d50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 --2- 172.21.15.150:0/784084228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584d59ac930 0x5584d59b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 >> 172.21.15.150:0/784084228 conn(0x5584d5909a10 msgr2=0x5584d59b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 shutdown_connections 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 1 -- 172.21.15.150:0/784084228 wait complete. 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f67a5588b80 20 client.15267 trim_cache size 0 max 16384 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 4, 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24445, 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 40.435988025, 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 91.800031761 2024-05-10T04:39:15.402 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:15.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67780a1260 msgr2=0x7f67780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67780a1260 0x7f67780a3770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6790101b10 tx=0x7f6780009260 comp rx=0 tx=0).stop 2024-05-10T04:39:15.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 msgr2=0x7f679019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f679019f430 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6788007930 tx=0x7f6788007960 comp rx=0 tx=0).stop 2024-05-10T04:39:15.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 shutdown_connections 2024-05-10T04:39:15.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67901025b0 0x7f679019f430 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f67780a1260 0x7f67780a3770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6790100af0 0x7f6790073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 --2- 172.21.15.150:0/2737979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6790101c10 0x7f679019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 >> 172.21.15.150:0/2737979384 conn(0x7f67900fc0e0 msgr2=0x7f6790104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 shutdown_connections 2024-05-10T04:39:15.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.394+0000 7f6795244700 1 -- 172.21.15.150:0/2737979384 wait complete. 2024-05-10T04:39:15.409 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-05-10T04:39:15.546 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 -- 172.21.15.150:0/3573725783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 msgr2=0x7f7758100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 --2- 172.21.15.150:0/3573725783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758100ed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7748009990 tx=0x7f7748009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 -- 172.21.15.150:0/3573725783 shutdown_connections 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 --2- 172.21.15.150:0/3573725783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7758102580 0x7f7758108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 --2- 172.21.15.150:0/3573725783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758100ed0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 --2- 172.21.15.150:0/3573725783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7758101be0 0x7f7758101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 -- 172.21.15.150:0/3573725783 >> 172.21.15.150:0/3573725783 conn(0x7f77580fc0d0 msgr2=0x7f77580fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 -- 172.21.15.150:0/3573725783 shutdown_connections 2024-05-10T04:39:15.547 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.545+0000 7f775ea2b700 1 -- 172.21.15.150:0/3573725783 wait complete. 2024-05-10T04:39:15.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 Processor -- start 2024-05-10T04:39:15.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 -- start start 2024-05-10T04:39:15.549 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7758101be0 0x7f775819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7758102580 0x7f775819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775819f9e0 con 0x7f7758100ac0 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775819fb50 con 0x7f7758101be0 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f775ea2b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775819fd10 con 0x7f7758102580 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f7757fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.546+0000 7f7757fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59492/0 (socket says 172.21.15.150:59492) 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f7757fff700 1 -- 172.21.15.150:0/1311526774 learned_addr learned my addr 172.21.15.150:0/1311526774 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f77577fe700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7758101be0 0x7f775819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f775cfc8700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7758102580 0x7f775819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f7757fff700 1 -- 172.21.15.150:0/1311526774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7758102580 msgr2=0x7f775819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f7757fff700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7758102580 0x7f775819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f7757fff700 1 -- 172.21.15.150:0/1311526774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7758101be0 msgr2=0x7f775819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f7757fff700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7758101be0 0x7f775819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f7757fff700 1 -- 172.21.15.150:0/1311526774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480095f0 con 0x7f7758100ac0 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f775cfc8700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7758102580 0x7f775819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f77577fe700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7758101be0 0x7f775819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f7757fff700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758068e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f77480098e0 tx=0x7f7748009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f77557fa700 1 -- 172.21.15.150:0/1311526774 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774801d070 con 0x7f7758100ac0 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f77557fa700 1 -- 172.21.15.150:0/1311526774 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774800fbd0 con 0x7f7758100ac0 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775819ff30 con 0x7f7758100ac0 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f77557fa700 1 -- 172.21.15.150:0/1311526774 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748017650 con 0x7f7758100ac0 2024-05-10T04:39:15.551 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.547+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77581a02a0 con 0x7f7758100ac0 2024-05-10T04:39:15.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.550+0000 7f77557fa700 1 -- 172.21.15.150:0/1311526774 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f77480177b0 con 0x7f7758100ac0 2024-05-10T04:39:15.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.550+0000 7f77557fa700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77400a1260 0x7f77400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.550+0000 7f77557fa700 1 -- 172.21.15.150:0/1311526774 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77480ce260 con 0x7f7758100ac0 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.550+0000 7f77577fe700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77400a1260 0x7f77400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f77577fe700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77400a1260 0x7f77400a3770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77580687f0 tx=0x7f774c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 Processor -- start 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 -- start start 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7806650 0x5587f7806aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807520 0x5587f7807930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807e70 0x5587f780e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f772d100 con 0x5587f7806650 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f772aac0 con 0x5587f7807520 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f77263b0 con 0x5587f7807e70 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f7757fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7806650 0x5587f7806aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f7757fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7806650 0x5587f7806aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59498/0 (socket says 172.21.15.150:59498) 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f7757fff700 1 -- 172.21.15.150:0/861728012 learned_addr learned my addr 172.21.15.150:0/861728012 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f775cfc8700 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807e70 0x5587f780e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f77577fe700 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807520 0x5587f7807930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f7757fff700 1 -- 172.21.15.150:0/861728012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807e70 msgr2=0x5587f780e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f7757fff700 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807e70 0x5587f780e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f7757fff700 1 -- 172.21.15.150:0/861728012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807520 msgr2=0x5587f7807930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.551+0000 7f7757fff700 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807520 0x5587f7807930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.552+0000 7f7757fff700 1 -- 172.21.15.150:0/861728012 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587f75a7360 con 0x5587f7806650 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.552+0000 7f7757fff700 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7806650 0x5587f7806aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f774809b310 tx=0x7f7748009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b689eafd59e162 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.552+0000 7f773e7fc700 1 -- 172.21.15.150:0/861728012 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774809ba80 con 0x5587f7806650 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.552+0000 7f773e7fc700 1 -- 172.21.15.150:0/861728012 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774809b380 con 0x5587f7806650 2024-05-10T04:39:15.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.552+0000 7f773e7fc700 1 -- 172.21.15.150:0/861728012 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774809a870 con 0x5587f7806650 2024-05-10T04:39:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 -- 172.21.15.150:0/861728012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7806650 msgr2=0x5587f7806aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7806650 0x5587f7806aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f774809b310 tx=0x7f7748009310 comp rx=0 tx=0).stop 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 -- 172.21.15.150:0/861728012 shutdown_connections 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807e70 0x5587f780e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7806650 0x5587f7806aa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/861728012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807520 0x5587f7807930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 -- 172.21.15.150:0/861728012 >> 172.21.15.150:0/861728012 conn(0x5587f7768a10 msgr2=0x5587f7800f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 -- 172.21.15.150:0/861728012 shutdown_connections 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 -- 172.21.15.150:0/861728012 wait complete. 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 -- start start 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.553+0000 7f776ed6fb80 1 Processor -- start 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7806650 0x5587f7810260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807520 0x5587f78107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807e70 0x5587f7817700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f76c2510 con 0x5587f7807e70 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f76c21b0 con 0x5587f7806650 2024-05-10T04:39:15.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f75a9650 con 0x5587f7807520 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f776ed6fb80 10 client.0 _ll_register_callbacks cb 0x5587f76cbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807e70 0x5587f7817700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f7757fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7806650 0x5587f7810260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f77577fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807520 0x5587f78107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807e70 0x5587f7817700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59504/0 (socket says 172.21.15.150:59504) 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 -- 172.21.15.150:0/3523021698 learned_addr learned my addr 172.21.15.150:0/3523021698 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 -- 172.21.15.150:0/3523021698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807520 msgr2=0x5587f78107a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807520 0x5587f78107a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 -- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7806650 msgr2=0x5587f7810260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7806650 0x5587f7810260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 -- 172.21.15.150:0/3523021698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77581a02a0 con 0x5587f7807e70 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f7757fff700 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7806650 0x5587f7810260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f775cfc8700 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807e70 0x5587f7817700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7750007ba0 tx=0x7f7750007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9d0ff0389014ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.554+0000 7f77377fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:15.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f77377fe700 1 -- 172.21.15.150:0/3523021698 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750010860 con 0x5587f7807e70 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f77377fe700 1 -- 172.21.15.150:0/3523021698 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77500109c0 con 0x5587f7807e70 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587f767f220 con 0x5587f7807e70 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f77377fe700 1 -- 172.21.15.150:0/3523021698 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7750009890 con 0x5587f7807e70 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 10 client.15279 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587f7553b10 con 0x5587f7807e70 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f77377fe700 1 -- 172.21.15.150:0/3523021698 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587f767f220 con 0x5587f7807e70 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f77377fe700 1 -- 172.21.15.150:0/3523021698 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7750016d20 con 0x5587f7807e70 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 10 client.15279 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 10 client.15279 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 20 client.15279 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7820720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 4 client.15279 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587f77d5730 con 0x5587f781b9f0 2024-05-10T04:39:15.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.555+0000 7f7757fff700 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7820720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.556+0000 7f77377fe700 10 client.15279 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.556+0000 7f7757fff700 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7820720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4b81b26d320495f server_cookie=836ac5409fcf587d in_seq=0 out_seq=0 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f77377fe700 1 -- 172.21.15.150:0/3523021698 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587f77d5730 con 0x5587f781b9f0 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f77377fe700 10 client.15279 handle_command_reply: tid=0 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 client.15279 shutdown 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807e70 msgr2=0x5587f7817700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807e70 0x5587f7817700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7750007ba0 tx=0x7f7750007eb0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 shutdown_connections 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7820720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807e70 0x5587f7817700 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f77377fe700 0 client.15279 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7806650 0x5587f7810260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/3523021698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7807520 0x5587f78107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 >> 172.21.15.150:0/3523021698 conn(0x5587f7768a10 msgr2=0x5587f7800f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 shutdown_connections 2024-05-10T04:39:15.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 1 -- 172.21.15.150:0/3523021698 wait complete. 2024-05-10T04:39:15.564 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.557+0000 7f776ed6fb80 20 client.15279 trim_cache size 0 max 16384 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 Processor -- start 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 -- start start 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f780b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7815540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f7815a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f77d5730 con 0x5587f7807520 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f767f220 con 0x5587f7807e70 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f771f150 con 0x5587f7806650 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f77577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7815540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f77577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7815540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59518/0 (socket says 172.21.15.150:59518) 2024-05-10T04:39:15.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f77577fe700 1 -- 172.21.15.150:0/1945028569 learned_addr learned my addr 172.21.15.150:0/1945028569 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f775cfc8700 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f7815a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.575+0000 7f7757fff700 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f780b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f77577fe700 1 -- 172.21.15.150:0/1945028569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 msgr2=0x5587f780b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f77577fe700 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f780b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f77577fe700 1 -- 172.21.15.150:0/1945028569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 msgr2=0x5587f7815a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f77577fe700 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f7815a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f77577fe700 1 -- 172.21.15.150:0/1945028569 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587f76ec510 con 0x5587f7807520 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f77577fe700 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7815540 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f774c012550 tx=0x7f774c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a0d535999ab6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f773effd700 1 -- 172.21.15.150:0/1945028569 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c02a640 con 0x5587f7807520 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f773effd700 1 -- 172.21.15.150:0/1945028569 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774c016070 con 0x5587f7807520 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.576+0000 7f773effd700 1 -- 172.21.15.150:0/1945028569 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c020960 con 0x5587f7807520 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- 172.21.15.150:0/1945028569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 msgr2=0x5587f7815540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7815540 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f774c012550 tx=0x7f774c014000 comp rx=0 tx=0).stop 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- 172.21.15.150:0/1945028569 shutdown_connections 2024-05-10T04:39:15.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f7815a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f780b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/1945028569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7815540 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- 172.21.15.150:0/1945028569 >> 172.21.15.150:0/1945028569 conn(0x5587f7768a10 msgr2=0x5587f7800f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- 172.21.15.150:0/1945028569 shutdown_connections 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- 172.21.15.150:0/1945028569 wait complete. 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- start start 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 Processor -- start 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f78157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7813ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f78143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f7814920 con 0x5587f7807520 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f7814a90 con 0x5587f7807e70 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.577+0000 7f776ed6fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587f7814c50 con 0x5587f7806650 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f776ed6fb80 10 client.0 _ll_register_callbacks cb 0x5587f76cbae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7813ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7813ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59526/0 (socket says 172.21.15.150:59526) 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 -- 172.21.15.150:0/2151212103 learned_addr learned my addr 172.21.15.150:0/2151212103 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f775cfc8700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f78143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f7757fff700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f78157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 -- 172.21.15.150:0/2151212103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 msgr2=0x5587f78157e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f78157e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 -- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 msgr2=0x5587f78143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f78143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 -- 172.21.15.150:0/2151212103 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c011e10 con 0x5587f7807520 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f7757fff700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f78157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77577fe700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7813ea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f774c020470 tx=0x7f774c02ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d96ee508995ddd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77377fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77377fe700 1 -- 172.21.15.150:0/2151212103 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c02f070 con 0x5587f7807520 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77377fe700 1 -- 172.21.15.150:0/2151212103 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f774c018070 con 0x5587f7807520 2024-05-10T04:39:15.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f774809b380 con 0x5587f7807520 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f77377fe700 1 -- 172.21.15.150:0/2151212103 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f774c0208f0 con 0x5587f7807520 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f776ed6fb80 10 client.15285 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.578+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5587f75abc80 con 0x5587f7807520 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f77377fe700 1 -- 172.21.15.150:0/2151212103 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774809b380 con 0x5587f7807520 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f77377fe700 1 -- 172.21.15.150:0/2151212103 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f774c01fb70 con 0x5587f7807520 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f776ed6fb80 10 client.15285 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f776ed6fb80 10 client.15285 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f776ed6fb80 20 client.15285 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7873850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f776ed6fb80 4 client.15285 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5587f771ee70 con 0x5587f781b9f0 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f7757fff700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7873850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f77377fe700 10 client.15285 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.579+0000 7f7757fff700 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7873850 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab96e0bb4e9f6440 server_cookie=88f38eafb86d6540 in_seq=0 out_seq=0 2024-05-10T04:39:15.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f77377fe700 1 -- 172.21.15.150:0/2151212103 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5587f771ee70 con 0x5587f781b9f0 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f77377fe700 10 client.15285 handle_command_reply: tid=0 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 client.15285 shutdown 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 msgr2=0x5587f7813ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7813ea0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f774c020470 tx=0x7f774c02ac00 comp rx=0 tx=0).stop 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 shutdown_connections 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5587f781b9f0 0x5587f7873850 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587f7807e70 0x5587f78143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5587f7806650 0x5587f78157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f77377fe700 0 client.15285 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 --2- 172.21.15.150:0/2151212103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587f7807520 0x5587f7813ea0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 >> 172.21.15.150:0/2151212103 conn(0x5587f7768a10 msgr2=0x5587f7824740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 shutdown_connections 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 1 -- 172.21.15.150:0/2151212103 wait complete. 2024-05-10T04:39:15.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f776ed6fb80 20 client.15285 trim_cache size 0 max 16384 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 3, 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24455, 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 41.626839244000003, 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:15.588 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 90.308054400000003 2024-05-10T04:39:15.589 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:15.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77400a1260 msgr2=0x7f77400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f775ea2b700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77400a1260 0x7f77400a3770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77580687f0 tx=0x7f774c009260 comp rx=0 tx=0).stop 2024-05-10T04:39:15.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 msgr2=0x7f7758068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.580+0000 7f775ea2b700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758068e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f77480098e0 tx=0x7f7748009580 comp rx=0 tx=0).stop 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 shutdown_connections 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77400a1260 0x7f77400a3770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7758102580 0x7f775819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7758100ac0 0x7f7758068e50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 --2- 172.21.15.150:0/1311526774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7758101be0 0x7f775819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 >> 172.21.15.150:0/1311526774 conn(0x7f77580fc0d0 msgr2=0x7f7758104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 shutdown_connections 2024-05-10T04:39:15.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.581+0000 7f775ea2b700 1 -- 172.21.15.150:0/1311526774 wait complete. 2024-05-10T04:39:15.593 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:15 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2069185111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:15.593 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:15 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2388218086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:15.594 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:15 smithi150 ceph-mon[42404]: pgmap v260: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 6 op/s; 189 B/s, 0 objects/s recovering 2024-05-10T04:39:15.595 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 -- 172.21.15.150:0/759970499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578102580 msgr2=0x7f9578108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 --2- 172.21.15.150:0/759970499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578102580 0x7f9578108e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f957000b0a0 tx=0x7f957000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 -- 172.21.15.150:0/759970499 shutdown_connections 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 --2- 172.21.15.150:0/759970499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578102580 0x7f9578108e30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 --2- 172.21.15.150:0/759970499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9578100ac0 0x7f9578100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 --2- 172.21.15.150:0/759970499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9578101be0 0x7f9578101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 -- 172.21.15.150:0/759970499 >> 172.21.15.150:0/759970499 conn(0x7f95780fc0d0 msgr2=0x7f95780fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 -- 172.21.15.150:0/759970499 shutdown_connections 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.727+0000 7f957dd49700 1 -- 172.21.15.150:0/759970499 wait complete. 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 Processor -- start 2024-05-10T04:39:15.731 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 -- start start 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578100ac0 0x7f9578197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9578101be0 0x7f95781984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9578102580 0x7f957819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957819fa90 con 0x7f9578100ac0 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957819fc00 con 0x7f9578102580 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f957dd49700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957819fdc0 con 0x7f9578101be0 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f95777fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578100ac0 0x7f9578197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f95777fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578100ac0 0x7f9578197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59548/0 (socket says 172.21.15.150:59548) 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f95777fe700 1 -- 172.21.15.150:0/4198689771 learned_addr learned my addr 172.21.15.150:0/4198689771 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f9576ffd700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9578101be0 0x7f95781984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f9577fff700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9578102580 0x7f957819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f95777fe700 1 -- 172.21.15.150:0/4198689771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9578101be0 msgr2=0x7f95781984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f95777fe700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9578101be0 0x7f95781984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.728+0000 7f95777fe700 1 -- 172.21.15.150:0/4198689771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9578102580 msgr2=0x7f957819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f95777fe700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9578102580 0x7f957819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f95777fe700 1 -- 172.21.15.150:0/4198689771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570009d00 con 0x7f9578100ac0 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f9577fff700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9578102580 0x7f957819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f95777fe700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578100ac0 0x7f9578197fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f956c00d950 tx=0x7f956c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f9574ff9700 1 -- 172.21.15.150:0/4198689771 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c004220 con 0x7f9578100ac0 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95781a00a0 con 0x7f9578100ac0 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f9574ff9700 1 -- 172.21.15.150:0/4198689771 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f956c004380 con 0x7f9578100ac0 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f9574ff9700 1 -- 172.21.15.150:0/4198689771 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c003bf0 con 0x7f9578100ac0 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.729+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95781a05b0 con 0x7f9578100ac0 2024-05-10T04:39:15.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.732+0000 7f9574ff9700 1 -- 172.21.15.150:0/4198689771 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f956c003d50 con 0x7f9578100ac0 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 Processor -- start 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 -- start start 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f9574ff9700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95600a1260 0x7f95600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f9577fff700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95600a1260 0x7f95600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f9574ff9700 1 -- 172.21.15.150:0/4198689771 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f956c0cc920 con 0x7f9578100ac0 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed6650 0x558191ed6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ed7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ede770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191dfd100 con 0x558191ed7e70 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191dfaac0 con 0x558191ed7520 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f958e08db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191df63b0 con 0x558191ed6650 2024-05-10T04:39:15.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f9577fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ede770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f9577fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ede770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59558/0 (socket says 172.21.15.150:59558) 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.733+0000 7f9577fff700 1 -- 172.21.15.150:0/3639484338 learned_addr learned my addr 172.21.15.150:0/3639484338 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f95777fe700 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed6650 0x558191ed6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9576ffd700 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ed7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9577fff700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95600a1260 0x7f95600a3770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9570015000 tx=0x7f9570008ff0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9577fff700 1 -- 172.21.15.150:0/3639484338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed6650 msgr2=0x558191ed6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9577fff700 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed6650 0x558191ed6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9577fff700 1 -- 172.21.15.150:0/3639484338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 msgr2=0x558191ed7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9577fff700 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ed7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9577fff700 1 -- 172.21.15.150:0/3639484338 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191c77360 con 0x558191ed7e70 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f95777fe700 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed6650 0x558191ed6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f9577fff700 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ede770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f95700075c0 tx=0x7f9570026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487be20d587d4fb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f955dffb700 1 -- 172.21.15.150:0/3639484338 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000e040 con 0x558191ed7e70 2024-05-10T04:39:15.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f955dffb700 1 -- 172.21.15.150:0/3639484338 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9570009460 con 0x558191ed7e70 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.734+0000 7f955dffb700 1 -- 172.21.15.150:0/3639484338 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957001d450 con 0x558191ed7e70 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 -- 172.21.15.150:0/3639484338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 msgr2=0x558191ede770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ede770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f95700075c0 tx=0x7f9570026000 comp rx=0 tx=0).stop 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 -- 172.21.15.150:0/3639484338 shutdown_connections 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ede770 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed6650 0x558191ed6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 --2- 172.21.15.150:0/3639484338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ed7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 -- 172.21.15.150:0/3639484338 >> 172.21.15.150:0/3639484338 conn(0x558191e38a10 msgr2=0x558191ed0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 -- 172.21.15.150:0/3639484338 shutdown_connections 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 -- 172.21.15.150:0/3639484338 wait complete. 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.735+0000 7f958e08db80 1 -- start start 2024-05-10T04:39:15.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 1 Processor -- start 2024-05-10T04:39:15.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed6650 0x558191ee0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ee0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191ee7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.744 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191d92510 con 0x558191ed6650 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191d921b0 con 0x558191ed7520 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191c79650 con 0x558191ed7e70 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f958e08db80 10 client.0 _ll_register_callbacks cb 0x558191d9bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f95777fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed6650 0x558191ee0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f95777fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed6650 0x558191ee0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59562/0 (socket says 172.21.15.150:59562) 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f95777fe700 1 -- 172.21.15.150:0/3230217802 learned_addr learned my addr 172.21.15.150:0/3230217802 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f9576ffd700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ee0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.736+0000 7f9577fff700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191ee7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9576ffd700 1 -- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 msgr2=0x558191ee7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9576ffd700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191ee7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9576ffd700 1 -- 172.21.15.150:0/3230217802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed6650 msgr2=0x558191ee0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9576ffd700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed6650 0x558191ee0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9576ffd700 1 -- 172.21.15.150:0/3230217802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95781098c0 con 0x558191ed7520 2024-05-10T04:39:15.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f95777fe700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed6650 0x558191ee0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9576ffd700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ee0ad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9568005950 tx=0x7f956800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ddd904102dd5613 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9556ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9556ffd700 1 -- 172.21.15.150:0/3230217802 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568016070 con 0x558191ed7520 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9556ffd700 1 -- 172.21.15.150:0/3230217802 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9568004520 con 0x558191ed7520 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558191d4f220 con 0x558191ed7520 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9556ffd700 1 -- 172.21.15.150:0/3230217802 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9568003d00 con 0x558191ed7520 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f958e08db80 10 client.25082 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558191c23b10 con 0x558191ed7520 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.737+0000 7f9556ffd700 1 -- 172.21.15.150:0/3230217802 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558191d4f220 con 0x558191ed7520 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f9556ffd700 1 -- 172.21.15.150:0/3230217802 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9568011560 con 0x558191ed7520 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f958e08db80 10 client.25082 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f958e08db80 10 client.25082 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f958e08db80 20 client.25082 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f958e08db80 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191eeb680 0x558191ef0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f958e08db80 4 client.25082 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558191ea5730 con 0x558191eeb680 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f95777fe700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191eeb680 0x558191ef0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f9556ffd700 10 client.25082 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.738+0000 7f95777fe700 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191eeb680 0x558191ef0980 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43a91290136059a0 server_cookie=251a9d7958471955 in_seq=0 out_seq=0 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f9556ffd700 1 -- 172.21.15.150:0/3230217802 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558191ea5730 con 0x558191eeb680 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f9556ffd700 10 client.25082 handle_command_reply: tid=0 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 client.25082 shutdown 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 msgr2=0x558191ee0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ee0ad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9568005950 tx=0x7f956800be20 comp rx=0 tx=0).stop 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 shutdown_connections 2024-05-10T04:39:15.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191eeb680 0x558191ef0980 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191ee7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed6650 0x558191ee0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 --2- 172.21.15.150:0/3230217802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed7520 0x558191ee0ad0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f9556ffd700 0 client.25082 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.739+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 >> 172.21.15.150:0/3230217802 conn(0x558191e38a10 msgr2=0x558191ed0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.740+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 shutdown_connections 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.740+0000 7f958e08db80 1 -- 172.21.15.150:0/3230217802 wait complete. 2024-05-10T04:39:15.748 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.740+0000 7f958e08db80 20 client.25082 trim_cache size 0 max 16384 2024-05-10T04:39:15.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 Processor -- start 2024-05-10T04:39:15.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 -- start start 2024-05-10T04:39:15.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ed7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7520 0x558191ef50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ee59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191db42b0 con 0x558191ed7e70 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191def150 con 0x558191ed6650 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f958e08db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191df3010 con 0x558191ed7520 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ee59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ee59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59564/0 (socket says 172.21.15.150:59564) 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 -- 172.21.15.150:0/3178448626 learned_addr learned my addr 172.21.15.150:0/3178448626 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f95777fe700 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ed7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9576ffd700 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7520 0x558191ef50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 -- 172.21.15.150:0/3178448626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7520 msgr2=0x558191ef50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.766 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7520 0x558191ef50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 -- 172.21.15.150:0/3178448626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 msgr2=0x558191ed7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ed7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 -- 172.21.15.150:0/3178448626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191dbc510 con 0x558191ed7e70 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9576ffd700 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7520 0x558191ef50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f9577fff700 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ee59e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9570009a20 tx=0x7f95700061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdd59243a9a36dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.758+0000 7f955e7fc700 1 -- 172.21.15.150:0/3178448626 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570029db0 con 0x558191ed7e70 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f955e7fc700 1 -- 172.21.15.150:0/3178448626 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9570029460 con 0x558191ed7e70 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f955e7fc700 1 -- 172.21.15.150:0/3178448626 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957001d7d0 con 0x558191ed7e70 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 -- 172.21.15.150:0/3178448626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 msgr2=0x558191ee59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ee59e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9570009a20 tx=0x7f95700061f0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 -- 172.21.15.150:0/3178448626 shutdown_connections 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7e70 0x558191ee59e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ed7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 --2- 172.21.15.150:0/3178448626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7520 0x558191ef50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.767 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 -- 172.21.15.150:0/3178448626 >> 172.21.15.150:0/3178448626 conn(0x558191e38a10 msgr2=0x558191ef49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 -- 172.21.15.150:0/3178448626 shutdown_connections 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.759+0000 7f958e08db80 1 -- 172.21.15.150:0/3178448626 wait complete. 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 -- start start 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 Processor -- start 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ee5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7520 0x558191ee59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191f40aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191f40fe0 con 0x558191ed7520 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191f41120 con 0x558191ed6650 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558191f412b0 con 0x558191ed7e70 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f958e08db80 10 client.0 _ll_register_callbacks cb 0x558191d9bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.760+0000 7f9576ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7520 0x558191ee59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.768 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7520 0x558191ee59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59580/0 (socket says 172.21.15.150:59580) 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 -- 172.21.15.150:0/1345391849 learned_addr learned my addr 172.21.15.150:0/1345391849 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f95777fe700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ee5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9577fff700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191f40aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 -- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 msgr2=0x558191f40aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191f40aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 -- 172.21.15.150:0/1345391849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 msgr2=0x558191ee5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ee5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 -- 172.21.15.150:0/1345391849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558191c77360 con 0x558191ed7520 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9577fff700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191f40aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9576ffd700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7520 0x558191ee59c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9568011c60 tx=0x7f9568012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94038b24993c496b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9556ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9556ffd700 1 -- 172.21.15.150:0/1345391849 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568016070 con 0x558191ed7520 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9556ffd700 1 -- 172.21.15.150:0/1345391849 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9568000e40 con 0x558191ed7520 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9570009460 con 0x558191ed7520 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9556ffd700 1 -- 172.21.15.150:0/1345391849 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9568011440 con 0x558191ed7520 2024-05-10T04:39:15.769 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f958e08db80 10 client.15303 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558191c7bc80 con 0x558191ed7520 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.761+0000 7f9556ffd700 1 -- 172.21.15.150:0/1345391849 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9570009460 con 0x558191ed7520 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f9556ffd700 1 -- 172.21.15.150:0/1345391849 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9568004690 con 0x558191ed7520 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f958e08db80 10 client.15303 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f958e08db80 10 client.15303 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f958e08db80 20 client.15303 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f958e08db80 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191f44490 0x558191f429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f958e08db80 4 client.15303 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558191deee70 con 0x558191f44490 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f95777fe700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191f44490 0x558191f429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f9556ffd700 10 client.15303 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.762+0000 7f95777fe700 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191f44490 0x558191f429f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc2371ac256d419a server_cookie=be34baf976ee88ea in_seq=0 out_seq=0 2024-05-10T04:39:15.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f9556ffd700 1 -- 172.21.15.150:0/1345391849 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558191deee70 con 0x558191f44490 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f9556ffd700 10 client.15303 handle_command_reply: tid=0 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 client.15303 shutdown 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7520 msgr2=0x558191ee59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7520 0x558191ee59c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9568011c60 tx=0x7f9568012e00 comp rx=0 tx=0).stop 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 shutdown_connections 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x558191f44490 0x558191f429f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558191ed7e70 0x558191f40aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558191ed6650 0x558191ee5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 --2- 172.21.15.150:0/1345391849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558191ed7520 0x558191ee59c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f9556ffd700 0 client.15303 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 >> 172.21.15.150:0/1345391849 conn(0x558191e38a10 msgr2=0x558191ed5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 shutdown_connections 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 1 -- 172.21.15.150:0/1345391849 wait complete. 2024-05-10T04:39:15.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f958e08db80 20 client.15303 trim_cache size 0 max 16384 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 2, 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24484, 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 42.823331453000002, 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:15.772 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:15.773 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 83.596526307000005 2024-05-10T04:39:15.773 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:15.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95600a1260 msgr2=0x7f95600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f957dd49700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95600a1260 0x7f95600a3770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9570015000 tx=0x7f9570008ff0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578100ac0 msgr2=0x7f9578197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.763+0000 7f957dd49700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578100ac0 0x7f9578197fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f956c00d950 tx=0x7f956c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 shutdown_connections 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9578102580 0x7f957819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9578100ac0 0x7f9578197fb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95600a1260 0x7f95600a3770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 --2- 172.21.15.150:0/4198689771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9578101be0 0x7f95781984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 >> 172.21.15.150:0/4198689771 conn(0x7f95780fc0d0 msgr2=0x7f95780fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 shutdown_connections 2024-05-10T04:39:15.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.764+0000 7f957dd49700 1 -- 172.21.15.150:0/4198689771 wait complete. 2024-05-10T04:39:15.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-10T04:39:15.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:15 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2069185111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:15.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:15 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2388218086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:15.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:15 smithi161 ceph-mon[46912]: pgmap v260: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 6 op/s; 189 B/s, 0 objects/s recovering 2024-05-10T04:39:15.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:15 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2069185111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:15.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:15 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2388218086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:15.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:15 smithi181 ceph-mon[47314]: pgmap v260: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 6 op/s; 189 B/s, 0 objects/s recovering 2024-05-10T04:39:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 -- 172.21.15.150:0/1017098237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4100ac0 msgr2=0x7f89f4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1017098237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4100ac0 0x7f89f4100ed0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009990 tx=0x7f89e8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 -- 172.21.15.150:0/1017098237 shutdown_connections 2024-05-10T04:39:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1017098237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89f4102580 0x7f89f4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1017098237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4100ac0 0x7f89f4100ed0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1017098237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89f4101be0 0x7f89f4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 -- 172.21.15.150:0/1017098237 >> 172.21.15.150:0/1017098237 conn(0x7f89f40fc0d0 msgr2=0x7f89f40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 -- 172.21.15.150:0/1017098237 shutdown_connections 2024-05-10T04:39:15.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.910+0000 7f89f9e62700 1 -- 172.21.15.150:0/1017098237 wait complete. 2024-05-10T04:39:15.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.911+0000 7f89f9e62700 1 Processor -- start 2024-05-10T04:39:15.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f9e62700 1 -- start start 2024-05-10T04:39:15.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f9e62700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89f4100ac0 0x7f89f4196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f9e62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89f4101be0 0x7f89f41972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f9e62700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4102580 0x7f89f419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f9e62700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419e820 con 0x7f89f4102580 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f9e62700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419e960 con 0x7f89f4100ac0 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f9e62700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419eaf0 con 0x7f89f4101be0 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4102580 0x7f89f419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4102580 0x7f89f419e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59596/0 (socket says 172.21.15.150:59596) 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 -- 172.21.15.150:0/1108814629 learned_addr learned my addr 172.21.15.150:0/1108814629 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89f4101be0 0x7f89f41972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f37fe700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89f4100ac0 0x7f89f4196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 -- 172.21.15.150:0/1108814629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89f4101be0 msgr2=0x7f89f41972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89f4101be0 0x7f89f41972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 -- 172.21.15.150:0/1108814629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89f4100ac0 msgr2=0x7f89f4196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89f4100ac0 0x7f89f4196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 -- 172.21.15.150:0/1108814629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80095f0 con 0x7f89f4102580 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.912+0000 7f89f3fff700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4102580 0x7f89f419e250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007f10 tx=0x7f89ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.913+0000 7f89f0ff9700 1 -- 172.21.15.150:0/1108814629 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00dd20 con 0x7f89f4102580 2024-05-10T04:39:15.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.913+0000 7f89f0ff9700 1 -- 172.21.15.150:0/1108814629 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ec00f040 con 0x7f89f4102580 2024-05-10T04:39:15.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.913+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f419edd0 con 0x7f89f4102580 2024-05-10T04:39:15.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.913+0000 7f89f0ff9700 1 -- 172.21.15.150:0/1108814629 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0048a0 con 0x7f89f4102580 2024-05-10T04:39:15.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.913+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f419f220 con 0x7f89f4102580 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.915+0000 7f89f0ff9700 1 -- 172.21.15.150:0/1108814629 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f89ec015400 con 0x7f89f4102580 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.915+0000 7f89f0ff9700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f89dc0a12e0 0x7f89dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f89f0ff9700 1 -- 172.21.15.150:0/1108814629 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f89ec0cc4f0 con 0x7f89f4102580 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f89f37fe700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f89dc0a12e0 0x7f89dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f89f37fe700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f89dc0a12e0 0x7f89dc0a37f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89e80098e0 tx=0x7f89e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 Processor -- start 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 -- start start 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888f1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d88817100 con 0x555d888f1520 2024-05-10T04:39:15.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d88814ac0 con 0x555d888f0650 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d888103b0 con 0x555d888f1e70 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59598/0 (socket says 172.21.15.150:59598) 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.916+0000 7f89f2ffd700 1 -- 172.21.15.150:0/3108200871 learned_addr learned my addr 172.21.15.150:0/3108200871 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f37fe700 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f3fff700 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f2ffd700 1 -- 172.21.15.150:0/3108200871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 msgr2=0x555d888f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f2ffd700 1 -- 172.21.15.150:0/3108200871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 msgr2=0x555d888f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f2ffd700 1 -- 172.21.15.150:0/3108200871 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d88691360 con 0x555d888f1520 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888f1930 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f89e400ac70 tx=0x7f89e400af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0b3391a2a3bc4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89d9ffb700 1 -- 172.21.15.150:0/3108200871 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4014070 con 0x555d888f1520 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89d9ffb700 1 -- 172.21.15.150:0/3108200871 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e40044f0 con 0x555d888f1520 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.917+0000 7f89d9ffb700 1 -- 172.21.15.150:0/3108200871 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400c7a0 con 0x555d888f1520 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/3108200871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 msgr2=0x555d888f1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888f1930 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f89e400ac70 tx=0x7f89e400af80 comp rx=0 tx=0).stop 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/3108200871 shutdown_connections 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/3108200871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888f1930 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/3108200871 >> 172.21.15.150:0/3108200871 conn(0x555d88852a10 msgr2=0x555d888eaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/3108200871 shutdown_connections 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/3108200871 wait complete. 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.918+0000 7f8a0a1a6b80 1 -- start start 2024-05-10T04:39:15.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 1 Processor -- start 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d88901ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d887ac510 con 0x555d888f1520 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d887ac1b0 con 0x555d888f0650 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d88693650 con 0x555d888f1e70 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f8a0a1a6b80 10 client.0 _ll_register_callbacks cb 0x555d887b5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59600/0 (socket says 172.21.15.150:59600) 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 -- 172.21.15.150:0/2949052537 learned_addr learned my addr 172.21.15.150:0/2949052537 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f37fe700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f3fff700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d88901ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 -- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 msgr2=0x555d88901ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d88901ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 -- 172.21.15.150:0/2949052537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 msgr2=0x555d888fa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f2ffd700 1 -- 172.21.15.150:0/2949052537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e400a950 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.919+0000 7f89f3fff700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d88901ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fab50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f89e4004020 tx=0x7f89e4004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba72bbb49b3aadc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f89d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f89d2ffd700 1 -- 172.21.15.150:0/2949052537 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4014040 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f89d2ffd700 1 -- 172.21.15.150:0/2949052537 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f89e4004320 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d88769220 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f89d2ffd700 1 -- 172.21.15.150:0/2949052537 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e400c750 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 10 client.15315 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d8863db10 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f89d2ffd700 1 -- 172.21.15.150:0/2949052537 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d88769220 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f89d2ffd700 1 -- 172.21.15.150:0/2949052537 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f89e4020070 con 0x555d888f1520 2024-05-10T04:39:15.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 10 client.15315 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 20 client.15315 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d88905cf0 0x555d8890aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 4 client.15315 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.920+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d888bf730 con 0x555d88905cf0 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.921+0000 7f89f37fe700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d88905cf0 0x555d8890aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.921+0000 7f89f37fe700 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d88905cf0 0x555d8890aa20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f5482866d39423a server_cookie=2666e9f5c378ec27 in_seq=0 out_seq=0 2024-05-10T04:39:15.928 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.921+0000 7f89d2ffd700 10 client.15315 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:15.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f89d2ffd700 1 -- 172.21.15.150:0/2949052537 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d888bf730 con 0x555d88905cf0 2024-05-10T04:39:15.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f89d2ffd700 10 client.15315 handle_command_reply: tid=0 2024-05-10T04:39:15.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 client.15315 shutdown 2024-05-10T04:39:15.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 msgr2=0x555d888fab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fab50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f89e4004020 tx=0x7f89e4004050 comp rx=0 tx=0).stop 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 shutdown_connections 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d88905cf0 0x555d8890aa20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d88901ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/2949052537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fab50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f89d2ffd700 0 client.15315 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 >> 172.21.15.150:0/2949052537 conn(0x555d88852a10 msgr2=0x555d888eaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 shutdown_connections 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/2949052537 wait complete. 2024-05-10T04:39:15.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.922+0000 7f8a0a1a6b80 20 client.15315 trim_cache size 0 max 16384 2024-05-10T04:39:15.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 Processor -- start 2024-05-10T04:39:15.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 -- start start 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f0650 0x555d888f4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f1e70 0x555d888fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8880d010 con 0x555d888f1520 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d887af5a0 con 0x555d888f1e70 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.940+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d888e5770 con 0x555d888f0650 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59612/0 (socket says 172.21.15.150:59612) 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 -- 172.21.15.150:0/559203337 learned_addr learned my addr 172.21.15.150:0/559203337 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f3fff700 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f1e70 0x555d888fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 -- 172.21.15.150:0/559203337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f0650 msgr2=0x555d888f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f0650 0x555d888f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 -- 172.21.15.150:0/559203337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f1e70 msgr2=0x555d888fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f1e70 0x555d888fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 -- 172.21.15.150:0/559203337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d887d6510 con 0x555d888f1520 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888ffa60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f89e4003ff0 tx=0x7f89e4003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a747a15a05d851 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89da7fc700 1 -- 172.21.15.150:0/559203337 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40057d0 con 0x555d888f1520 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89da7fc700 1 -- 172.21.15.150:0/559203337 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e4018c00 con 0x555d888f1520 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.941+0000 7f89da7fc700 1 -- 172.21.15.150:0/559203337 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4018400 con 0x555d888f1520 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/559203337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 msgr2=0x555d888ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888ffa60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f89e4003ff0 tx=0x7f89e4003ad0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/559203337 shutdown_connections 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f1e70 0x555d888fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f0650 0x555d888f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/559203337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888ffa60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/559203337 >> 172.21.15.150:0/559203337 conn(0x555d88852a10 msgr2=0x555d888e7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/559203337 shutdown_connections 2024-05-10T04:39:15.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/559203337 wait complete. 2024-05-10T04:39:15.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 -- start start 2024-05-10T04:39:15.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.942+0000 7f8a0a1a6b80 1 Processor -- start 2024-05-10T04:39:15.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f8a0a1a6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d888ff570 con 0x555d888f1520 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d888ff6e0 con 0x555d888f0650 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f8a0a1a6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d888ff8a0 con 0x555d888f1e70 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f8a0a1a6b80 10 client.0 _ll_register_callbacks cb 0x555d887b5ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59624/0 (socket says 172.21.15.150:59624) 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 -- 172.21.15.150:0/1736354022 learned_addr learned my addr 172.21.15.150:0/1736354022 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f37fe700 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f3fff700 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 -- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 msgr2=0x555d888ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888ff030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 -- 172.21.15.150:0/1736354022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 msgr2=0x555d888fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 -- 172.21.15.150:0/1736354022 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d8863db10 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89f2ffd700 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fe100 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f89e40039f0 tx=0x7f89e4018ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8bd03c2bfb410d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89d2ffd700 1 -- 172.21.15.150:0/1736354022 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4014070 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.943+0000 7f89d2ffd700 1 -- 172.21.15.150:0/1736354022 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f89e4000e40 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d888ecb70 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f89d2ffd700 1 -- 172.21.15.150:0/1736354022 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e4004310 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 10 client.15321 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d88695c80 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f89d2ffd700 1 -- 172.21.15.150:0/1736354022 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d888ecb70 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f89d2ffd700 1 -- 172.21.15.150:0/1736354022 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f89e400f600 con 0x555d888f1520 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 10 client.15321 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 20 client.15321 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:15.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d8895ecd0 0x555d8895ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 4 client.15321 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f89e40044f0 con 0x555d8895ecd0 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.944+0000 7f89f37fe700 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d8895ecd0 0x555d8895ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f89d2ffd700 10 client.15321 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f89f37fe700 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d8895ecd0 0x555d8895ccb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cb76244a44c160e server_cookie=973b20cef8e3cf81 in_seq=0 out_seq=0 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f89d2ffd700 1 -- 172.21.15.150:0/1736354022 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f89e40044f0 con 0x555d8895ecd0 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f89d2ffd700 10 client.15321 handle_command_reply: tid=0 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 client.15321 shutdown 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 msgr2=0x555d888fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fe100 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f89e40039f0 tx=0x7f89e4018ad0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 shutdown_connections 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x555d8895ecd0 0x555d8895ccb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d888f1e70 0x555d888ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d888f0650 0x555d888fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 --2- 172.21.15.150:0/1736354022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d888f1520 0x555d888fe100 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f89d2ffd700 0 client.15321 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 >> 172.21.15.150:0/1736354022 conn(0x555d88852a10 msgr2=0x555d888e7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 shutdown_connections 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 1 -- 172.21.15.150:0/1736354022 wait complete. 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.945+0000 7f8a0a1a6b80 20 client.15321 trim_cache size 0 max 16384 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 1, 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24520, 2024-05-10T04:39:15.954 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 43.546709737999997, 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 78.253618302999996 2024-05-10T04:39:15.955 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f89dc0a12e0 msgr2=0x7f89dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f89dc0a12e0 0x7f89dc0a37f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89e80098e0 tx=0x7f89e801a040 comp rx=0 tx=0).stop 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4102580 msgr2=0x7f89f419e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4102580 0x7f89f419e250 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007f10 tx=0x7f89ec00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 shutdown_connections 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89f4102580 0x7f89f419e250 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89f4100ac0 0x7f89f4196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f89dc0a12e0 0x7f89dc0a37f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 --2- 172.21.15.150:0/1108814629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89f4101be0 0x7f89f41972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 >> 172.21.15.150:0/1108814629 conn(0x7f89f40fc0d0 msgr2=0x7f89f4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 shutdown_connections 2024-05-10T04:39:15.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:15.946+0000 7f89f9e62700 1 -- 172.21.15.150:0/1108814629 wait complete. 2024-05-10T04:39:15.960 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T04:39:16.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 -- 172.21.15.150:0/3829878003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 msgr2=0x7fecb4109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 --2- 172.21.15.150:0/3829878003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb4109250 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fecac00b0a0 tx=0x7fecac00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 -- 172.21.15.150:0/3829878003 shutdown_connections 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 --2- 172.21.15.150:0/3829878003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb4109250 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 --2- 172.21.15.150:0/3829878003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb4101110 0x7fecb4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 --2- 172.21.15.150:0/3829878003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecb4102000 0x7fecb4102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 -- 172.21.15.150:0/3829878003 >> 172.21.15.150:0/3829878003 conn(0x7fecb40fb8f0 msgr2=0x7fecb40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 -- 172.21.15.150:0/3829878003 shutdown_connections 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 -- 172.21.15.150:0/3829878003 wait complete. 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 Processor -- start 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.093+0000 7fecbc271700 1 -- start start 2024-05-10T04:39:16.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecbc271700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb4101110 0x7fecb4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecbc271700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecb4102000 0x7fecb4197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecbc271700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecbc271700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419ebe0 con 0x7fecb4102950 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecbc271700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419ed50 con 0x7fecb4102000 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecbc271700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb419ef10 con 0x7fecb4101110 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59632/0 (socket says 172.21.15.150:59632) 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 -- 172.21.15.150:0/1009586254 learned_addr learned my addr 172.21.15.150:0/1009586254 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecb980c700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecb4102000 0x7fecb4197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba00d700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb4101110 0x7fecb4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 -- 172.21.15.150:0/1009586254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb4101110 msgr2=0x7fecb4197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb4101110 0x7fecb4197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 -- 172.21.15.150:0/1009586254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecb4102000 msgr2=0x7fecb4197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecb4102000 0x7fecb4197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 -- 172.21.15.150:0/1009586254 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac009d00 con 0x7fecb4102950 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba00d700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb4101110 0x7fecb4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecba80e700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb419e5d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fecac006070 tx=0x7fecac00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecab7fe700 1 -- 172.21.15.150:0/1009586254 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac00e040 con 0x7fecb4102950 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecab7fe700 1 -- 172.21.15.150:0/1009586254 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecac003c00 con 0x7fecb4102950 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.094+0000 7fecab7fe700 1 -- 172.21.15.150:0/1009586254 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac01b690 con 0x7fecb4102950 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.095+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb419f190 con 0x7fecb4102950 2024-05-10T04:39:16.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.095+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb419f5c0 con 0x7fecb4102950 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.097+0000 7fecab7fe700 1 -- 172.21.15.150:0/1009586254 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fecac01b7f0 con 0x7fecb4102950 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.097+0000 7fecab7fe700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feca00a12e0 0x7feca00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.097+0000 7fecba00d700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feca00a12e0 0x7feca00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.097+0000 7fecab7fe700 1 -- 172.21.15.150:0/1009586254 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fecac0ce0a0 con 0x7fecb4102950 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.098+0000 7fecba00d700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feca00a12e0 0x7feca00a37f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fecb0005950 tx=0x7fecb00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.098+0000 7feccc5b5b80 1 Processor -- start 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7feccc5b5b80 1 -- start start 2024-05-10T04:39:16.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14151860 0x55ad14151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14152b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14076100 con 0x55ad14152730 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14073ac0 con 0x55ad14153080 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1406f3b0 con 0x55ad14151860 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59636/0 (socket says 172.21.15.150:59636) 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 -- 172.21.15.150:0/3632091060 learned_addr learned my addr 172.21.15.150:0/3632091060 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecba80e700 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecba00d700 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14151860 0x55ad14151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 -- 172.21.15.150:0/3632091060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14151860 msgr2=0x55ad14151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14151860 0x55ad14151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 -- 172.21.15.150:0/3632091060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 msgr2=0x55ad14159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 -- 172.21.15.150:0/3632091060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad13ef0360 con 0x55ad14152730 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecba80e700 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.099+0000 7fecb980c700 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14152b40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feca4009d20 tx=0x7feca4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a1e42b6692000a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feca8d7f700 1 -- 172.21.15.150:0/3632091060 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4004a30 con 0x55ad14152730 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feca8d7f700 1 -- 172.21.15.150:0/3632091060 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca40054e0 con 0x55ad14152730 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feca8d7f700 1 -- 172.21.15.150:0/3632091060 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40197d0 con 0x55ad14152730 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3632091060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 msgr2=0x55ad14152b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14152b40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feca4009d20 tx=0x7feca4011720 comp rx=0 tx=0).stop 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3632091060 shutdown_connections 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14151860 0x55ad14151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3632091060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14152b40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.100+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3632091060 >> 172.21.15.150:0/3632091060 conn(0x55ad140b1a10 msgr2=0x55ad14149f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3632091060 shutdown_connections 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3632091060 wait complete. 2024-05-10T04:39:16.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 -- start start 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 Processor -- start 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad1415b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14153080 0x55ad14162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1400b510 con 0x55ad14153080 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1400b1b0 con 0x55ad14151860 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad13ef2650 con 0x55ad14152730 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7feccc5b5b80 10 client.0 _ll_register_callbacks cb 0x55ad14014ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7fecba80e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14153080 0x55ad14162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7fecba80e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14153080 0x55ad14162ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59646/0 (socket says 172.21.15.150:59646) 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.101+0000 7fecba80e700 1 -- 172.21.15.150:0/3151079319 learned_addr learned my addr 172.21.15.150:0/3151079319 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba00d700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad1415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecb980c700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba80e700 1 -- 172.21.15.150:0/3151079319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 msgr2=0x55ad1415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba80e700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba80e700 1 -- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 msgr2=0x55ad1415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba80e700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad1415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba80e700 1 -- 172.21.15.150:0/3151079319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb419f5c0 con 0x55ad14153080 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba00d700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad1415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fecba80e700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14153080 0x55ad14162ce0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fecac09b0a0 tx=0x7fecac0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1eb63d0e22421a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fec9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fec9dffb700 1 -- 172.21.15.150:0/3151079319 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0b2070 con 0x55ad14153080 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fec9dffb700 1 -- 172.21.15.150:0/3151079319 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fecac09ef40 con 0x55ad14153080 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad13fc8220 con 0x55ad14153080 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fec9dffb700 1 -- 172.21.15.150:0/3151079319 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecac0a3500 con 0x55ad14153080 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7feccc5b5b80 10 client.15333 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7feca4009980 con 0x55ad14153080 2024-05-10T04:39:16.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.102+0000 7fec9dffb700 1 -- 172.21.15.150:0/3151079319 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad13fc8220 con 0x55ad14153080 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7fec9dffb700 1 -- 172.21.15.150:0/3151079319 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecac09b400 con 0x55ad14153080 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7feccc5b5b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7feccc5b5b80 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7feccc5b5b80 20 client.15333 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad14166b30 0x55ad1416b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7feccc5b5b80 4 client.15333 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad1411e730 con 0x55ad14166b30 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.103+0000 7fecba00d700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad14166b30 0x55ad1416b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.104+0000 7fec9dffb700 10 client.15333 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.104+0000 7fecba00d700 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad14166b30 0x55ad1416b860 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cae3a327248f00 server_cookie=8f99ba87ace5a082 in_seq=0 out_seq=0 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.104+0000 7fec9dffb700 1 -- 172.21.15.150:0/3151079319 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad1411e730 con 0x55ad14166b30 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.104+0000 7fec9dffb700 10 client.15333 handle_command_reply: tid=0 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.104+0000 7feccc5b5b80 1 client.15333 shutdown 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.104+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14153080 msgr2=0x55ad14162ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.104+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14153080 0x55ad14162ce0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fecac09b0a0 tx=0x7fecac0099e0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 shutdown_connections 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad14166b30 0x55ad1416b860 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14153080 0x55ad14162ce0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad1415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/3151079319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7fec9dffb700 0 client.15333 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 >> 172.21.15.150:0/3151079319 conn(0x55ad140b1a10 msgr2=0x55ad14149f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 shutdown_connections 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 1 -- 172.21.15.150:0/3151079319 wait complete. 2024-05-10T04:39:16.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.105+0000 7feccc5b5b80 20 client.15333 trim_cache size 0 max 16384 2024-05-10T04:39:16.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 Processor -- start 2024-05-10T04:39:16.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 -- start start 2024-05-10T04:39:16.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad14155960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14153080 0x55ad141611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14068150 con 0x55ad14152730 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1406c010 con 0x55ad14151860 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1400e5a0 con 0x55ad14153080 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7fecb980c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7fecb980c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59662/0 (socket says 172.21.15.150:59662) 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7fecb980c700 1 -- 172.21.15.150:0/833463343 learned_addr learned my addr 172.21.15.150:0/833463343 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.123+0000 7fecba00d700 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad14155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7fecb980c700 1 -- 172.21.15.150:0/833463343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14153080 msgr2=0x55ad141611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7fecba80e700 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14153080 0x55ad141611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7fecb980c700 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14153080 0x55ad141611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7fecb980c700 1 -- 172.21.15.150:0/833463343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 msgr2=0x55ad14155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7fecb980c700 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad14155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7fecb980c700 1 -- 172.21.15.150:0/833463343 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad13e9cb10 con 0x55ad14152730 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7fecb980c700 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14160c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feca401b040 tx=0x7feca4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a24d11645c4d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:16.131 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7feca9580700 1 -- 172.21.15.150:0/833463343 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4005650 con 0x55ad14152730 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7feca9580700 1 -- 172.21.15.150:0/833463343 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca4003680 con 0x55ad14152730 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.124+0000 7feca9580700 1 -- 172.21.15.150:0/833463343 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4025690 con 0x55ad14152730 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 -- 172.21.15.150:0/833463343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 msgr2=0x55ad14160c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14160c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feca401b040 tx=0x7feca4005e30 comp rx=0 tx=0).stop 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 -- 172.21.15.150:0/833463343 shutdown_connections 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14153080 0x55ad141611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14151860 0x55ad14155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/833463343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14152730 0x55ad14160c90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 -- 172.21.15.150:0/833463343 >> 172.21.15.150:0/833463343 conn(0x55ad140b1a10 msgr2=0x55ad14146130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 -- 172.21.15.150:0/833463343 shutdown_connections 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 -- 172.21.15.150:0/833463343 wait complete. 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 -- start start 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.125+0000 7feccc5b5b80 1 Processor -- start 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14151860 0x55ad1415f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14160b50 con 0x55ad14151860 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14160cc0 con 0x55ad14153080 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14160e30 con 0x55ad14152730 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 10 client.0 _ll_register_callbacks cb 0x55ad14014ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14151860 0x55ad1415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14151860 0x55ad1415f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59678/0 (socket says 172.21.15.150:59678) 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 -- 172.21.15.150:0/2773759804 learned_addr learned my addr 172.21.15.150:0/2773759804 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba80e700 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecb980c700 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 -- 172.21.15.150:0/2773759804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 msgr2=0x55ad1415f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 -- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 msgr2=0x55ad14160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:16.133 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 -- 172.21.15.150:0/2773759804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad13ef0360 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fecba00d700 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14151860 0x55ad1415f380 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fecb000baa0 tx=0x7fecb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab49052060bdd9c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fec9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fec9dffb700 1 -- 172.21.15.150:0/2773759804 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0017040 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fec9dffb700 1 -- 172.21.15.150:0/2773759804 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fecb0004500 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feca40054e0 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7fec9dffb700 1 -- 172.21.15.150:0/2773759804 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecb00281d0 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 10 client.15339 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.126+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ad14035510 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7fec9dffb700 1 -- 172.21.15.150:0/2773759804 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca40054e0 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7fec9dffb700 1 -- 172.21.15.150:0/2773759804 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecb0003c80 con 0x55ad14151860 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7feccc5b5b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7feccc5b5b80 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:39:16.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7feccc5b5b80 20 client.15339 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad141bfec0 0x55ad141be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7feccc5b5b80 4 client.15339 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ad14067e70 con 0x55ad141bfec0 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.127+0000 7fecb980c700 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad141bfec0 0x55ad141be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7fec9dffb700 10 client.15339 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7fecb980c700 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad141bfec0 0x55ad141be400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73ca94ee22fdb84 server_cookie=fd868961a134b285 in_seq=0 out_seq=0 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7fec9dffb700 1 -- 172.21.15.150:0/2773759804 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ad14067e70 con 0x55ad141bfec0 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7fec9dffb700 10 client.15339 handle_command_reply: tid=0 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 client.15339 shutdown 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14151860 msgr2=0x55ad1415f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14151860 0x55ad1415f380 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fecb000baa0 tx=0x7fecb00047c0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 shutdown_connections 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ad141bfec0 0x55ad141be400 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ad14153080 0x55ad14160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ad14151860 0x55ad1415f380 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7fec9dffb700 0 client.15339 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 --2- 172.21.15.150:0/2773759804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ad14152730 0x55ad1415f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 >> 172.21.15.150:0/2773759804 conn(0x55ad140b1a10 msgr2=0x55ad14146130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 shutdown_connections 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 1 -- 172.21.15.150:0/2773759804 wait complete. 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.128+0000 7feccc5b5b80 20 client.15339 trim_cache size 0 max 16384 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24548, 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 67.376072770999997, 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 73.350698542999993 2024-05-10T04:39:16.137 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:16.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feca00a12e0 msgr2=0x7feca00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feca00a12e0 0x7feca00a37f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fecb0005950 tx=0x7fecb00058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 msgr2=0x7fecb419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb419e5d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fecac006070 tx=0x7fecac00ba50 comp rx=0 tx=0).stop 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 shutdown_connections 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecb4102950 0x7fecb419e5d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7feca00a12e0 0x7feca00a37f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb4101110 0x7fecb4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 --2- 172.21.15.150:0/1009586254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fecb4102000 0x7fecb4197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 >> 172.21.15.150:0/1009586254 conn(0x7fecb40fb8f0 msgr2=0x7fecb4105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 shutdown_connections 2024-05-10T04:39:16.139 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:16.129+0000 7fecbc271700 1 -- 172.21.15.150:0/1009586254 wait complete. 2024-05-10T04:39:16.143 INFO:tasks.cephadm:Setup complete, yielding 2024-05-10T04:39:16.143 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T04:39:16.153 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:39:16.153 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-10T04:39:16.373 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 -- 172.21.15.150:0/2343989775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc97006a590 msgr2=0x7fc970104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 --2- 172.21.15.150:0/2343989775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc97006a590 0x7fc970104840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc964009990 tx=0x7fc964009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 -- 172.21.15.150:0/2343989775 shutdown_connections 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 --2- 172.21.15.150:0/2343989775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc970107980 0x7fc970109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 --2- 172.21.15.150:0/2343989775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc97006a590 0x7fc970104840 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 --2- 172.21.15.150:0/2343989775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc970104e10 0x7fc9701072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 -- 172.21.15.150:0/2343989775 >> 172.21.15.150:0/2343989775 conn(0x7fc9700fc210 msgr2=0x7fc9700fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 -- 172.21.15.150:0/2343989775 shutdown_connections 2024-05-10T04:39:17.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.276+0000 7fc976586700 1 -- 172.21.15.150:0/2343989775 wait complete. 2024-05-10T04:39:17.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 Processor -- start 2024-05-10T04:39:17.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 -- start start 2024-05-10T04:39:17.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97006a590 0x7fc9701980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc970104e10 0x7fc970198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc970107980 0x7fc9701993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc970199990 con 0x7fc970104e10 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc970199ad0 con 0x7fc970107980 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc976586700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc970199c90 con 0x7fc97006a590 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc970104e10 0x7fc970198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc970104e10 0x7fc970198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53630/0 (socket says 172.21.15.150:53630) 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.277+0000 7fc96f7fe700 1 -- 172.21.15.150:0/1919683909 learned_addr learned my addr 172.21.15.150:0/1919683909 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc974b23700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc970107980 0x7fc9701993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96ffff700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97006a590 0x7fc9701980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96f7fe700 1 -- 172.21.15.150:0/1919683909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97006a590 msgr2=0x7fc9701980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96f7fe700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97006a590 0x7fc9701980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96f7fe700 1 -- 172.21.15.150:0/1919683909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc970107980 msgr2=0x7fc9701993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96f7fe700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc970107980 0x7fc9701993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96f7fe700 1 -- 172.21.15.150:0/1919683909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640095f0 con 0x7fc970104e10 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96ffff700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97006a590 0x7fc9701980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96f7fe700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc970104e10 0x7fc970198610 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc96000d950 tx=0x7fc96000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96d7fa700 1 -- 172.21.15.150:0/1919683909 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960009940 con 0x7fc970104e10 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96d7fa700 1 -- 172.21.15.150:0/1919683909 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc960010460 con 0x7fc970104e10 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc970199f70 con 0x7fc970104e10 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc96d7fa700 1 -- 172.21.15.150:0/1919683909 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000f620 con 0x7fc970104e10 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.278+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9701a07e0 con 0x7fc970104e10 2024-05-10T04:39:17.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.279+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9700fddc0 con 0x7fc970104e10 2024-05-10T04:39:17.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.280+0000 7fc96d7fa700 1 -- 172.21.15.150:0/1919683909 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fc9600108d0 con 0x7fc970104e10 2024-05-10T04:39:17.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.281+0000 7fc96d7fa700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc9580a11b0 0x7fc9580a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:17.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.281+0000 7fc96ffff700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc9580a11b0 0x7fc9580a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:17.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.281+0000 7fc96d7fa700 1 -- 172.21.15.150:0/1919683909 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9600ccc20 con 0x7fc970104e10 2024-05-10T04:39:17.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.281+0000 7fc96ffff700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc9580a11b0 0x7fc9580a36c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc9640037f0 tx=0x7fc96401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:17.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.284+0000 7fc96d7fa700 1 -- 172.21.15.150:0/1919683909 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc960095f40 con 0x7fc970104e10 2024-05-10T04:39:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.494+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc97010d600 con 0x7fc9580a11b0 2024-05-10T04:39:17.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.495+0000 7fc96d7fa700 1 -- 172.21.15.150:0/1919683909 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc97010d600 con 0x7fc9580a11b0 2024-05-10T04:39:17.497 INFO:teuthology.orchestra.run.smithi150.stdout:Backend: cephadm 2024-05-10T04:39:17.497 INFO:teuthology.orchestra.run.smithi150.stdout:Available: Yes 2024-05-10T04:39:17.497 INFO:teuthology.orchestra.run.smithi150.stdout:Paused: No 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.497+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc9580a11b0 msgr2=0x7fc9580a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.497+0000 7fc976586700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc9580a11b0 0x7fc9580a36c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc9640037f0 tx=0x7fc96401a040 comp rx=0 tx=0).stop 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.497+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc970104e10 msgr2=0x7fc970198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.497+0000 7fc976586700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc970104e10 0x7fc970198610 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc96000d950 tx=0x7fc96000dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 shutdown_connections 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc970107980 0x7fc9701993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc9580a11b0 0x7fc9580a36c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc97006a590 0x7fc9701980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 --2- 172.21.15.150:0/1919683909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc970104e10 0x7fc970198610 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:17.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 >> 172.21.15.150:0/1919683909 conn(0x7fc9700fc210 msgr2=0x7fc970102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:17.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 shutdown_connections 2024-05-10T04:39:17.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:17.498+0000 7fc976586700 1 -- 172.21.15.150:0/1919683909 wait complete. 2024-05-10T04:39:17.854 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-10T04:39:18.079 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:18.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:18 smithi181 ceph-mon[47314]: pgmap v261: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3 op/s; 203 B/s, 1 objects/s recovering 2024-05-10T04:39:18.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:18 smithi150 ceph-mon[42404]: pgmap v261: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3 op/s; 203 B/s, 1 objects/s recovering 2024-05-10T04:39:18.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:18 smithi161 ceph-mon[46912]: pgmap v261: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3 op/s; 203 B/s, 1 objects/s recovering 2024-05-10T04:39:19.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.218+0000 7ff100b62700 1 -- 172.21.15.150:0/1444376091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc103770 msgr2=0x7ff0fc10a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:19.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.218+0000 7ff100b62700 1 --2- 172.21.15.150:0/1444376091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc103770 0x7ff0fc10a020 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00b0a0 tx=0x7ff0ec00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.218+0000 7ff100b62700 1 -- 172.21.15.150:0/1444376091 shutdown_connections 2024-05-10T04:39:19.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.218+0000 7ff100b62700 1 --2- 172.21.15.150:0/1444376091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc103770 0x7ff0fc10a020 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.218+0000 7ff100b62700 1 --2- 172.21.15.150:0/1444376091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0fc101ca0 0x7ff0fc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.218+0000 7ff100b62700 1 --2- 172.21.15.150:0/1444376091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0fc102dd0 0x7ff0fc1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.218+0000 7ff100b62700 1 -- 172.21.15.150:0/1444376091 >> 172.21.15.150:0/1444376091 conn(0x7ff0fc0fd300 msgr2=0x7ff0fc0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:19.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.219+0000 7ff100b62700 1 -- 172.21.15.150:0/1444376091 shutdown_connections 2024-05-10T04:39:19.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.219+0000 7ff100b62700 1 -- 172.21.15.150:0/1444376091 wait complete. 2024-05-10T04:39:19.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.219+0000 7ff100b62700 1 Processor -- start 2024-05-10T04:39:19.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.219+0000 7ff100b62700 1 -- start start 2024-05-10T04:39:19.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff100b62700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc101ca0 0x7ff0fc075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:19.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff100b62700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0fc102dd0 0x7ff0fc073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:19.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff100b62700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0fc103770 0x7ff0fc073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:19.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff100b62700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc0742f0 con 0x7ff0fc101ca0 2024-05-10T04:39:19.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff100b62700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc1a21f0 con 0x7ff0fc102dd0 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff100b62700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc1a2330 con 0x7ff0fc103770 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc101ca0 0x7ff0fc075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc101ca0 0x7ff0fc075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53640/0 (socket says 172.21.15.150:53640) 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 -- 172.21.15.150:0/165840931 learned_addr learned my addr 172.21.15.150:0/165840931 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0f9d9b700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0fc102dd0 0x7ff0fc073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fad9d700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0fc103770 0x7ff0fc073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 -- 172.21.15.150:0/165840931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0fc103770 msgr2=0x7ff0fc073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0fc103770 0x7ff0fc073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 -- 172.21.15.150:0/165840931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0fc102dd0 msgr2=0x7ff0fc073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0fc102dd0 0x7ff0fc073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 -- 172.21.15.150:0/165840931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec009d00 con 0x7ff0fc101ca0 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0f9d9b700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0fc102dd0 0x7ff0fc073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:19.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0fa59c700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc101ca0 0x7ff0fc075090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400d980 tx=0x7ff0f400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:19.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0f37fe700 1 -- 172.21.15.150:0/165840931 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4004220 con 0x7ff0fc101ca0 2024-05-10T04:39:19.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0f37fe700 1 -- 172.21.15.150:0/165840931 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f400de60 con 0x7ff0fc101ca0 2024-05-10T04:39:19.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc1a2590 con 0x7ff0fc101ca0 2024-05-10T04:39:19.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.220+0000 7ff0f37fe700 1 -- 172.21.15.150:0/165840931 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003ce0 con 0x7ff0fc101ca0 2024-05-10T04:39:19.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.221+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc1a2a90 con 0x7ff0fc101ca0 2024-05-10T04:39:19.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.221+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc108ec0 con 0x7ff0fc101ca0 2024-05-10T04:39:19.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.223+0000 7ff0f37fe700 1 -- 172.21.15.150:0/165840931 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff0f4004380 con 0x7ff0fc101ca0 2024-05-10T04:39:19.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.224+0000 7ff0f37fe700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0e80a11b0 0x7ff0e80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:19.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.224+0000 7ff0f9d9b700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0e80a11b0 0x7ff0e80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:19.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.224+0000 7ff0f37fe700 1 -- 172.21.15.150:0/165840931 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff0f40cd0f0 con 0x7ff0fc101ca0 2024-05-10T04:39:19.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.225+0000 7ff0f9d9b700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0e80a11b0 0x7ff0e80a36c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005950 tx=0x7ff0e4005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:19.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.228+0000 7ff0f37fe700 1 -- 172.21.15.150:0/165840931 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff0f4096410 con 0x7ff0fc101ca0 2024-05-10T04:39:19.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.434+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0fc10d6d0 con 0x7ff0e80a11b0 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.447+0000 7ff0f37fe700 1 -- 172.21.15.150:0/165840931 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7ff0fc10d6d0 con 0x7ff0e80a11b0 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:mds.a smithi150 running (78s) 11s ago 78s 84.0M - 18.2.2-1330-g50174015 0e8066d08b37 00c25cdc0638 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:mds.b smithi161 running (76s) 12s ago 76s 97.5M - 18.2.2-1330-g50174015 0e8066d08b37 acf4ac310ae4 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:mds.c smithi181 running (85s) 12s ago 85s 86.9M - 18.2.2-1330-g50174015 0e8066d08b37 998d55b8922f 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:mds.d smithi150 running (91s) 11s ago 90s 84.9M - 18.2.2-1330-g50174015 0e8066d08b37 d71287ddeb46 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:mds.e smithi161 running (89s) 12s ago 89s 86.2M - 18.2.2-1330-g50174015 0e8066d08b37 89037170548b 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:mds.f smithi181 running (87s) 12s ago 87s 86.2M - 18.2.2-1330-g50174015 0e8066d08b37 59e9a070ffb5 2024-05-10T04:39:19.449 INFO:teuthology.orchestra.run.smithi150.stdout:mds.g smithi150 running (80s) 11s ago 80s 84.7M - 18.2.2-1330-g50174015 0e8066d08b37 1210f90d1396 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mds.h smithi161 running (84s) 12s ago 83s 86.1M - 18.2.2-1330-g50174015 0e8066d08b37 9da4c01be77d 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mds.i smithi181 running (82s) 12s ago 81s 87.9M - 18.2.2-1330-g50174015 0e8066d08b37 eb02523e4b74 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mds.j smithi150 running (92s) 11s ago 92s 85.3M - 18.2.2-1330-g50174015 0e8066d08b37 397e3b7dc3e7 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mds.k smithi161 running (94s) 12s ago 94s 85.5M - 18.2.2-1330-g50174015 0e8066d08b37 6d66cdf37c45 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mds.l smithi181 running (96s) 12s ago 95s 86.3M - 18.2.2-1330-g50174015 0e8066d08b37 dfe76ae50d6b 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mgr.x smithi150 *:9283,8765 running (6m) 11s ago 6m 601M - 18.2.2-1330-g50174015 0e8066d08b37 506365544db8 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mgr.y smithi161 *:8443,8765 running (5m) 12s ago 5m 528M - 18.2.2-1330-g50174015 0e8066d08b37 ba3a10cc8f2b 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mgr.z smithi181 *:8443,8765 running (5m) 12s ago 5m 526M - 18.2.2-1330-g50174015 0e8066d08b37 4acab310f4d2 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mon.a smithi150 running (6m) 11s ago 6m 159M 2048M 18.2.2-1330-g50174015 0e8066d08b37 aedb3acd44a1 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mon.b smithi161 running (5m) 12s ago 5m 128M 2048M 18.2.2-1330-g50174015 0e8066d08b37 97beb74ed8d5 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:mon.c smithi181 running (5m) 12s ago 5m 128M 2048M 18.2.2-1330-g50174015 0e8066d08b37 24c568b60ba4 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0 smithi150 running (5m) 11s ago 5m 138M 4278M 18.2.2-1330-g50174015 0e8066d08b37 80e23612bbe2 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1 smithi161 running (4m) 12s ago 4m 213M 4278M 18.2.2-1330-g50174015 0e8066d08b37 3db4af595d60 2024-05-10T04:39:19.450 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2 smithi181 running (4m) 12s ago 4m 140M 4278M 18.2.2-1330-g50174015 0e8066d08b37 add41bf0b739 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3 smithi150 running (4m) 11s ago 4m 133M 4278M 18.2.2-1330-g50174015 0e8066d08b37 6af5f8d5f63d 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4 smithi161 running (3m) 12s ago 3m 128M 4278M 18.2.2-1330-g50174015 0e8066d08b37 d79ccc237b0d 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5 smithi181 running (3m) 12s ago 3m 203M 4278M 18.2.2-1330-g50174015 0e8066d08b37 5a7e27f90a95 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6 smithi150 running (3m) 11s ago 3m 127M 4278M 18.2.2-1330-g50174015 0e8066d08b37 19d0a7c7309e 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7 smithi161 running (3m) 12s ago 3m 202M 4278M 18.2.2-1330-g50174015 0e8066d08b37 8861be0f8a0f 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.8 smithi181 running (2m) 12s ago 2m 197M 4278M 18.2.2-1330-g50174015 0e8066d08b37 18978e5c776e 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.9 smithi150 running (2m) 11s ago 2m 130M 4278M 18.2.2-1330-g50174015 0e8066d08b37 7af54270bf5b 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.10 smithi161 running (2m) 12s ago 2m 128M 4278M 18.2.2-1330-g50174015 0e8066d08b37 ac0f9b2e21da 2024-05-10T04:39:19.451 INFO:teuthology.orchestra.run.smithi150.stdout:osd.11 smithi181 running (110s) 12s ago 110s 190M 4278M 18.2.2-1330-g50174015 0e8066d08b37 d30a41f2e610 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0e80a11b0 msgr2=0x7ff0e80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0e80a11b0 0x7ff0e80a36c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005950 tx=0x7ff0e4005f50 comp rx=0 tx=0).stop 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc101ca0 msgr2=0x7ff0fc075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc101ca0 0x7ff0fc075090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400d980 tx=0x7ff0f400dd40 comp rx=0 tx=0).stop 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 shutdown_connections 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff0e80a11b0 0x7ff0e80a36c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0fc103770 0x7ff0fc073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0fc101ca0 0x7ff0fc075090 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 --2- 172.21.15.150:0/165840931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0fc102dd0 0x7ff0fc073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 >> 172.21.15.150:0/165840931 conn(0x7ff0fc0fd300 msgr2=0x7ff0fc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 shutdown_connections 2024-05-10T04:39:19.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:19.450+0000 7ff100b62700 1 -- 172.21.15.150:0/165840931 wait complete. 2024-05-10T04:39:19.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:19 smithi181 ceph-mon[47314]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:19.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:19 smithi150 ceph-mon[42404]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:19.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-10T04:39:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:19 smithi161 ceph-mon[46912]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:20.035 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:20.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:20 smithi181 ceph-mon[47314]: pgmap v262: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5 op/s; 203 B/s, 1 objects/s recovering 2024-05-10T04:39:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:20 smithi181 ceph-mon[47314]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-10T04:39:20.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:20 smithi181 ceph-mon[47314]: Cluster is now healthy 2024-05-10T04:39:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:20 smithi150 ceph-mon[42404]: pgmap v262: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5 op/s; 203 B/s, 1 objects/s recovering 2024-05-10T04:39:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:20 smithi150 ceph-mon[42404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-10T04:39:20.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:20 smithi150 ceph-mon[42404]: Cluster is now healthy 2024-05-10T04:39:20.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:20 smithi161 ceph-mon[46912]: pgmap v262: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5 op/s; 203 B/s, 1 objects/s recovering 2024-05-10T04:39:20.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:20 smithi161 ceph-mon[46912]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-05-10T04:39:20.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:20 smithi161 ceph-mon[46912]: Cluster is now healthy 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 -- 172.21.15.150:0/2999034233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 msgr2=0x7f49a01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 --2- 172.21.15.150:0/2999034233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a01020c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4988009990 tx=0x7f4988009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 -- 172.21.15.150:0/2999034233 shutdown_connections 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 --2- 172.21.15.150:0/2999034233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a0102650 0x7f49a0108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 --2- 172.21.15.150:0/2999034233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0100b80 0x7f49a0100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 --2- 172.21.15.150:0/2999034233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a01020c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 -- 172.21.15.150:0/2999034233 >> 172.21.15.150:0/2999034233 conn(0x7f49a00fc1c0 msgr2=0x7f49a00fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 -- 172.21.15.150:0/2999034233 shutdown_connections 2024-05-10T04:39:20.953 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.951+0000 7f49a58d7700 1 -- 172.21.15.150:0/2999034233 wait complete. 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 Processor -- start 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 -- start start 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0100b80 0x7f49a019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a0102650 0x7f49a019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a019fde0 con 0x7f49a0101cb0 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a019ff20 con 0x7f49a0102650 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.952+0000 7f49a58d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a01a0060 con 0x7f49a0100b80 2024-05-10T04:39:20.955 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0100b80 0x7f49a019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499effd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0100b80 0x7f49a019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:40528/0 (socket says 172.21.15.150:40528) 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53658/0 (socket says 172.21.15.150:53658) 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499effd700 1 -- 172.21.15.150:0/1215088168 learned_addr learned my addr 172.21.15.150:0/1215088168 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499f7fe700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a0102650 0x7f49a019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 -- 172.21.15.150:0/1215088168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0100b80 msgr2=0x7f49a019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0100b80 0x7f49a019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 -- 172.21.15.150:0/1215088168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a0102650 msgr2=0x7f49a019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a0102650 0x7f49a019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 -- 172.21.15.150:0/1215088168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880095f0 con 0x7f49a0101cb0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499f7fe700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a0102650 0x7f49a019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f499e7fc700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a019d350 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f498800b5c0 tx=0x7f498800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f49a48d5700 1 -- 172.21.15.150:0/1215088168 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498801d070 con 0x7f49a0101cb0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a01a02e0 con 0x7f49a0101cb0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f49a48d5700 1 -- 172.21.15.150:0/1215088168 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49880054f0 con 0x7f49a0101cb0 2024-05-10T04:39:20.956 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f49a48d5700 1 -- 172.21.15.150:0/1215088168 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988004a30 con 0x7f49a0101cb0 2024-05-10T04:39:20.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.953+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a01a07a0 con 0x7f49a0101cb0 2024-05-10T04:39:20.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.954+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a0107e10 con 0x7f49a0101cb0 2024-05-10T04:39:20.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.956+0000 7f49a48d5700 1 -- 172.21.15.150:0/1215088168 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4988004b90 con 0x7f49a0101cb0 2024-05-10T04:39:20.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.956+0000 7f49a48d5700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f498c0a11b0 0x7f498c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:20.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.956+0000 7f49a48d5700 1 -- 172.21.15.150:0/1215088168 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f49880ce000 con 0x7f49a0101cb0 2024-05-10T04:39:20.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.956+0000 7f499effd700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f498c0a11b0 0x7f498c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:20.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.957+0000 7f499effd700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f498c0a11b0 0x7f498c0a36c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f49a0069700 tx=0x7f499000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:20.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:20.960+0000 7f49a48d5700 1 -- 172.21.15.150:0/1215088168 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f49880973b0 con 0x7f49a0101cb0 2024-05-10T04:39:21.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.163+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f49a010c510 con 0x7f498c0a11b0 2024-05-10T04:39:21.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.168+0000 7f49a48d5700 1 -- 172.21.15.150:0/1215088168 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f49a010c510 con 0x7f498c0a11b0 2024-05-10T04:39:21.169 INFO:teuthology.orchestra.run.smithi150.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-10T04:39:21.170 INFO:teuthology.orchestra.run.smithi150.stdout:mds.all 12/12 14s ago 99s smithi150=a;smithi150=d;smithi150=g;smithi150=j;smithi161=b;smithi161=e;smithi161=h;smithi161=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2024-05-10T04:39:21.170 INFO:teuthology.orchestra.run.smithi150.stdout:mgr 3/3 14s ago 5m smithi150=x;smithi161=y;smithi181=z;count:3 2024-05-10T04:39:21.170 INFO:teuthology.orchestra.run.smithi150.stdout:mon 3/3 14s ago 5m smithi150:172.21.15.150=a;smithi161:172.21.15.161=b;smithi181:172.21.15.181=c;count:3 2024-05-10T04:39:21.170 INFO:teuthology.orchestra.run.smithi150.stdout:osd 12 14s ago - 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.170+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f498c0a11b0 msgr2=0x7f498c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.170+0000 7f49a58d7700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f498c0a11b0 0x7f498c0a36c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f49a0069700 tx=0x7f499000b480 comp rx=0 tx=0).stop 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.170+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 msgr2=0x7f49a019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.170+0000 7f49a58d7700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a019d350 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f498800b5c0 tx=0x7f498800f710 comp rx=0 tx=0).stop 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 shutdown_connections 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49a0102650 0x7f49a019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f498c0a11b0 0x7f498c0a36c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a0100b80 0x7f49a019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 --2- 172.21.15.150:0/1215088168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49a0101cb0 0x7f49a019d350 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:21.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 >> 172.21.15.150:0/1215088168 conn(0x7f49a00fc1c0 msgr2=0x7f49a0104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:21.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 shutdown_connections 2024-05-10T04:39:21.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:21.171+0000 7f49a58d7700 1 -- 172.21.15.150:0/1215088168 wait complete. 2024-05-10T04:39:21.460 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:21 smithi150 ceph-mon[42404]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:21.519 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-10T04:39:21.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:21 smithi181 ceph-mon[47314]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:21.742 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:21.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:21 smithi161 ceph-mon[46912]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:22.498 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:22 smithi150 ceph-mon[42404]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:22.499 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:22 smithi150 ceph-mon[42404]: pgmap v263: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5 op/s; 164 B/s, 1 objects/s recovering 2024-05-10T04:39:22.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:22 smithi181 ceph-mon[47314]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:22.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:22 smithi181 ceph-mon[47314]: pgmap v263: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5 op/s; 164 B/s, 1 objects/s recovering 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 -- 172.21.15.150:0/1734402629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 msgr2=0x7f72cc1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1734402629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc1020c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009990 tx=0x7f72c0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 -- 172.21.15.150:0/1734402629 shutdown_connections 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1734402629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72cc102650 0x7f72cc108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1734402629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72cc100b80 0x7f72cc100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1734402629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc1020c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 -- 172.21.15.150:0/1734402629 >> 172.21.15.150:0/1734402629 conn(0x7f72cc0fc1c0 msgr2=0x7f72cc0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 -- 172.21.15.150:0/1734402629 shutdown_connections 2024-05-10T04:39:22.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.693+0000 7f72d37fd700 1 -- 172.21.15.150:0/1734402629 wait complete. 2024-05-10T04:39:22.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.694+0000 7f72d37fd700 1 Processor -- start 2024-05-10T04:39:22.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.694+0000 7f72d37fd700 1 -- start start 2024-05-10T04:39:22.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d37fd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72cc100b80 0x7f72cc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:22.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d37fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:22.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d37fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72cc102650 0x7f72cc19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d37fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19bc40 con 0x7f72cc101cb0 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d37fd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19bd80 con 0x7f72cc102650 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d37fd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19bf40 con 0x7f72cc100b80 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d1d9a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72cc102650 0x7f72cc19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53682/0 (socket says 172.21.15.150:53682) 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 -- 172.21.15.150:0/1552594544 learned_addr learned my addr 172.21.15.150:0/1552594544 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d1599700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72cc100b80 0x7f72cc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 -- 172.21.15.150:0/1552594544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72cc100b80 msgr2=0x7f72cc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72cc100b80 0x7f72cc19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 -- 172.21.15.150:0/1552594544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72cc102650 msgr2=0x7f72cc19b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72cc102650 0x7f72cc19b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 -- 172.21.15.150:0/1552594544 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00095f0 con 0x7f72cc101cb0 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d1d9a700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72cc102650 0x7f72cc19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.695+0000 7f72d0d98700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc19a830 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098e0 tx=0x7f72c0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.696+0000 7f72ca7fc700 1 -- 172.21.15.150:0/1552594544 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c001d070 con 0x7f72cc101cb0 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.696+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc19c1c0 con 0x7f72cc101cb0 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.696+0000 7f72ca7fc700 1 -- 172.21.15.150:0/1552594544 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c0021410 con 0x7f72cc101cb0 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.696+0000 7f72ca7fc700 1 -- 172.21.15.150:0/1552594544 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0017650 con 0x7f72cc101cb0 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.696+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1a2a00 con 0x7f72cc101cb0 2024-05-10T04:39:22.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.696+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc107da0 con 0x7f72cc101cb0 2024-05-10T04:39:22.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.698+0000 7f72ca7fc700 1 -- 172.21.15.150:0/1552594544 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f72c0021580 con 0x7f72cc101cb0 2024-05-10T04:39:22.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.699+0000 7f72ca7fc700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72b40a1190 0x7f72b40a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:22.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.699+0000 7f72ca7fc700 1 -- 172.21.15.150:0/1552594544 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f72c00ce7f0 con 0x7f72cc101cb0 2024-05-10T04:39:22.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.699+0000 7f72d1599700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72b40a1190 0x7f72b40a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:22.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.700+0000 7f72d1599700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72b40a1190 0x7f72b40a36a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f72cc101bb0 tx=0x7f72c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:22.704 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.703+0000 7f72ca7fc700 1 -- 172.21.15.150:0/1552594544 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f72c0097ba0 con 0x7f72cc101cb0 2024-05-10T04:39:22.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:22 smithi161 ceph-mon[46912]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:22.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:22 smithi161 ceph-mon[46912]: pgmap v263: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5 op/s; 164 B/s, 1 objects/s recovering 2024-05-10T04:39:22.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.899+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f72cc10b430 con 0x7f72b40a1190 2024-05-10T04:39:22.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.900+0000 7f72ca7fc700 1 -- 172.21.15.150:0/1552594544 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f72cc10b430 con 0x7f72b40a1190 2024-05-10T04:39:22.902 INFO:teuthology.orchestra.run.smithi150.stdout:HOST ADDR LABELS STATUS 2024-05-10T04:39:22.902 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150 172.21.15.150 2024-05-10T04:39:22.902 INFO:teuthology.orchestra.run.smithi150.stdout:smithi161 172.21.15.161 2024-05-10T04:39:22.902 INFO:teuthology.orchestra.run.smithi150.stdout:smithi181 172.21.15.181 2024-05-10T04:39:22.902 INFO:teuthology.orchestra.run.smithi150.stdout:3 hosts in cluster 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72b40a1190 msgr2=0x7f72b40a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72b40a1190 0x7f72b40a36a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f72cc101bb0 tx=0x7f72c400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 msgr2=0x7f72cc19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc19a830 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098e0 tx=0x7f72c0009580 comp rx=0 tx=0).stop 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 shutdown_connections 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72cc102650 0x7f72cc19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72b40a1190 0x7f72b40a36a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.902+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72cc100b80 0x7f72cc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.903+0000 7f72d37fd700 1 --2- 172.21.15.150:0/1552594544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72cc101cb0 0x7f72cc19a830 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.903+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 >> 172.21.15.150:0/1552594544 conn(0x7f72cc0fc1c0 msgr2=0x7f72cc104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:22.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.903+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 shutdown_connections 2024-05-10T04:39:22.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:22.903+0000 7f72d37fd700 1 -- 172.21.15.150:0/1552594544 wait complete. 2024-05-10T04:39:23.302 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-10T04:39:23.526 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:24.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 -- 172.21.15.150:0/771573760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158100610 msgr2=0x7fa15810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:24.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 --2- 172.21.15.150:0/771573760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158100610 0x7fa15810a230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b0a0 tx=0x7fa15000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 -- 172.21.15.150:0/771573760 shutdown_connections 2024-05-10T04:39:24.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 --2- 172.21.15.150:0/771573760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158100610 0x7fa15810a230 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 --2- 172.21.15.150:0/771573760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa158103270 0x7fa158103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 --2- 172.21.15.150:0/771573760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1580ffcc0 0x7fa1581000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 -- 172.21.15.150:0/771573760 >> 172.21.15.150:0/771573760 conn(0x7fa1580fb990 msgr2=0x7fa1580fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 -- 172.21.15.150:0/771573760 shutdown_connections 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.495+0000 7fa15d48f700 1 -- 172.21.15.150:0/771573760 wait complete. 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 Processor -- start 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 -- start start 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1580ffcc0 0x7fa1581981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa158100610 0x7fa158198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158103270 0x7fa1581994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158199ab0 con 0x7fa158103270 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158199bf0 con 0x7fa158100610 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.496+0000 7fa15d48f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158199db0 con 0x7fa1580ffcc0 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158103270 0x7fa1581994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158103270 0x7fa1581994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53696/0 (socket says 172.21.15.150:53696) 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 -- 172.21.15.150:0/1093897250 learned_addr learned my addr 172.21.15.150:0/1093897250 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:24.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1567fc700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa158100610 0x7fa158198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa156ffd700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1580ffcc0 0x7fa1581981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 -- 172.21.15.150:0/1093897250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1580ffcc0 msgr2=0x7fa1581981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1580ffcc0 0x7fa1581981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 -- 172.21.15.150:0/1093897250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa158100610 msgr2=0x7fa158198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa158100610 0x7fa158198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 -- 172.21.15.150:0/1093897250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150009d00 con 0x7fa158103270 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa156ffd700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1580ffcc0 0x7fa1581981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa1577fe700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158103270 0x7fa1581994e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa150006070 tx=0x7fa150004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa13ffff700 1 -- 172.21.15.150:0/1093897250 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15000e040 con 0x7fa158103270 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15819a030 con 0x7fa158103270 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa13ffff700 1 -- 172.21.15.150:0/1093897250 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15000cec0 con 0x7fa158103270 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1581a0900 con 0x7fa158103270 2024-05-10T04:39:24.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.497+0000 7fa13ffff700 1 -- 172.21.15.150:0/1093897250 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150004020 con 0x7fa158103270 2024-05-10T04:39:24.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.498+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1581090d0 con 0x7fa158103270 2024-05-10T04:39:24.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.499+0000 7fa13ffff700 1 -- 172.21.15.150:0/1093897250 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa15001d690 con 0x7fa158103270 2024-05-10T04:39:24.502 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.500+0000 7fa13ffff700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1400a11b0 0x7fa1400a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:24.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.500+0000 7fa13ffff700 1 -- 172.21.15.150:0/1093897250 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1500ceda0 con 0x7fa158103270 2024-05-10T04:39:24.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.500+0000 7fa156ffd700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1400a11b0 0x7fa1400a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:24.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.501+0000 7fa156ffd700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1400a11b0 0x7fa1400a36c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa14c005fd0 tx=0x7fa14c005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:24.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.504+0000 7fa13ffff700 1 -- 172.21.15.150:0/1093897250 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa150098150 con 0x7fa158103270 2024-05-10T04:39:24.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:24 smithi181 ceph-mon[47314]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:24.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:24 smithi181 ceph-mon[47314]: pgmap v264: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5 op/s; 149 B/s, 1 objects/s recovering 2024-05-10T04:39:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:24 smithi150 ceph-mon[42404]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:24.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:24 smithi150 ceph-mon[42404]: pgmap v264: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5 op/s; 149 B/s, 1 objects/s recovering 2024-05-10T04:39:24.706 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.705+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa15810d570 con 0x7fa1400a11b0 2024-05-10T04:39:24.708 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.707+0000 7fa13ffff700 1 -- 172.21.15.150:0/1093897250 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fa15810d570 con 0x7fa1400a11b0 2024-05-10T04:39:24.708 INFO:teuthology.orchestra.run.smithi150.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-10T04:39:24.708 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-10T04:39:24.708 INFO:teuthology.orchestra.run.smithi150.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-10T04:39:24.708 INFO:teuthology.orchestra.run.smithi150.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 110s ago Has a FileSystem, LVM detected 2024-05-10T04:39:24.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1400a11b0 msgr2=0x7fa1400a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:24.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1400a11b0 0x7fa1400a36c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa14c005fd0 tx=0x7fa14c005e40 comp rx=0 tx=0).stop 2024-05-10T04:39:24.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158103270 msgr2=0x7fa1581994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:24.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158103270 0x7fa1581994e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa150006070 tx=0x7fa150004d30 comp rx=0 tx=0).stop 2024-05-10T04:39:24.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 shutdown_connections 2024-05-10T04:39:24.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa158103270 0x7fa1581994e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa1400a11b0 0x7fa1400a36c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1580ffcc0 0x7fa1581981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 --2- 172.21.15.150:0/1093897250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa158100610 0x7fa158198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:24.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 >> 172.21.15.150:0/1093897250 conn(0x7fa1580fb990 msgr2=0x7fa158106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:24.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 shutdown_connections 2024-05-10T04:39:24.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:24.709+0000 7fa15d48f700 1 -- 172.21.15.150:0/1093897250 wait complete. 2024-05-10T04:39:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:24 smithi161 ceph-mon[46912]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:24 smithi161 ceph-mon[46912]: pgmap v264: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5 op/s; 149 B/s, 1 objects/s recovering 2024-05-10T04:39:25.086 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-10T04:39:25.093 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:39:25.094 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-10T04:39:25.319 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 -- 172.21.15.150:0/2252184209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 msgr2=0x7f6be8107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 --2- 172.21.15.150:0/2252184209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8107500 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009990 tx=0x7f6bd8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 -- 172.21.15.150:0/2252184209 shutdown_connections 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 --2- 172.21.15.150:0/2252184209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6be8107a40 0x7f6be8109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 --2- 172.21.15.150:0/2252184209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be8102780 0x7f6be8104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 --2- 172.21.15.150:0/2252184209 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8107500 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 -- 172.21.15.150:0/2252184209 >> 172.21.15.150:0/2252184209 conn(0x7f6be80fc1c0 msgr2=0x7f6be80fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:26.321 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 -- 172.21.15.150:0/2252184209 shutdown_connections 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.317+0000 7f6bf037d700 1 -- 172.21.15.150:0/2252184209 wait complete. 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 Processor -- start 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 -- start start 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6be8102780 0x7f6be8198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be8107a40 0x7f6be8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be8199a70 con 0x7f6be81050e0 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be8199bb0 con 0x7f6be8107a40 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bf037d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be8199d70 con 0x7f6be8102780 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:53712/0 (socket says 172.21.15.150:53712) 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 -- 172.21.15.150:0/3679713715 learned_addr learned my addr 172.21.15.150:0/3679713715 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bee119700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6be8102780 0x7f6be8198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:26.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bee91a700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be8107a40 0x7f6be8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 -- 172.21.15.150:0/3679713715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6be8102780 msgr2=0x7f6be8198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6be8102780 0x7f6be8198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 -- 172.21.15.150:0/3679713715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be8107a40 msgr2=0x7f6be8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be8107a40 0x7f6be8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.318+0000 7f6bed918700 1 -- 172.21.15.150:0/3679713715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80095f0 con 0x7f6be81050e0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.319+0000 7f6bed918700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8198660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80095c0 tx=0x7f6bd800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.319+0000 7f6bdf7fe700 1 -- 172.21.15.150:0/3679713715 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd801d070 con 0x7f6be81050e0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.319+0000 7f6bdf7fe700 1 -- 172.21.15.150:0/3679713715 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bd8005030 con 0x7f6be81050e0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.319+0000 7f6bdf7fe700 1 -- 172.21.15.150:0/3679713715 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8004ae0 con 0x7f6be81050e0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.319+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be8199ff0 con 0x7f6be81050e0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.319+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be819a270 con 0x7f6be81050e0 2024-05-10T04:39:26.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.320+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be80fdd70 con 0x7f6be81050e0 2024-05-10T04:39:26.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.321+0000 7f6bdf7fe700 1 -- 172.21.15.150:0/3679713715 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6bd80051a0 con 0x7f6be81050e0 2024-05-10T04:39:26.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.321+0000 7f6bdf7fe700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6bd40a11b0 0x7f6bd40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:26.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.322+0000 7f6bee119700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6bd40a11b0 0x7f6bd40a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:26.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.322+0000 7f6bdf7fe700 1 -- 172.21.15.150:0/3679713715 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6bd80ce0e0 con 0x7f6be81050e0 2024-05-10T04:39:26.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.322+0000 7f6bee119700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6bd40a11b0 0x7f6bd40a36c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005950 tx=0x7f6be400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:26.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.325+0000 7f6bdf7fe700 1 -- 172.21.15.150:0/3679713715 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6bd809af20 con 0x7f6be81050e0 2024-05-10T04:39:26.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.561+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6be80ffe80 con 0x7f6be81050e0 2024-05-10T04:39:26.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.561+0000 7f6bdf7fe700 1 -- 172.21.15.150:0/3679713715 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2939 (secure 0 0 0) 0x7f6bd809aab0 con 0x7f6be81050e0 2024-05-10T04:39:26.563 INFO:teuthology.orchestra.run.smithi150.stdout:e28 2024-05-10T04:39:26.563 INFO:teuthology.orchestra.run.smithi150.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-10T04:39:26.563 INFO:teuthology.orchestra.run.smithi150.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T04:39:26.563 INFO:teuthology.orchestra.run.smithi150.stdout:legacy client fscid: 1 2024-05-10T04:39:26.563 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:Filesystem 'cephfs' (1) 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:fs_name cephfs 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 28 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-05-10T04:38:08.750329+0000 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:modified 2024-05-10T04:38:35.978906+0000 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:tableserver 0 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:root 0 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:session_timeout 300 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:session_autoclose 300 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:max_file_size 1099511627776 2024-05-10T04:39:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:max_xattr_size 65536 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:required_client_features {} 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:last_failure 0 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:last_failure_osd_epoch 0 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:max_mds 5 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:in 0,1,2,3,4 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:up {0=24548,1=24520,2=24484,3=24455,4=24445} 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:failed 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:damaged 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:stopped 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:data_pools [3,4] 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:metadata_pool 2 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:inline_data disabled 2024-05-10T04:39:26.565 INFO:teuthology.orchestra.run.smithi150.stdout:balancer 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:bal_rank_mask -1 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:standby_count_wanted 0 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b{0:24548} state up:active seq 3 addr [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.h{0:24509} state up:standby-replay seq 1 addr [v2:172.21.15.161:6836/55043330,v1:172.21.15.161:6837/55043330] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.i{1:24520} state up:active seq 10 addr [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c{1:24496} state up:standby-replay seq 1 addr [v2:172.21.15.181:6836/1032605592,v1:172.21.15.181:6837/1032605592] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.f{2:24484} state up:active seq 12 addr [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a{2:14676} state up:standby-replay seq 1 addr [v2:172.21.15.150:6840/3789106275,v1:172.21.15.150:6841/3789106275] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.k{3:24455} state up:active seq 14 addr [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.g{3:14664} state up:standby-replay seq 1 addr [v2:172.21.15.150:6838/308347873,v1:172.21.15.150:6839/308347873] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.l{4:24445} state up:active seq 14 addr [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.e{4:14622} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/2306790268,v1:172.21.15.161:6835/2306790268] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:26.566 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:26.567 INFO:teuthology.orchestra.run.smithi150.stdout:Standby daemons: 2024-05-10T04:39:26.567 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:26.567 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.j{-1:14598} state up:standby seq 1 addr [v2:172.21.15.150:6834/2690665605,v1:172.21.15.150:6835/2690665605] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.567 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d{-1:14610} state up:standby seq 1 addr [v2:172.21.15.150:6836/2935316346,v1:172.21.15.150:6837/2935316346] compat {c=[1],r=[1],i=[7ff]}] 2024-05-10T04:39:26.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6bd40a11b0 msgr2=0x7f6bd40a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:26.567 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6bd40a11b0 0x7f6bd40a36c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005950 tx=0x7f6be400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 msgr2=0x7f6be8198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8198660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80095c0 tx=0x7f6bd800f690 comp rx=0 tx=0).stop 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 shutdown_connections 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be8107a40 0x7f6be8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6bd40a11b0 0x7f6bd40a36c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6be8102780 0x7f6be8198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 --2- 172.21.15.150:0/3679713715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6be81050e0 0x7f6be8198660 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 >> 172.21.15.150:0/3679713715 conn(0x7f6be80fc1c0 msgr2=0x7f6be81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 shutdown_connections 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:26.564+0000 7f6bf037d700 1 -- 172.21.15.150:0/3679713715 wait complete. 2024-05-10T04:39:26.568 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 28 2024-05-10T04:39:26.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:26 smithi181 ceph-mon[47314]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:26.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:26 smithi181 ceph-mon[47314]: pgmap v265: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s; 135 B/s, 1 objects/s recovering 2024-05-10T04:39:26.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:26 smithi150 ceph-mon[42404]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:26.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:26 smithi150 ceph-mon[42404]: pgmap v265: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s; 135 B/s, 1 objects/s recovering 2024-05-10T04:39:26.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:26 smithi161 ceph-mon[46912]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:26.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:26 smithi161 ceph-mon[46912]: pgmap v265: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s; 135 B/s, 1 objects/s recovering 2024-05-10T04:39:26.919 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-10T04:39:27.145 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:27.563 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:27 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3679713715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T04:39:27.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:27 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3679713715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T04:39:27.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:27 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3679713715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 -- 172.21.15.150:0/1690634415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 msgr2=0x7f02c01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 --2- 172.21.15.150:0/1690634415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c01020c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009990 tx=0x7f02b0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 -- 172.21.15.150:0/1690634415 shutdown_connections 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 --2- 172.21.15.150:0/1690634415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0102650 0x7f02c0108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 --2- 172.21.15.150:0/1690634415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0100b80 0x7f02c0100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 --2- 172.21.15.150:0/1690634415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c01020c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 -- 172.21.15.150:0/1690634415 >> 172.21.15.150:0/1690634415 conn(0x7f02c00fc1c0 msgr2=0x7f02c00fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 -- 172.21.15.150:0/1690634415 shutdown_connections 2024-05-10T04:39:28.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.100+0000 7f02c6800700 1 -- 172.21.15.150:0/1690634415 wait complete. 2024-05-10T04:39:28.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.101+0000 7f02c6800700 1 Processor -- start 2024-05-10T04:39:28.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.101+0000 7f02c6800700 1 -- start start 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.101+0000 7f02c6800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0100b80 0x7f02c0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.101+0000 7f02c6800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.101+0000 7f02c6800700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0102650 0x7f02c0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02c6800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0199a80 con 0x7f02c0101cb0 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02c6800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0199bc0 con 0x7f02c0100b80 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02c6800700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0199d80 con 0x7f02c0102650 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40668/0 (socket says 172.21.15.150:40668) 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 -- 172.21.15.150:0/2752240834 learned_addr learned my addr 172.21.15.150:0/2752240834 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bffff700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0100b80 0x7f02c0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02c4d9d700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0102650 0x7f02c0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 -- 172.21.15.150:0/2752240834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0102650 msgr2=0x7f02c0199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0102650 0x7f02c0199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 -- 172.21.15.150:0/2752240834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0100b80 msgr2=0x7f02c0198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0100b80 0x7f02c0198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 -- 172.21.15.150:0/2752240834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00095f0 con 0x7f02c0101cb0 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bf7fe700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c0198670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f02b00095c0 tx=0x7f02b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bd7fa700 1 -- 172.21.15.150:0/2752240834 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b001d070 con 0x7f02c0101cb0 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bd7fa700 1 -- 172.21.15.150:0/2752240834 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b0005030 con 0x7f02c0101cb0 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c019a000 con 0x7f02c0101cb0 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02bd7fa700 1 -- 172.21.15.150:0/2752240834 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0004ae0 con 0x7f02c0101cb0 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.102+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c01a0840 con 0x7f02c0101cb0 2024-05-10T04:39:28.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.103+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c00ffe80 con 0x7f02c0101cb0 2024-05-10T04:39:28.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.106+0000 7f02bd7fa700 1 -- 172.21.15.150:0/2752240834 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f02b00051a0 con 0x7f02c0101cb0 2024-05-10T04:39:28.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.106+0000 7f02bd7fa700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02ac0a11b0 0x7f02ac0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.106+0000 7f02bffff700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02ac0a11b0 0x7f02ac0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.106+0000 7f02bd7fa700 1 -- 172.21.15.150:0/2752240834 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f02b00ce0e0 con 0x7f02c0101cb0 2024-05-10T04:39:28.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.107+0000 7f02bffff700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02ac0a11b0 0x7f02ac0a36c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f02c0101bb0 tx=0x7f02a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:28.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.110+0000 7f02bd7fa700 1 -- 172.21.15.150:0/2752240834 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f02b009af20 con 0x7f02c0101cb0 2024-05-10T04:39:28.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.306+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f02c004edc0 con 0x7f02c0101cb0 2024-05-10T04:39:28.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.307+0000 7f02bd7fa700 1 -- 172.21.15.150:0/2752240834 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5618 (secure 0 0 0) 0x7f02b009aab0 con 0x7f02c0101cb0 2024-05-10T04:39:28.308 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 88 2024-05-10T04:39:28.308 INFO:teuthology.orchestra.run.smithi150.stdout:fsid 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:39:28.308 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-05-10T04:32:24.457948+0000 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:modified 2024-05-10T04:39:12.434464+0000 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:crush_version 27 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:full_ratio 0.95 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:backfillfull_ratio 0.9 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:nearfull_ratio 0.85 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:require_min_compat_client luminous 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:min_compat_client luminous 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:require_osd_release reef 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:stretch_mode_enabled false 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:max_osd 12 2024-05-10T04:39:28.309 INFO:teuthology.orchestra.run.smithi150.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6802/4183993127,v1:172.21.15.150:6803/4183993127] [v2:172.21.15.150:6804/4183993127,v1:172.21.15.150:6805/4183993127] exists,up cc48a860-ef1e-4d94-be5a-6c8758a2b7e3 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/2361791264,v1:172.21.15.161:6801/2361791264] [v2:172.21.15.161:6802/2361791264,v1:172.21.15.161:6803/2361791264] exists,up 13a6ad73-ce28-44f3-bdf4-178f42703543 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/2822398363,v1:172.21.15.181:6801/2822398363] [v2:172.21.15.181:6802/2822398363,v1:172.21.15.181:6803/2822398363] exists,up d1c4f641-65de-4cd3-8a02-b1694aa6e086 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6810/217079495,v1:172.21.15.150:6811/217079495] [v2:172.21.15.150:6812/217079495,v1:172.21.15.150:6813/217079495] exists,up 589765c5-703f-4ac8-b3d4-07fdbc0de50e 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/402152898,v1:172.21.15.161:6809/402152898] [v2:172.21.15.161:6810/402152898,v1:172.21.15.161:6811/402152898] exists,up 8507ef53-8650-4e6f-b8a2-482d3b632548 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/189565731,v1:172.21.15.181:6809/189565731] [v2:172.21.15.181:6810/189565731,v1:172.21.15.181:6811/189565731] exists,up 4632c68c-e68e-44c9-9f14-8f5a2c836c06 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.6 up in weight 1 up_from 45 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6818/3315977759,v1:172.21.15.150:6819/3315977759] [v2:172.21.15.150:6820/3315977759,v1:172.21.15.150:6821/3315977759] exists,up 89f0a895-52f5-4cb3-97df-71df296938c8 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.7 up in weight 1 up_from 50 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/724331973,v1:172.21.15.161:6817/724331973] [v2:172.21.15.161:6818/724331973,v1:172.21.15.161:6819/724331973] exists,up 09b5c539-71b8-42c8-97c4-3a3de4864a5a 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.8 up in weight 1 up_from 57 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/646590427,v1:172.21.15.181:6817/646590427] [v2:172.21.15.181:6818/646590427,v1:172.21.15.181:6819/646590427] exists,up 4aceac05-7393-4887-8b99-8ed52159a6a0 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.9 up in weight 1 up_from 64 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6826/2987376476,v1:172.21.15.150:6827/2987376476] [v2:172.21.15.150:6828/2987376476,v1:172.21.15.150:6829/2987376476] exists,up f84f2137-43f9-4072-8828-316f6a5f3582 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.10 up in weight 1 up_from 69 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2621232559,v1:172.21.15.161:6825/2621232559] [v2:172.21.15.161:6826/2621232559,v1:172.21.15.161:6827/2621232559] exists,up 0a0228ed-9ae2-4aa7-bd86-db1412ad4969 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:osd.11 up in weight 1 up_from 77 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/2057565898,v1:172.21.15.181:6825/2057565898] [v2:172.21.15.181:6826/2057565898,v1:172.21.15.181:6827/2057565898] exists,up 815f00ef-e11c-42c7-a1b4-52bbfefc253e 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 2.0 [7,11] 2024-05-10T04:39:28.310 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 2.c [2,4] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 2.e [2,1] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 2.10 [8,11] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 3.e [4,8] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 3.11 [4,6] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.12 [9,10] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.18 [9,8] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.1c [9,8] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:pg_upmap_items 4.1d [5,8] 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/1928870080 expires 2024-05-11T04:33:11.238386+0000 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/3285068402 expires 2024-05-11T04:33:11.238386+0000 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6801/881426482 expires 2024-05-11T04:33:11.238386+0000 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/618215791 expires 2024-05-11T04:32:48.043455+0000 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/2517418767 expires 2024-05-11T04:32:48.043455+0000 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/3236490116 expires 2024-05-11T04:32:48.043455+0000 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6800/881426482 expires 2024-05-11T04:33:11.238386+0000 2024-05-10T04:39:28.311 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6801/2970915543 expires 2024-05-11T04:32:48.043455+0000 2024-05-10T04:39:28.312 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:6800/2970915543 expires 2024-05-11T04:32:48.043455+0000 2024-05-10T04:39:28.312 INFO:teuthology.orchestra.run.smithi150.stdout:blocklist 172.21.15.150:0/3890124032 expires 2024-05-11T04:33:11.238386+0000 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02ac0a11b0 msgr2=0x7f02ac0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02ac0a11b0 0x7f02ac0a36c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f02c0101bb0 tx=0x7f02a800b410 comp rx=0 tx=0).stop 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 msgr2=0x7f02c0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c0198670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f02b00095c0 tx=0x7f02b000f690 comp rx=0 tx=0).stop 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 shutdown_connections 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f02ac0a11b0 0x7f02ac0a36c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0102650 0x7f02c0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02c0100b80 0x7f02c0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 --2- 172.21.15.150:0/2752240834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02c0101cb0 0x7f02c0198670 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 >> 172.21.15.150:0/2752240834 conn(0x7f02c00fc1c0 msgr2=0x7f02c00fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:28.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 shutdown_connections 2024-05-10T04:39:28.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.309+0000 7f02c6800700 1 -- 172.21.15.150:0/2752240834 wait complete. 2024-05-10T04:39:28.660 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:28 smithi150 ceph-mon[42404]: pgmap v266: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-05-10T04:39:28.660 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:28 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2752240834' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T04:39:28.711 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-10T04:39:28.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:28.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:28 smithi161 ceph-mon[46912]: pgmap v266: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-05-10T04:39:28.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:28 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2752240834' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T04:39:28.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 -- 172.21.15.150:0/386885448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338102580 msgr2=0x7f8338108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 --2- 172.21.15.150:0/386885448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338102580 0x7f8338108e30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f833000b0a0 tx=0x7f833000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 -- 172.21.15.150:0/386885448 shutdown_connections 2024-05-10T04:39:28.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 --2- 172.21.15.150:0/386885448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338102580 0x7f8338108e30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 --2- 172.21.15.150:0/386885448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8338100ac0 0x7f8338100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 --2- 172.21.15.150:0/386885448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8338101be0 0x7f8338101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 -- 172.21.15.150:0/386885448 >> 172.21.15.150:0/386885448 conn(0x7f83380fc0d0 msgr2=0x7f83380fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:28.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 -- 172.21.15.150:0/386885448 shutdown_connections 2024-05-10T04:39:28.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.887+0000 7f833f589700 1 -- 172.21.15.150:0/386885448 wait complete. 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 Processor -- start 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 -- start start 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8338100ac0 0x7f8338196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338101be0 0x7f8338197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8338102580 0x7f833819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833819eaa0 con 0x7f8338101be0 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833819ec10 con 0x7f8338100ac0 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833f589700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833819edd0 con 0x7f8338102580 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833cb24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338101be0 0x7f8338197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833cb24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338101be0 0x7f8338197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40672/0 (socket says 172.21.15.150:40672) 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833cb24700 1 -- 172.21.15.150:0/166213377 learned_addr learned my addr 172.21.15.150:0/166213377 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:28.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.888+0000 7f833d325700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8338100ac0 0x7f8338196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833db26700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8338102580 0x7f833819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833cb24700 1 -- 172.21.15.150:0/166213377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8338102580 msgr2=0x7f833819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833cb24700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8338102580 0x7f833819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833cb24700 1 -- 172.21.15.150:0/166213377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8338100ac0 msgr2=0x7f8338196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833cb24700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8338100ac0 0x7f8338196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833cb24700 1 -- 172.21.15.150:0/166213377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8330009d00 con 0x7f8338101be0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833cb24700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338101be0 0x7f8338197500 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f833400d950 tx=0x7f833400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f832a7fc700 1 -- 172.21.15.150:0/166213377 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334009940 con 0x7f8338101be0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f832a7fc700 1 -- 172.21.15.150:0/166213377 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8334010460 con 0x7f8338101be0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833819f0b0 con 0x7f8338101be0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f832a7fc700 1 -- 172.21.15.150:0/166213377 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400f620 con 0x7f8338101be0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.889+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833819f5c0 con 0x7f8338101be0 2024-05-10T04:39:28.894 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.890+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338107cd0 con 0x7f8338101be0 2024-05-10T04:39:28.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.891+0000 7f832a7fc700 1 -- 172.21.15.150:0/166213377 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f83340108d0 con 0x7f8338101be0 2024-05-10T04:39:28.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.892+0000 7f832a7fc700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f83240a11b0 0x7f83240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:28.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.892+0000 7f833d325700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f83240a11b0 0x7f83240a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:28.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.892+0000 7f832a7fc700 1 -- 172.21.15.150:0/166213377 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83340ccc80 con 0x7f8338101be0 2024-05-10T04:39:28.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.892+0000 7f833d325700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f83240a11b0 0x7f83240a36c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8338101ae0 tx=0x7f832c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:28.896 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:28.895+0000 7f832a7fc700 1 -- 172.21.15.150:0/166213377 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8334095f40 con 0x7f8338101be0 2024-05-10T04:39:28.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:28 smithi181 ceph-mon[47314]: pgmap v266: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-05-10T04:39:28.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:28 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2752240834' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-10T04:39:29.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.099+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83380694d0 con 0x7f8338101be0 2024-05-10T04:39:29.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.101+0000 7f832a7fc700 1 -- 172.21.15.150:0/166213377 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f8334016090 con 0x7f8338101be0 2024-05-10T04:39:29.103 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:29.103 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.103+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f83240a11b0 msgr2=0x7f83240a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.103+0000 7f833f589700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f83240a11b0 0x7f83240a36c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8338101ae0 tx=0x7f832c009260 comp rx=0 tx=0).stop 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.103+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338101be0 msgr2=0x7f8338197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.103+0000 7f833f589700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338101be0 0x7f8338197500 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f833400d950 tx=0x7f833400dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 shutdown_connections 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f83240a11b0 0x7f83240a36c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8338102580 0x7f833819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8338100ac0 0x7f8338196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 --2- 172.21.15.150:0/166213377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8338101be0 0x7f8338197500 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 >> 172.21.15.150:0/166213377 conn(0x7f83380fc0d0 msgr2=0x7f8338104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:29.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 shutdown_connections 2024-05-10T04:39:29.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.104+0000 7f833f589700 1 -- 172.21.15.150:0/166213377 wait complete. 2024-05-10T04:39:29.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:39:29.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:39:29.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 -- 172.21.15.150:0/926652478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc01050f0 msgr2=0x7f4bc0107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/926652478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc01050f0 0x7f4bc0107510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009990 tx=0x7f4bb0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 -- 172.21.15.150:0/926652478 shutdown_connections 2024-05-10T04:39:29.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/926652478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc0107a50 0x7f4bc0109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/926652478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc00fff80 0x7f4bc0100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/926652478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc01050f0 0x7f4bc0107510 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 -- 172.21.15.150:0/926652478 >> 172.21.15.150:0/926652478 conn(0x7f4bc00fb8f0 msgr2=0x7f4bc00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 -- 172.21.15.150:0/926652478 shutdown_connections 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.254+0000 7f4bc58a7700 1 -- 172.21.15.150:0/926652478 wait complete. 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 Processor -- start 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 -- start start 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc00fff80 0x7f4bc0102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc01050f0 0x7f4bc0100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc0107a50 0x7f4bc0101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0101ab0 con 0x7f4bc0107a50 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0101c20 con 0x7f4bc00fff80 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bc58a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0101de0 con 0x7f4bc01050f0 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc0107a50 0x7f4bc0101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc0107a50 0x7f4bc0101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40692/0 (socket says 172.21.15.150:40692) 2024-05-10T04:39:29.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bbf7fe700 1 -- 172.21.15.150:0/1459572582 learned_addr learned my addr 172.21.15.150:0/1459572582 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bbe7fc700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc01050f0 0x7f4bc0100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.255+0000 7f4bbeffd700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc00fff80 0x7f4bc0102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bbf7fe700 1 -- 172.21.15.150:0/1459572582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc01050f0 msgr2=0x7f4bc0100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bbf7fe700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc01050f0 0x7f4bc0100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bbf7fe700 1 -- 172.21.15.150:0/1459572582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc00fff80 msgr2=0x7f4bc0102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bbf7fe700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc00fff80 0x7f4bc0102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bbf7fe700 1 -- 172.21.15.150:0/1459572582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00095f0 con 0x7f4bc0107a50 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bbeffd700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc00fff80 0x7f4bc0102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bbf7fe700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc0107a50 0x7f4bc0101450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007d70 tx=0x7f4bb800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bc48a5700 1 -- 172.21.15.150:0/1459572582 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800ad40 con 0x7f4bc0107a50 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc01a2110 con 0x7f4bc0107a50 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bc48a5700 1 -- 172.21.15.150:0/1459572582 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bb80128e0 con 0x7f4bc0107a50 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bc48a5700 1 -- 172.21.15.150:0/1459572582 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80116b0 con 0x7f4bc0107a50 2024-05-10T04:39:29.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.256+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc01a25d0 con 0x7f4bc0107a50 2024-05-10T04:39:29.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.257+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc00fd4a0 con 0x7f4bc0107a50 2024-05-10T04:39:29.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.258+0000 7f4bc48a5700 1 -- 172.21.15.150:0/1459572582 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4bb8012420 con 0x7f4bc0107a50 2024-05-10T04:39:29.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.259+0000 7f4bc48a5700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4ba80a1260 0x7f4ba80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.259+0000 7f4bc48a5700 1 -- 172.21.15.150:0/1459572582 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4bb80cce60 con 0x7f4bc0107a50 2024-05-10T04:39:29.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.259+0000 7f4bbeffd700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4ba80a1260 0x7f4ba80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.259+0000 7f4bbeffd700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4ba80a1260 0x7f4ba80a3770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0103070 tx=0x7f4bb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:29.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.263+0000 7f4bc48a5700 1 -- 172.21.15.150:0/1459572582 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4bb8096290 con 0x7f4bc0107a50 2024-05-10T04:39:29.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.462+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bc010d3c0 con 0x7f4bc0107a50 2024-05-10T04:39:29.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.462+0000 7f4bc48a5700 1 -- 172.21.15.150:0/1459572582 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bb80998b0 con 0x7f4bc0107a50 2024-05-10T04:39:29.464 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.464+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4ba80a1260 msgr2=0x7f4ba80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.464+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4ba80a1260 0x7f4ba80a3770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0103070 tx=0x7f4bb4009260 comp rx=0 tx=0).stop 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.464+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc0107a50 msgr2=0x7f4bc0101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.464+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc0107a50 0x7f4bc0101450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007d70 tx=0x7f4bb800a340 comp rx=0 tx=0).stop 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 shutdown_connections 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc0107a50 0x7f4bc0101450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4bc00fff80 0x7f4bc0102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4ba80a1260 0x7f4ba80a3770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 --2- 172.21.15.150:0/1459572582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4bc01050f0 0x7f4bc0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 >> 172.21.15.150:0/1459572582 conn(0x7f4bc00fb8f0 msgr2=0x7f4bc0106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:29.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 shutdown_connections 2024-05-10T04:39:29.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.465+0000 7f4bc58a7700 1 -- 172.21.15.150:0/1459572582 wait complete. 2024-05-10T04:39:29.479 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 -- 172.21.15.150:0/4194667648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 msgr2=0x7fa468104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 --2- 172.21.15.150:0/4194667648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468104ae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009990 tx=0x7fa45c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 -- 172.21.15.150:0/4194667648 shutdown_connections 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 --2- 172.21.15.150:0/4194667648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468107980 0x7fa468109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 --2- 172.21.15.150:0/4194667648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468104ae0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 --2- 172.21.15.150:0/4194667648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468105020 0x7fa468107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 -- 172.21.15.150:0/4194667648 >> 172.21.15.150:0/4194667648 conn(0x7fa4680fc0c0 msgr2=0x7fa4680fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 -- 172.21.15.150:0/4194667648 shutdown_connections 2024-05-10T04:39:29.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.614+0000 7fa46c930700 1 -- 172.21.15.150:0/4194667648 wait complete. 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 Processor -- start 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 -- start start 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468105020 0x7fa468198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468107980 0x7fa46819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46819f9d0 con 0x7fa4681026c0 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46819fb40 con 0x7fa468107980 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46c930700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46819fd00 con 0x7fa468105020 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40710/0 (socket says 172.21.15.150:40710) 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa46659c700 1 -- 172.21.15.150:0/765499930 learned_addr learned my addr 172.21.15.150:0/765499930 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa465d9b700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468105020 0x7fa468198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.615+0000 7fa466d9d700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468107980 0x7fa46819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46659c700 1 -- 172.21.15.150:0/765499930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468105020 msgr2=0x7fa468198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46659c700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468105020 0x7fa468198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46659c700 1 -- 172.21.15.150:0/765499930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468107980 msgr2=0x7fa46819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46659c700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468107980 0x7fa46819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46659c700 1 -- 172.21.15.150:0/765499930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c0095f0 con 0x7fa4681026c0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa466d9d700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468107980 0x7fa46819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46659c700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468197ef0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa45c006010 tx=0x7fa45c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa4577fe700 1 -- 172.21.15.150:0/765499930 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c01d070 con 0x7fa4681026c0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46819ff80 con 0x7fa4681026c0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa4577fe700 1 -- 172.21.15.150:0/765499930 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45c022470 con 0x7fa4681026c0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa4577fe700 1 -- 172.21.15.150:0/765499930 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c017650 con 0x7fa4681026c0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.616+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4681a02f0 con 0x7fa4681026c0 2024-05-10T04:39:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.617+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4680fdc70 con 0x7fa4681026c0 2024-05-10T04:39:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.618+0000 7fa4577fe700 1 -- 172.21.15.150:0/765499930 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa45c0225e0 con 0x7fa4681026c0 2024-05-10T04:39:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.619+0000 7fa4577fe700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4500a1260 0x7fa4500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.619+0000 7fa4577fe700 1 -- 172.21.15.150:0/765499930 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa45c0ce400 con 0x7fa4681026c0 2024-05-10T04:39:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.619+0000 7fa465d9b700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4500a1260 0x7fa4500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.619+0000 7fa465d9b700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4500a1260 0x7fa4500a3770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa468198f90 tx=0x7fa45800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:29.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.622+0000 7fa4577fe700 1 -- 172.21.15.150:0/765499930 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa45c097840 con 0x7fa4681026c0 2024-05-10T04:39:29.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.820+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa46810d3d0 con 0x7fa4681026c0 2024-05-10T04:39:29.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.821+0000 7fa4577fe700 1 -- 172.21.15.150:0/765499930 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa45c09ae60 con 0x7fa4681026c0 2024-05-10T04:39:29.822 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4500a1260 msgr2=0x7fa4500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4500a1260 0x7fa4500a3770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa468198f90 tx=0x7fa45800b410 comp rx=0 tx=0).stop 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 msgr2=0x7fa468197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468197ef0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa45c006010 tx=0x7fa45c00f690 comp rx=0 tx=0).stop 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 shutdown_connections 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa468107980 0x7fa46819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4681026c0 0x7fa468197ef0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4500a1260 0x7fa4500a3770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 --2- 172.21.15.150:0/765499930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa468105020 0x7fa468198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 >> 172.21.15.150:0/765499930 conn(0x7fa4680fc0c0 msgr2=0x7fa468103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 shutdown_connections 2024-05-10T04:39:29.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.823+0000 7fa46c930700 1 -- 172.21.15.150:0/765499930 wait complete. 2024-05-10T04:39:29.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:29 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/166213377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:29.836 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:29 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/166213377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:29.837 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:39:29.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:29 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/166213377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:29.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/3511687370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 msgr2=0x7f2f24109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/3511687370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f24109d90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00b0a0 tx=0x7f2f1c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/3511687370 shutdown_connections 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/3511687370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f24109d90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/3511687370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24069070 0x7f2f24104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/3511687370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f24104d10 0x7f2f241071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/3511687370 >> 172.21.15.150:0/3511687370 conn(0x7f2f240fc0d0 msgr2=0x7f2f240fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/3511687370 shutdown_connections 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/3511687370 wait complete. 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 Processor -- start 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.972+0000 7f2f2b5cd700 1 -- start start 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f24069070 0x7f2f24197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24104d10 0x7f2f24198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f2419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2419fab0 con 0x7f2f24107880 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2419fc20 con 0x7f2f24069070 2024-05-10T04:39:29.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2419fde0 con 0x7f2f24104d10 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f2419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f2419f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40730/0 (socket says 172.21.15.150:40730) 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 -- 172.21.15.150:0/1266618068 learned_addr learned my addr 172.21.15.150:0/1266618068 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29369700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f24069070 0x7f2f24197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f28b68700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24104d10 0x7f2f24198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 -- 172.21.15.150:0/1266618068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24104d10 msgr2=0x7f2f24198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24104d10 0x7f2f24198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 -- 172.21.15.150:0/1266618068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f24069070 msgr2=0x7f2f24197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f24069070 0x7f2f24197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 -- 172.21.15.150:0/1266618068 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c009d00 con 0x7f2f24107880 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f29b6a700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f2419f4a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00b070 tx=0x7f2f1c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f1a7fc700 1 -- 172.21.15.150:0/1266618068 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00e040 con 0x7f2f24107880 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f1a7fc700 1 -- 172.21.15.150:0/1266618068 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1c003c00 con 0x7f2f24107880 2024-05-10T04:39:29.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f241a0060 con 0x7f2f24107880 2024-05-10T04:39:29.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f1a7fc700 1 -- 172.21.15.150:0/1266618068 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c01b6c0 con 0x7f2f24107880 2024-05-10T04:39:29.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.973+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241a0480 con 0x7f2f24107880 2024-05-10T04:39:29.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.974+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f240fdc80 con 0x7f2f24107880 2024-05-10T04:39:29.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.976+0000 7f2f1a7fc700 1 -- 172.21.15.150:0/1266618068 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2f1c01d470 con 0x7f2f24107880 2024-05-10T04:39:29.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.976+0000 7f2f1a7fc700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f100a1260 0x7f2f100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:29.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.976+0000 7f2f1a7fc700 1 -- 172.21.15.150:0/1266618068 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f1c0ce1e0 con 0x7f2f24107880 2024-05-10T04:39:29.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.976+0000 7f2f29369700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f100a1260 0x7f2f100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:29.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.977+0000 7f2f29369700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f100a1260 0x7f2f100a3770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005950 tx=0x7f2f2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:29.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:29.980+0000 7f2f1a7fc700 1 -- 172.21.15.150:0/1266618068 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2f1c097590 con 0x7f2f24107880 2024-05-10T04:39:30.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.177+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f240fd660 con 0x7f2f24107880 2024-05-10T04:39:30.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.177+0000 7f2f1a7fc700 1 -- 172.21.15.150:0/1266618068 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f1c095e60 con 0x7f2f24107880 2024-05-10T04:39:30.179 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f100a1260 msgr2=0x7f2f100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f100a1260 0x7f2f100a3770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005950 tx=0x7f2f2000b410 comp rx=0 tx=0).stop 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 msgr2=0x7f2f2419f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f2419f4a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00b070 tx=0x7f2f1c0095a0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 shutdown_connections 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f24107880 0x7f2f2419f4a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f24069070 0x7f2f24197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f100a1260 0x7f2f100a3770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 --2- 172.21.15.150:0/1266618068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24104d10 0x7f2f24198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.179+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 >> 172.21.15.150:0/1266618068 conn(0x7f2f240fc0d0 msgr2=0x7f2f24103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:30.181 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.180+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 shutdown_connections 2024-05-10T04:39:30.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.180+0000 7f2f2b5cd700 1 -- 172.21.15.150:0/1266618068 wait complete. 2024-05-10T04:39:30.194 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 -- 172.21.15.150:0/2044177103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed8107a50 msgr2=0x7f5ed8109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/2044177103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed8107a50 0x7f5ed8109e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000b0a0 tx=0x7f5ed000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 -- 172.21.15.150:0/2044177103 shutdown_connections 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/2044177103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed8107a50 0x7f5ed8109e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/2044177103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed80fff80 0x7f5ed8100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/2044177103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed81050f0 0x7f5ed8107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 -- 172.21.15.150:0/2044177103 >> 172.21.15.150:0/2044177103 conn(0x7f5ed80fb8f0 msgr2=0x7f5ed80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 -- 172.21.15.150:0/2044177103 shutdown_connections 2024-05-10T04:39:30.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.329+0000 7f5ede7a7700 1 -- 172.21.15.150:0/2044177103 wait complete. 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 Processor -- start 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 -- start start 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed80fff80 0x7f5ed8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed81050f0 0x7f5ed8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed8107a50 0x7f5ed819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed819faf0 con 0x7f5ed81050f0 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed819fc60 con 0x7f5ed80fff80 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.330+0000 7f5ede7a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed819fe20 con 0x7f5ed8107a50 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed81050f0 0x7f5ed8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed81050f0 0x7f5ed8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40748/0 (socket says 172.21.15.150:40748) 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 -- 172.21.15.150:0/3476451586 learned_addr learned my addr 172.21.15.150:0/3476451586 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed7fff700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed80fff80 0x7f5ed8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5edcd44700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed8107a50 0x7f5ed819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 -- 172.21.15.150:0/3476451586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed8107a50 msgr2=0x7f5ed819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed8107a50 0x7f5ed819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 -- 172.21.15.150:0/3476451586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed80fff80 msgr2=0x7f5ed8198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed80fff80 0x7f5ed8198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 -- 172.21.15.150:0/3476451586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed0009d00 con 0x7f5ed81050f0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed77fe700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed81050f0 0x7f5ed8198550 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800d950 tx=0x7f5ec800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed57fa700 1 -- 172.21.15.150:0/3476451586 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800f890 con 0x7f5ed81050f0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed57fa700 1 -- 172.21.15.150:0/3476451586 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ec8010460 con 0x7f5ed81050f0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed81a0100 con 0x7f5ed81050f0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ed57fa700 1 -- 172.21.15.150:0/3476451586 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800e610 con 0x7f5ed81050f0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.331+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed81a0610 con 0x7f5ed81050f0 2024-05-10T04:39:30.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.332+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed80fd4a0 con 0x7f5ed81050f0 2024-05-10T04:39:30.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.334+0000 7f5ed57fa700 1 -- 172.21.15.150:0/3476451586 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5ec80108d0 con 0x7f5ed81050f0 2024-05-10T04:39:30.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.334+0000 7f5ed57fa700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5ec00a1260 0x7f5ec00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.335+0000 7f5ed57fa700 1 -- 172.21.15.150:0/3476451586 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ec80ccba0 con 0x7f5ed81050f0 2024-05-10T04:39:30.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.335+0000 7f5ed7fff700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5ec00a1260 0x7f5ec00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.335+0000 7f5ed7fff700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5ec00a1260 0x7f5ec00a3770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc005950 tx=0x7f5ecc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:30.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.338+0000 7f5ed57fa700 1 -- 172.21.15.150:0/3476451586 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5ec8095e60 con 0x7f5ed81050f0 2024-05-10T04:39:30.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.534+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5ed810d3c0 con 0x7f5ed81050f0 2024-05-10T04:39:30.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.534+0000 7f5ed57fa700 1 -- 172.21.15.150:0/3476451586 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5ec8020020 con 0x7f5ed81050f0 2024-05-10T04:39:30.536 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5ec00a1260 msgr2=0x7f5ec00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5ec00a1260 0x7f5ec00a3770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc005950 tx=0x7f5ecc00b410 comp rx=0 tx=0).stop 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed81050f0 msgr2=0x7f5ed8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed81050f0 0x7f5ed8198550 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800d950 tx=0x7f5ec800dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 shutdown_connections 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5ec00a1260 0x7f5ec00a3770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed8107a50 0x7f5ed819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ed80fff80 0x7f5ed8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 --2- 172.21.15.150:0/3476451586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed81050f0 0x7f5ed8198550 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 >> 172.21.15.150:0/3476451586 conn(0x7f5ed80fb8f0 msgr2=0x7f5ed8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 shutdown_connections 2024-05-10T04:39:30.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.537+0000 7f5ede7a7700 1 -- 172.21.15.150:0/3476451586 wait complete. 2024-05-10T04:39:30.550 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:30 smithi150 ceph-mon[42404]: pgmap v267: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-10T04:39:30.550 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:30 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1459572582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.550 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:30 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/765499930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.550 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:30 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1266618068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.551 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:39:30.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 -- 172.21.15.150:0/2525344216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20103690 msgr2=0x7f2f20109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 --2- 172.21.15.150:0/2525344216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20103690 0x7f2f20109f40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b0a0 tx=0x7f2f1800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 -- 172.21.15.150:0/2525344216 shutdown_connections 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 --2- 172.21.15.150:0/2525344216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20103690 0x7f2f20109f40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 --2- 172.21.15.150:0/2525344216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f20101bd0 0x7f2f20101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 --2- 172.21.15.150:0/2525344216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20102cf0 0x7f2f20103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 -- 172.21.15.150:0/2525344216 >> 172.21.15.150:0/2525344216 conn(0x7f2f200fd1e0 msgr2=0x7f2f200ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 -- 172.21.15.150:0/2525344216 shutdown_connections 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 -- 172.21.15.150:0/2525344216 wait complete. 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.684+0000 7f2f24b91700 1 Processor -- start 2024-05-10T04:39:30.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f24b91700 1 -- start start 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f24b91700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20101bd0 0x7f2f201980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f24b91700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20102cf0 0x7f2f20198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f24b91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f20103690 0x7f2f2019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f24b91700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019fbb0 con 0x7f2f20102cf0 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f24b91700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019fd20 con 0x7f2f20101bd0 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f24b91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2019fee0 con 0x7f2f20103690 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20102cf0 0x7f2f20198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1e59c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20101bd0 0x7f2f201980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20102cf0 0x7f2f20198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40764/0 (socket says 172.21.15.150:40764) 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 -- 172.21.15.150:0/3066585958 learned_addr learned my addr 172.21.15.150:0/3066585958 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1ed9d700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f20103690 0x7f2f2019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 -- 172.21.15.150:0/3066585958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f20103690 msgr2=0x7f2f2019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f20103690 0x7f2f2019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 -- 172.21.15.150:0/3066585958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20101bd0 msgr2=0x7f2f201980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20101bd0 0x7f2f201980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 -- 172.21.15.150:0/3066585958 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18009d00 con 0x7f2f20102cf0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.685+0000 7f2f1dd9b700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20102cf0 0x7f2f20198610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000d980 tx=0x7f2f1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.686+0000 7f2f177fe700 1 -- 172.21.15.150:0/3066585958 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10004220 con 0x7f2f20102cf0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.686+0000 7f2f177fe700 1 -- 172.21.15.150:0/3066585958 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f10007640 con 0x7f2f20102cf0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.686+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f201a01c0 con 0x7f2f20102cf0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.686+0000 7f2f177fe700 1 -- 172.21.15.150:0/3066585958 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10005690 con 0x7f2f20102cf0 2024-05-10T04:39:30.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.686+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f201a06d0 con 0x7f2f20102cf0 2024-05-10T04:39:30.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.687+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f20108de0 con 0x7f2f20102cf0 2024-05-10T04:39:30.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.688+0000 7f2f177fe700 1 -- 172.21.15.150:0/3066585958 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f2f10004750 con 0x7f2f20102cf0 2024-05-10T04:39:30.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.688+0000 7f2f177fe700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f0c0a11b0 0x7f2f0c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:30.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.688+0000 7f2f177fe700 1 -- 172.21.15.150:0/3066585958 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f100ccaf0 con 0x7f2f20102cf0 2024-05-10T04:39:30.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.688+0000 7f2f1e59c700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f0c0a11b0 0x7f2f0c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:30.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.689+0000 7f2f1e59c700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f0c0a11b0 0x7f2f0c0a36c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f20102bf0 tx=0x7f2f08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:30.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.693+0000 7f2f177fe700 1 -- 172.21.15.150:0/3066585958 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2f10095e10 con 0x7f2f20102cf0 2024-05-10T04:39:30.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:30 smithi161 ceph-mon[46912]: pgmap v267: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-10T04:39:30.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:30 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1459572582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:30 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/765499930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:30 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1266618068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:30 smithi181 ceph-mon[47314]: pgmap v267: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-10T04:39:30.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:30 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1459572582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:30 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/765499930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:30 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1266618068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:30.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.933+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f2010d430 con 0x7f2f20102cf0 2024-05-10T04:39:30.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.934+0000 7f2f177fe700 1 -- 172.21.15.150:0/3066585958 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f2f10016020 con 0x7f2f20102cf0 2024-05-10T04:39:30.935 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:30.935 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:35.978906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:39:30.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f0c0a11b0 msgr2=0x7f2f0c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f0c0a11b0 0x7f2f0c0a36c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f20102bf0 tx=0x7f2f08009260 comp rx=0 tx=0).stop 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20102cf0 msgr2=0x7f2f20198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20102cf0 0x7f2f20198610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000d980 tx=0x7f2f1000dc90 comp rx=0 tx=0).stop 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 shutdown_connections 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2f0c0a11b0 0x7f2f0c0a36c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f20103690 0x7f2f2019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f20101bd0 0x7f2f201980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 --2- 172.21.15.150:0/3066585958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f20102cf0 0x7f2f20198610 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 >> 172.21.15.150:0/3066585958 conn(0x7f2f200fd1e0 msgr2=0x7f2f20105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 shutdown_connections 2024-05-10T04:39:30.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:30.936+0000 7f2f24b91700 1 -- 172.21.15.150:0/3066585958 wait complete. 2024-05-10T04:39:30.939 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 28 2024-05-10T04:39:30.951 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 -- 172.21.15.150:0/3783020069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c107970 msgr2=0x7f161c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 --2- 172.21.15.150:0/3783020069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c107970 0x7f161c109d90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f161400b0a0 tx=0x7f161400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 -- 172.21.15.150:0/3783020069 shutdown_connections 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 --2- 172.21.15.150:0/3783020069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c107970 0x7f161c109d90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 --2- 172.21.15.150:0/3783020069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f161c1026b0 0x7f161c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 --2- 172.21.15.150:0/3783020069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161c105010 0x7f161c107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 -- 172.21.15.150:0/3783020069 >> 172.21.15.150:0/3783020069 conn(0x7f161c0fc0d0 msgr2=0x7f161c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 -- 172.21.15.150:0/3783020069 shutdown_connections 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 -- 172.21.15.150:0/3783020069 wait complete. 2024-05-10T04:39:31.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.083+0000 7f16224e5700 1 Processor -- start 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f16224e5700 1 -- start start 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f16224e5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161c1026b0 0x7f161c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f16224e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c105010 0x7f161c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f16224e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f161c107970 0x7f161c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f16224e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c19fb70 con 0x7f161c105010 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f16224e5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c19fce0 con 0x7f161c107970 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f16224e5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c19fea0 con 0x7f161c1026b0 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c105010 0x7f161c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c105010 0x7f161c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40784/0 (socket says 172.21.15.150:40784) 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 -- 172.21.15.150:0/3903662652 learned_addr learned my addr 172.21.15.150:0/3903662652 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f1620a82700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f161c107970 0x7f161c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161bfff700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161c1026b0 0x7f161c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 -- 172.21.15.150:0/3903662652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161c1026b0 msgr2=0x7f161c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161c1026b0 0x7f161c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 -- 172.21.15.150:0/3903662652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f161c107970 msgr2=0x7f161c19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f161c107970 0x7f161c19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 -- 172.21.15.150:0/3903662652 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009d00 con 0x7f161c105010 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161bfff700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161c1026b0 0x7f161c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f1620a82700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f161c107970 0x7f161c19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.084+0000 7f161b7fe700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c105010 0x7f161c1985d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f160400c920 tx=0x7f160400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.085+0000 7f16197fa700 1 -- 172.21.15.150:0/3903662652 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604007b00 con 0x7f161c105010 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.085+0000 7f16197fa700 1 -- 172.21.15.150:0/3903662652 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1604004cf0 con 0x7f161c105010 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.085+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c1a0180 con 0x7f161c105010 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.085+0000 7f16197fa700 1 -- 172.21.15.150:0/3903662652 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604005610 con 0x7f161c105010 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.085+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c1a0690 con 0x7f161c105010 2024-05-10T04:39:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.086+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c0fdc80 con 0x7f161c105010 2024-05-10T04:39:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.088+0000 7f16197fa700 1 -- 172.21.15.150:0/3903662652 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1604004750 con 0x7f161c105010 2024-05-10T04:39:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.089+0000 7f16197fa700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16080a11b0 0x7f16080a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.089+0000 7f16197fa700 1 -- 172.21.15.150:0/3903662652 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f16040ccda0 con 0x7f161c105010 2024-05-10T04:39:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.089+0000 7f161bfff700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16080a11b0 0x7f16080a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.089+0000 7f161bfff700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16080a11b0 0x7f16080a36c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f160c005d90 tx=0x7f160c005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.092+0000 7f16197fa700 1 -- 172.21.15.150:0/3903662652 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1604096060 con 0x7f161c105010 2024-05-10T04:39:31.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.289+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f161c0694d0 con 0x7f161c105010 2024-05-10T04:39:31.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.291+0000 7f16197fa700 1 -- 172.21.15.150:0/3903662652 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f160401d020 con 0x7f161c105010 2024-05-10T04:39:31.293 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:31.293 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16080a11b0 msgr2=0x7f16080a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16080a11b0 0x7f16080a36c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f160c005d90 tx=0x7f160c005d20 comp rx=0 tx=0).stop 2024-05-10T04:39:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c105010 msgr2=0x7f161c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c105010 0x7f161c1985d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f160400c920 tx=0x7f160400cce0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 shutdown_connections 2024-05-10T04:39:31.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f161c107970 0x7f161c19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16080a11b0 0x7f16080a36c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161c1026b0 0x7f161c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 --2- 172.21.15.150:0/3903662652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f161c105010 0x7f161c1985d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 >> 172.21.15.150:0/3903662652 conn(0x7f161c0fc0d0 msgr2=0x7f161c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 shutdown_connections 2024-05-10T04:39:31.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.294+0000 7f16224e5700 1 -- 172.21.15.150:0/3903662652 wait complete. 2024-05-10T04:39:31.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:39:31.309 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:39:31.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 -- 172.21.15.150:0/2283496160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18102580 msgr2=0x7f4f18108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 --2- 172.21.15.150:0/2283496160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18102580 0x7f4f18108e30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b0a0 tx=0x7f4f1000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 -- 172.21.15.150:0/2283496160 shutdown_connections 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 --2- 172.21.15.150:0/2283496160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18102580 0x7f4f18108e30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 --2- 172.21.15.150:0/2283496160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f18100ac0 0x7f4f18100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 --2- 172.21.15.150:0/2283496160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f18101be0 0x7f4f18101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 -- 172.21.15.150:0/2283496160 >> 172.21.15.150:0/2283496160 conn(0x7f4f180fc0d0 msgr2=0x7f4f180fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 -- 172.21.15.150:0/2283496160 shutdown_connections 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.445+0000 7f4f1d507700 1 -- 172.21.15.150:0/2283496160 wait complete. 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 Processor -- start 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 -- start start 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18100ac0 0x7f4f18068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f18101be0 0x7f4f1819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f18102580 0x7f4f1819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1819fb10 con 0x7f4f18100ac0 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1819fc80 con 0x7f4f18101be0 2024-05-10T04:39:31.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f1d507700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1819fe40 con 0x7f4f18102580 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f16ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18100ac0 0x7f4f18068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f16ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18100ac0 0x7f4f18068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40794/0 (socket says 172.21.15.150:40794) 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.446+0000 7f4f16ffd700 1 -- 172.21.15.150:0/947266173 learned_addr learned my addr 172.21.15.150:0/947266173 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f177fe700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f18102580 0x7f4f1819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f167fc700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f18101be0 0x7f4f1819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f16ffd700 1 -- 172.21.15.150:0/947266173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f18102580 msgr2=0x7f4f1819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f16ffd700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f18102580 0x7f4f1819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f16ffd700 1 -- 172.21.15.150:0/947266173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f18101be0 msgr2=0x7f4f1819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f16ffd700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f18101be0 0x7f4f1819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f16ffd700 1 -- 172.21.15.150:0/947266173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10009d00 con 0x7f4f18100ac0 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f177fe700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f18102580 0x7f4f1819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f16ffd700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18100ac0 0x7f4f18068e50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d950 tx=0x7f4f0c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4efffff700 1 -- 172.21.15.150:0/947266173 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c004220 con 0x7f4f18100ac0 2024-05-10T04:39:31.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f181a00c0 con 0x7f4f18100ac0 2024-05-10T04:39:31.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4efffff700 1 -- 172.21.15.150:0/947266173 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c00de30 con 0x7f4f18100ac0 2024-05-10T04:39:31.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4efffff700 1 -- 172.21.15.150:0/947266173 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c003d00 con 0x7f4f18100ac0 2024-05-10T04:39:31.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.447+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f181a0620 con 0x7f4f18100ac0 2024-05-10T04:39:31.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.448+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f18107cd0 con 0x7f4f18100ac0 2024-05-10T04:39:31.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.449+0000 7f4efffff700 1 -- 172.21.15.150:0/947266173 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4f0c004380 con 0x7f4f18100ac0 2024-05-10T04:39:31.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.450+0000 7f4efffff700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4f000a1260 0x7f4f000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.450+0000 7f4f167fc700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4f000a1260 0x7f4f000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.450+0000 7f4efffff700 1 -- 172.21.15.150:0/947266173 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4f0c0cc990 con 0x7f4f18100ac0 2024-05-10T04:39:31.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.450+0000 7f4f167fc700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4f000a1260 0x7f4f000a3770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4f180687f0 tx=0x7f4f08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:31.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.455+0000 7f4efffff700 1 -- 172.21.15.150:0/947266173 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4f0c095c50 con 0x7f4f18100ac0 2024-05-10T04:39:31.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.650+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f18069470 con 0x7f4f18100ac0 2024-05-10T04:39:31.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.650+0000 7f4efffff700 1 -- 172.21.15.150:0/947266173 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f0c099270 con 0x7f4f18100ac0 2024-05-10T04:39:31.651 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4f000a1260 msgr2=0x7f4f000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4f000a1260 0x7f4f000a3770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4f180687f0 tx=0x7f4f08009260 comp rx=0 tx=0).stop 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18100ac0 msgr2=0x7f4f18068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18100ac0 0x7f4f18068e50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d950 tx=0x7f4f0c00dd10 comp rx=0 tx=0).stop 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 shutdown_connections 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4f000a1260 0x7f4f000a3770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f18102580 0x7f4f1819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f18100ac0 0x7f4f18068e50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 --2- 172.21.15.150:0/947266173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f18101be0 0x7f4f1819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 >> 172.21.15.150:0/947266173 conn(0x7f4f180fc0d0 msgr2=0x7f4f18104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 shutdown_connections 2024-05-10T04:39:31.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.652+0000 7f4f1d507700 1 -- 172.21.15.150:0/947266173 wait complete. 2024-05-10T04:39:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3476451586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3066585958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3903662652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:31.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:31 smithi150 ceph-mon[42404]: pgmap v268: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-10T04:39:31.667 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:39:31.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 -- 172.21.15.150:0/782549584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8101be0 msgr2=0x7f64e8101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 --2- 172.21.15.150:0/782549584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8101be0 0x7f64e8101ff0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009990 tx=0x7f64d4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 -- 172.21.15.150:0/782549584 shutdown_connections 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 --2- 172.21.15.150:0/782549584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8102580 0x7f64e8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 --2- 172.21.15.150:0/782549584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e8100ac0 0x7f64e8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 --2- 172.21.15.150:0/782549584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8101be0 0x7f64e8101ff0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 -- 172.21.15.150:0/782549584 >> 172.21.15.150:0/782549584 conn(0x7f64e80fc0d0 msgr2=0x7f64e80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 -- 172.21.15.150:0/782549584 shutdown_connections 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.802+0000 7f64eec96700 1 -- 172.21.15.150:0/782549584 wait complete. 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 Processor -- start 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 -- start start 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100ac0 0x7f64e8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8101be0 0x7f64e8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e8102580 0x7f64e819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819e8d0 con 0x7f64e8100ac0 2024-05-10T04:39:31.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819ea40 con 0x7f64e8101be0 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eec96700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819ec00 con 0x7f64e8102580 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eca32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100ac0 0x7f64e8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eca32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100ac0 0x7f64e8196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40824/0 (socket says 172.21.15.150:40824) 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64eca32700 1 -- 172.21.15.150:0/1661900258 learned_addr learned my addr 172.21.15.150:0/1661900258 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.803+0000 7f64e7fff700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8101be0 0x7f64e8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64ed233700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e8102580 0x7f64e819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eca32700 1 -- 172.21.15.150:0/1661900258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e8102580 msgr2=0x7f64e819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eca32700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e8102580 0x7f64e819e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eca32700 1 -- 172.21.15.150:0/1661900258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8101be0 msgr2=0x7f64e8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eca32700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8101be0 0x7f64e8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eca32700 1 -- 172.21.15.150:0/1661900258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40095f0 con 0x7f64e8100ac0 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64e7fff700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8101be0 0x7f64e8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eca32700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100ac0 0x7f64e8196e00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e940 tx=0x7f64dc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:31.808 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64e5ffb700 1 -- 172.21.15.150:0/1661900258 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00cbd0 con 0x7f64e8100ac0 2024-05-10T04:39:31.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64e5ffb700 1 -- 172.21.15.150:0/1661900258 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64dc0044e0 con 0x7f64e8100ac0 2024-05-10T04:39:31.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64e5ffb700 1 -- 172.21.15.150:0/1661900258 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc003dc0 con 0x7f64e8100ac0 2024-05-10T04:39:31.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e819eee0 con 0x7f64e8100ac0 2024-05-10T04:39:31.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.804+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e819f310 con 0x7f64e8100ac0 2024-05-10T04:39:31.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.805+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e8107cd0 con 0x7f64e8100ac0 2024-05-10T04:39:31.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.806+0000 7f64e5ffb700 1 -- 172.21.15.150:0/1661900258 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f64dc003f20 con 0x7f64e8100ac0 2024-05-10T04:39:31.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.807+0000 7f64e5ffb700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f64d80a12e0 0x7f64d80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:31.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.807+0000 7f64e5ffb700 1 -- 172.21.15.150:0/1661900258 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f64dc014070 con 0x7f64e8100ac0 2024-05-10T04:39:31.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.807+0000 7f64e7fff700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f64d80a12e0 0x7f64d80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:31.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.807+0000 7f64e7fff700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f64d80a12e0 0x7f64d80a37f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b2e0 tx=0x7f64d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:31.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:31.811+0000 7f64e5ffb700 1 -- 172.21.15.150:0/1661900258 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f64dc096390 con 0x7f64e8100ac0 2024-05-10T04:39:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3476451586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3066585958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3903662652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:31.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:31 smithi161 ceph-mon[46912]: pgmap v268: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-10T04:39:31.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3476451586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:31.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3066585958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:31.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3903662652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:31.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:31 smithi181 ceph-mon[47314]: pgmap v268: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-10T04:39:32.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.012+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64e810c2c0 con 0x7f64e8100ac0 2024-05-10T04:39:32.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.013+0000 7f64e5ffb700 1 -- 172.21.15.150:0/1661900258 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64dc0999b0 con 0x7f64e8100ac0 2024-05-10T04:39:32.014 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:32.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f64d80a12e0 msgr2=0x7f64d80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f64d80a12e0 0x7f64d80a37f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b2e0 tx=0x7f64d401a040 comp rx=0 tx=0).stop 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100ac0 msgr2=0x7f64e8196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100ac0 0x7f64e8196e00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e940 tx=0x7f64dc00ed00 comp rx=0 tx=0).stop 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 shutdown_connections 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f64d80a12e0 0x7f64d80a37f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e8102580 0x7f64e819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100ac0 0x7f64e8196e00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 --2- 172.21.15.150:0/1661900258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8101be0 0x7f64e8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 >> 172.21.15.150:0/1661900258 conn(0x7f64e80fc0d0 msgr2=0x7f64e8104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 shutdown_connections 2024-05-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.015+0000 7f64eec96700 1 -- 172.21.15.150:0/1661900258 wait complete. 2024-05-10T04:39:32.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:39:32.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.163+0000 7f5c79037700 1 -- 172.21.15.150:0/902461065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74101be0 msgr2=0x7f5c74101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.163+0000 7f5c79037700 1 --2- 172.21.15.150:0/902461065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74101be0 0x7f5c74101ff0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009990 tx=0x7f5c68009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.163+0000 7f5c79037700 1 -- 172.21.15.150:0/902461065 shutdown_connections 2024-05-10T04:39:32.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.163+0000 7f5c79037700 1 --2- 172.21.15.150:0/902461065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74102580 0x7f5c74108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.163+0000 7f5c79037700 1 --2- 172.21.15.150:0/902461065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74100ac0 0x7f5c74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.163+0000 7f5c79037700 1 --2- 172.21.15.150:0/902461065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74101be0 0x7f5c74101ff0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.163+0000 7f5c79037700 1 -- 172.21.15.150:0/902461065 >> 172.21.15.150:0/902461065 conn(0x7f5c740fc0d0 msgr2=0x7f5c740fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:32.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.164+0000 7f5c79037700 1 -- 172.21.15.150:0/902461065 shutdown_connections 2024-05-10T04:39:32.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.164+0000 7f5c79037700 1 -- 172.21.15.150:0/902461065 wait complete. 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.164+0000 7f5c79037700 1 Processor -- start 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c79037700 1 -- start start 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c79037700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74100ac0 0x7f5c74197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c79037700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74101be0 0x7f5c74198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c79037700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74102580 0x7f5c7419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c79037700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419f9c0 con 0x7f5c74100ac0 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c79037700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419fb30 con 0x7f5c74101be0 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c79037700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419fcf0 con 0x7f5c74102580 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74100ac0 0x7f5c74197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74100ac0 0x7f5c74197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40854/0 (socket says 172.21.15.150:40854) 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 -- 172.21.15.150:0/2693300752 learned_addr learned my addr 172.21.15.150:0/2693300752 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c7259c700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74101be0 0x7f5c74198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c7359e700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74102580 0x7f5c7419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 -- 172.21.15.150:0/2693300752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74102580 msgr2=0x7f5c7419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74102580 0x7f5c7419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 -- 172.21.15.150:0/2693300752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74101be0 msgr2=0x7f5c74198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74101be0 0x7f5c74198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 -- 172.21.15.150:0/2693300752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c680095f0 con 0x7f5c74100ac0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c7259c700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74101be0 0x7f5c74198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c72d9d700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74100ac0 0x7f5c74197ee0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400d950 tx=0x7f5c6400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.165+0000 7f5c5bfff700 1 -- 172.21.15.150:0/2693300752 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004220 con 0x7f5c74100ac0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.166+0000 7f5c5bfff700 1 -- 172.21.15.150:0/2693300752 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6400de30 con 0x7f5c74100ac0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.166+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7419ffd0 con 0x7f5c74100ac0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.166+0000 7f5c5bfff700 1 -- 172.21.15.150:0/2693300752 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64003cb0 con 0x7f5c74100ac0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.166+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c741a0400 con 0x7f5c74100ac0 2024-05-10T04:39:32.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.166+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c740694d0 con 0x7f5c74100ac0 2024-05-10T04:39:32.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.168+0000 7f5c5bfff700 1 -- 172.21.15.150:0/2693300752 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5c64004380 con 0x7f5c74100ac0 2024-05-10T04:39:32.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.169+0000 7f5c5bfff700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5c5c0a1260 0x7f5c5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.169+0000 7f5c7259c700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5c5c0a1260 0x7f5c5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.169+0000 7f5c5bfff700 1 -- 172.21.15.150:0/2693300752 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c640cd1a0 con 0x7f5c74100ac0 2024-05-10T04:39:32.171 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.170+0000 7f5c7259c700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5c5c0a1260 0x7f5c5c0a3770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c680038c0 tx=0x7f5c6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:32.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.173+0000 7f5c5bfff700 1 -- 172.21.15.150:0/2693300752 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5c64096460 con 0x7f5c74100ac0 2024-05-10T04:39:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.370+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c7410c510 con 0x7f5c74100ac0 2024-05-10T04:39:32.371 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.370+0000 7f5c5bfff700 1 -- 172.21.15.150:0/2693300752 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c64017020 con 0x7f5c74100ac0 2024-05-10T04:39:32.371 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5c5c0a1260 msgr2=0x7f5c5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5c5c0a1260 0x7f5c5c0a3770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c680038c0 tx=0x7f5c6801a040 comp rx=0 tx=0).stop 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74100ac0 msgr2=0x7f5c74197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74100ac0 0x7f5c74197ee0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400d950 tx=0x7f5c6400dd10 comp rx=0 tx=0).stop 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 shutdown_connections 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5c5c0a1260 0x7f5c5c0a3770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c74102580 0x7f5c7419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c74100ac0 0x7f5c74197ee0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 --2- 172.21.15.150:0/2693300752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c74101be0 0x7f5c74198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 >> 172.21.15.150:0/2693300752 conn(0x7f5c740fc0d0 msgr2=0x7f5c740683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 shutdown_connections 2024-05-10T04:39:32.375 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.372+0000 7f5c79037700 1 -- 172.21.15.150:0/2693300752 wait complete. 2024-05-10T04:39:32.387 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:39:32.526 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 -- 172.21.15.150:0/1366014364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 msgr2=0x7f778c107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 --2- 172.21.15.150:0/1366014364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c107430 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7788009990 tx=0x7f7788009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 -- 172.21.15.150:0/1366014364 shutdown_connections 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 --2- 172.21.15.150:0/1366014364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778c107970 0x7f778c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 --2- 172.21.15.150:0/1366014364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778c1026b0 0x7f778c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 --2- 172.21.15.150:0/1366014364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c107430 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 -- 172.21.15.150:0/1366014364 >> 172.21.15.150:0/1366014364 conn(0x7f778c0fc0d0 msgr2=0x7f778c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 -- 172.21.15.150:0/1366014364 shutdown_connections 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 -- 172.21.15.150:0/1366014364 wait complete. 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 Processor -- start 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.523+0000 7f7793b03700 1 -- start start 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7793b03700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778c1026b0 0x7f778c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7793b03700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7793b03700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778c107970 0x7f778c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7793b03700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c19fa30 con 0x7f778c105010 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7793b03700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c19fba0 con 0x7f778c1026b0 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7793b03700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c19fd60 con 0x7f778c107970 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40864/0 (socket says 172.21.15.150:40864) 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 -- 172.21.15.150:0/690641177 learned_addr learned my addr 172.21.15.150:0/690641177 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779189f700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778c1026b0 0x7f778c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f77920a0700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778c107970 0x7f778c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 -- 172.21.15.150:0/690641177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778c107970 msgr2=0x7f778c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778c107970 0x7f778c19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 -- 172.21.15.150:0/690641177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778c1026b0 msgr2=0x7f778c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778c1026b0 0x7f778c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 -- 172.21.15.150:0/690641177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880095f0 con 0x7f778c105010 2024-05-10T04:39:32.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779189f700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778c1026b0 0x7f778c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:32.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f779109e700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c198490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77880098e0 tx=0x7f77880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:32.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7782ffd700 1 -- 172.21.15.150:0/690641177 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778801d070 con 0x7f778c105010 2024-05-10T04:39:32.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c19ffe0 con 0x7f778c105010 2024-05-10T04:39:32.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7782ffd700 1 -- 172.21.15.150:0/690641177 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f778800fb90 con 0x7f778c105010 2024-05-10T04:39:32.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.524+0000 7f7782ffd700 1 -- 172.21.15.150:0/690641177 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788017650 con 0x7f778c105010 2024-05-10T04:39:32.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.525+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c1a0350 con 0x7f778c105010 2024-05-10T04:39:32.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.525+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778c0fdc80 con 0x7f778c105010 2024-05-10T04:39:32.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.527+0000 7f7782ffd700 1 -- 172.21.15.150:0/690641177 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f778800fd00 con 0x7f778c105010 2024-05-10T04:39:32.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.527+0000 7f7782ffd700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77780a1260 0x7f77780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.527+0000 7f7782ffd700 1 -- 172.21.15.150:0/690641177 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77880ce9b0 con 0x7f778c105010 2024-05-10T04:39:32.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.528+0000 7f779189f700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77780a1260 0x7f77780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.528+0000 7f779189f700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77780a1260 0x7f77780a3770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f777c005950 tx=0x7f777c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:32.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.531+0000 7f7782ffd700 1 -- 172.21.15.150:0/690641177 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7788097d00 con 0x7f778c105010 2024-05-10T04:39:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/947266173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1661900258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2693300752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.729 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.728+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f778c10d5c0 con 0x7f778c105010 2024-05-10T04:39:32.730 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.729+0000 7f7782ffd700 1 -- 172.21.15.150:0/690641177 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f778809b320 con 0x7f778c105010 2024-05-10T04:39:32.730 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:32.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77780a1260 msgr2=0x7f77780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77780a1260 0x7f77780a3770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f777c005950 tx=0x7f777c0058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 msgr2=0x7f778c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c198490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77880098e0 tx=0x7f77880057f0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 shutdown_connections 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77780a1260 0x7f77780a3770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778c107970 0x7f778c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f778c1026b0 0x7f778c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 --2- 172.21.15.150:0/690641177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f778c105010 0x7f778c198490 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 >> 172.21.15.150:0/690641177 conn(0x7f778c0fc0d0 msgr2=0x7f778c103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 shutdown_connections 2024-05-10T04:39:32.733 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.731+0000 7f7793b03700 1 -- 172.21.15.150:0/690641177 wait complete. 2024-05-10T04:39:32.745 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:39:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:32 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/947266173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:32 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1661900258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:32 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2693300752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 -- 172.21.15.150:0/2198313568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40100af0 msgr2=0x7f4e40100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/2198313568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40100af0 0x7f4e40100f00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e340099c0 tx=0x7f4e34009cd0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 -- 172.21.15.150:0/2198313568 shutdown_connections 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/2198313568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e401025b0 0x7f4e40108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/2198313568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40100af0 0x7f4e40100f00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/2198313568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e40101c10 0x7f4e40102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 -- 172.21.15.150:0/2198313568 >> 172.21.15.150:0/2198313568 conn(0x7f4e400fc100 msgr2=0x7f4e400fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 -- 172.21.15.150:0/2198313568 shutdown_connections 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.880+0000 7f4e46ae3700 1 -- 172.21.15.150:0/2198313568 wait complete. 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 Processor -- start 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 -- start start 2024-05-10T04:39:32.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e40100af0 0x7f4e4010b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40101c10 0x7f4e40109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e401025b0 0x7f4e40109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4010a630 con 0x7f4e40100af0 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4010a7a0 con 0x7f4e40101c10 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e46ae3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e401a2080 con 0x7f4e401025b0 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e4487f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e40100af0 0x7f4e4010b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e4487f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e40100af0 0x7f4e4010b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40874/0 (socket says 172.21.15.150:40874) 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e4487f700 1 -- 172.21.15.150:0/1170627333 learned_addr learned my addr 172.21.15.150:0/1170627333 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e3ffff700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40101c10 0x7f4e40109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e45080700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e401025b0 0x7f4e40109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e4487f700 1 -- 172.21.15.150:0/1170627333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e401025b0 msgr2=0x7f4e40109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.881+0000 7f4e4487f700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e401025b0 0x7f4e40109f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e4487f700 1 -- 172.21.15.150:0/1170627333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40101c10 msgr2=0x7f4e40109a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e4487f700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40101c10 0x7f4e40109a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:32.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e4487f700 1 -- 172.21.15.150:0/1170627333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e340095f0 con 0x7f4e40100af0 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e3ffff700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40101c10 0x7f4e40109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e45080700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e401025b0 0x7f4e40109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e4487f700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e40100af0 0x7f4e4010b370 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009fd0 tx=0x7f4e340036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e3dffb700 1 -- 172.21.15.150:0/1170627333 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3401d070 con 0x7f4e40100af0 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e401a2280 con 0x7f4e40100af0 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e3dffb700 1 -- 172.21.15.150:0/1170627333 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e340054f0 con 0x7f4e40100af0 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e3dffb700 1 -- 172.21.15.150:0/1170627333 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34004b30 con 0x7f4e40100af0 2024-05-10T04:39:32.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.882+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e401a2700 con 0x7f4e40100af0 2024-05-10T04:39:32.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.883+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e40107d00 con 0x7f4e40100af0 2024-05-10T04:39:32.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.885+0000 7f4e3dffb700 1 -- 172.21.15.150:0/1170627333 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4e3400fbc0 con 0x7f4e40100af0 2024-05-10T04:39:32.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.885+0000 7f4e3dffb700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4e300a11b0 0x7f4e300a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:32.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.886+0000 7f4e3ffff700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4e300a11b0 0x7f4e300a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:32.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.886+0000 7f4e3dffb700 1 -- 172.21.15.150:0/1170627333 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4e340cdf70 con 0x7f4e40100af0 2024-05-10T04:39:32.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.886+0000 7f4e3ffff700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4e300a11b0 0x7f4e300a36c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e4010ad10 tx=0x7f4e2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:32.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:32.889+0000 7f4e3dffb700 1 -- 172.21.15.150:0/1170627333 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e340972c0 con 0x7f4e40100af0 2024-05-10T04:39:32.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:32 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/947266173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:32 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1661900258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:32.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:32 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2693300752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:33.134 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.132+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e400678b0 con 0x7f4e40100af0 2024-05-10T04:39:33.135 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.133+0000 7f4e3dffb700 1 -- 172.21.15.150:0/1170627333 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f4e3409a8e0 con 0x7f4e40100af0 2024-05-10T04:39:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:33.135 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:38:35.978906+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4e300a11b0 msgr2=0x7f4e300a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4e300a11b0 0x7f4e300a36c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e4010ad10 tx=0x7f4e2c009260 comp rx=0 tx=0).stop 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e40100af0 msgr2=0x7f4e4010b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e40100af0 0x7f4e4010b370 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009fd0 tx=0x7f4e340036f0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 shutdown_connections 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4e300a11b0 0x7f4e300a36c0 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e4010ad10 tx=0x7f4e2c009260 comp rx=0 tx=0).stop 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e401025b0 0x7f4e40109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e40100af0 0x7f4e4010b370 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 --2- 172.21.15.150:0/1170627333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e40101c10 0x7f4e40109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.135+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 >> 172.21.15.150:0/1170627333 conn(0x7f4e400fc100 msgr2=0x7f4e40104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.136+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 shutdown_connections 2024-05-10T04:39:33.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.136+0000 7f4e46ae3700 1 -- 172.21.15.150:0/1170627333 wait complete. 2024-05-10T04:39:33.138 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 28 2024-05-10T04:39:33.150 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 -- 172.21.15.150:0/2271284494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a41025d0 msgr2=0x7f14a4109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 --2- 172.21.15.150:0/2271284494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a41025d0 0x7f14a4109050 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f149c00b0a0 tx=0x7f149c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 -- 172.21.15.150:0/2271284494 shutdown_connections 2024-05-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 --2- 172.21.15.150:0/2271284494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a41025d0 0x7f14a4109050 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 --2- 172.21.15.150:0/2271284494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4101090 0x7f14a41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 --2- 172.21.15.150:0/2271284494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a4101a70 0x7f14a4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 -- 172.21.15.150:0/2271284494 >> 172.21.15.150:0/2271284494 conn(0x7f14a40fb8f0 msgr2=0x7f14a40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 -- 172.21.15.150:0/2271284494 shutdown_connections 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.282+0000 7f14ac774700 1 -- 172.21.15.150:0/2271284494 wait complete. 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 Processor -- start 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 -- start start 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a4101090 0x7f14a4100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4101a70 0x7f14a419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41025d0 0x7f14a419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a419fa50 con 0x7f14a4101090 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a419fbc0 con 0x7f14a41025d0 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14ac774700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a419fd80 con 0x7f14a4101a70 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14aa510700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a4101090 0x7f14a4100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14aa510700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a4101090 0x7f14a4100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40886/0 (socket says 172.21.15.150:40886) 2024-05-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14aa510700 1 -- 172.21.15.150:0/2806714967 learned_addr learned my addr 172.21.15.150:0/2806714967 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14aad11700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41025d0 0x7f14a419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.283+0000 7f14a9d0f700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4101a70 0x7f14a419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14aa510700 1 -- 172.21.15.150:0/2806714967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4101a70 msgr2=0x7f14a419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14aa510700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4101a70 0x7f14a419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14aa510700 1 -- 172.21.15.150:0/2806714967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41025d0 msgr2=0x7f14a419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14aa510700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41025d0 0x7f14a419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14aa510700 1 -- 172.21.15.150:0/2806714967 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c009d00 con 0x7f14a4101090 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14aa510700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a4101090 0x7f14a4100750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f149800b4d0 tx=0x7f149800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14977fe700 1 -- 172.21.15.150:0/2806714967 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498011830 con 0x7f14a4101090 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14977fe700 1 -- 172.21.15.150:0/2806714967 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1498011e50 con 0x7f14a4101090 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a41a0060 con 0x7f14a4101090 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14977fe700 1 -- 172.21.15.150:0/2806714967 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498010e50 con 0x7f14a4101090 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.284+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a41a05f0 con 0x7f14a4101090 2024-05-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.285+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a4107ef0 con 0x7f14a4101090 2024-05-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.286+0000 7f14977fe700 1 -- 172.21.15.150:0/2806714967 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f14980104f0 con 0x7f14a4101090 2024-05-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.287+0000 7f14977fe700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f14900a11b0 0x7f14900a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.287+0000 7f14a9d0f700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f14900a11b0 0x7f14900a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.287+0000 7f14977fe700 1 -- 172.21.15.150:0/2806714967 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f14980ccc90 con 0x7f14a4101090 2024-05-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.287+0000 7f14a9d0f700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f14900a11b0 0x7f14900a36c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f14a0005950 tx=0x7f14a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.291 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.290+0000 7f14977fe700 1 -- 172.21.15.150:0/2806714967 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1498096060 con 0x7f14a4101090 2024-05-10T04:39:33.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.493+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14a40678b0 con 0x7f14a4101090 2024-05-10T04:39:33.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.494+0000 7f14977fe700 1 -- 172.21.15.150:0/2806714967 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f1498099680 con 0x7f14a4101090 2024-05-10T04:39:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:33.496 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f14900a11b0 msgr2=0x7f14900a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f14900a11b0 0x7f14900a36c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f14a0005950 tx=0x7f14a00058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a4101090 msgr2=0x7f14a4100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a4101090 0x7f14a4100750 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f149800b4d0 tx=0x7f149800b7e0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 shutdown_connections 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41025d0 0x7f14a419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14a4101090 0x7f14a4100750 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f14900a11b0 0x7f14900a36c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 --2- 172.21.15.150:0/2806714967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4101a70 0x7f14a419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 >> 172.21.15.150:0/2806714967 conn(0x7f14a40fb8f0 msgr2=0x7f14a4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 shutdown_connections 2024-05-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.497+0000 7f14ac774700 1 -- 172.21.15.150:0/2806714967 wait complete. 2024-05-10T04:39:33.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-10T04:38:08.750329+0000', 'modified': '2024-05-10T04:38:35.978906+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24548, 'mds_1': 24520, 'mds_2': 24484, 'mds_3': 24455, 'mds_4': 24445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14622': {'gid': 14622, 'name': 'e', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/2306790268', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 2306790268}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 2306790268}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14664': {'gid': 14664, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6839/308347873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 308347873}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 308347873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14676': {'gid': 14676, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6841/3789106275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6840', 'nonce': 3789106275}, {'type': 'v1', 'addr': '172.21.15.150:6841', 'nonce': 3789106275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.181:6833/1504167586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6832', 'nonce': 1504167586}, {'type': 'v1', 'addr': '172.21.15.181:6833', 'nonce': 1504167586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24455': {'gid': 24455, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.161:6833/825127258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 825127258}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 825127258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.181:6835/3090072990', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 3090072990}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 3090072990}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/1032605592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1032605592}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1032605592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/55043330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 55043330}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 55043330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.181:6839/4287682926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 4287682926}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 4287682926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6839/3001288873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 3001288873}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 3001288873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-10T04:39:33.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-10T04:39:33.513 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-10T04:39:33.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 -- 172.21.15.150:0/3457901711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 msgr2=0x7f9054102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 --2- 172.21.15.150:0/3457901711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054102020 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9050009990 tx=0x7f9050009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 -- 172.21.15.150:0/3457901711 shutdown_connections 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 --2- 172.21.15.150:0/3457901711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90541025b0 0x7f9054108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 --2- 172.21.15.150:0/3457901711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9054100af0 0x7f9054100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 --2- 172.21.15.150:0/3457901711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054102020 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 -- 172.21.15.150:0/3457901711 >> 172.21.15.150:0/3457901711 conn(0x7f90540fc100 msgr2=0x7f90540fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 -- 172.21.15.150:0/3457901711 shutdown_connections 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 -- 172.21.15.150:0/3457901711 wait complete. 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 Processor -- start 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.646+0000 7f905b36c700 1 -- start start 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9054100af0 0x7f9054197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90541025b0 0x7f905419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905419f9b0 con 0x7f9054101c10 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905419fb20 con 0x7f9054100af0 2024-05-10T04:39:33.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905419fce0 con 0x7f90541025b0 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40906/0 (socket says 172.21.15.150:40906) 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 -- 172.21.15.150:0/889206614 learned_addr learned my addr 172.21.15.150:0/889206614 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9059108700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9054100af0 0x7f9054197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9059909700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90541025b0 0x7f905419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 -- 172.21.15.150:0/889206614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90541025b0 msgr2=0x7f905419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90541025b0 0x7f905419f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 -- 172.21.15.150:0/889206614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9054100af0 msgr2=0x7f9054197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9054100af0 0x7f9054197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 -- 172.21.15.150:0/889206614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90500095f0 con 0x7f9054101c10 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9059108700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9054100af0 0x7f9054197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f9058907700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054198410 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9050005b40 tx=0x7f905000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f904a7fc700 1 -- 172.21.15.150:0/889206614 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905001d070 con 0x7f9054101c10 2024-05-10T04:39:33.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f904a7fc700 1 -- 172.21.15.150:0/889206614 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905000fc30 con 0x7f9054101c10 2024-05-10T04:39:33.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905419ff60 con 0x7f9054101c10 2024-05-10T04:39:33.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f904a7fc700 1 -- 172.21.15.150:0/889206614 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050017650 con 0x7f9054101c10 2024-05-10T04:39:33.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.647+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90541a02d0 con 0x7f9054101c10 2024-05-10T04:39:33.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.650+0000 7f904a7fc700 1 -- 172.21.15.150:0/889206614 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9050022470 con 0x7f9054101c10 2024-05-10T04:39:33.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.650+0000 7f904a7fc700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90400a1260 0x7f90400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.651+0000 7f9059108700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90400a1260 0x7f90400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.651+0000 7f904a7fc700 1 -- 172.21.15.150:0/889206614 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f90500ce140 con 0x7f9054101c10 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.651+0000 7f9059108700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90400a1260 0x7f90400a3770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9044005950 tx=0x7f90440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 Processor -- start 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 -- start start 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf8650 0x5593cebf8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cebf9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec00770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ceb1f100 con 0x5593cebf8650 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ceb1cac0 con 0x5593cebf9520 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ceb183b0 con 0x5593cebf9e70 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf8650 0x5593cebf8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf8650 0x5593cebf8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40916/0 (socket says 172.21.15.150:40916) 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 -- 172.21.15.150:0/1635851512 learned_addr learned my addr 172.21.15.150:0/1635851512 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059909700 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec00770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9058907700 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cebf9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 -- 172.21.15.150:0/1635851512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 msgr2=0x5593cec00770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec00770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 -- 172.21.15.150:0/1635851512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 msgr2=0x5593cebf9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cebf9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9059108700 1 -- 172.21.15.150:0/1635851512 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593ce999360 con 0x5593cebf8650 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.652+0000 7f9058907700 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cebf9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.653+0000 7f9059108700 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf8650 0x5593cebf8aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f904400fed0 tx=0x7f9044012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1789030a6a299a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.653+0000 7f903f7fe700 1 -- 172.21.15.150:0/1635851512 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044011c60 con 0x5593cebf8650 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.653+0000 7f903f7fe700 1 -- 172.21.15.150:0/1635851512 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9044011dc0 con 0x5593cebf8650 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.653+0000 7f903f7fe700 1 -- 172.21.15.150:0/1635851512 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904401d6c0 con 0x5593cebf8650 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 -- 172.21.15.150:0/1635851512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf8650 msgr2=0x5593cebf8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf8650 0x5593cebf8aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f904400fed0 tx=0x7f9044012000 comp rx=0 tx=0).stop 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 -- 172.21.15.150:0/1635851512 shutdown_connections 2024-05-10T04:39:33.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec00770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf8650 0x5593cebf8aa0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/1635851512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cebf9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 -- 172.21.15.150:0/1635851512 >> 172.21.15.150:0/1635851512 conn(0x5593ceb5aa10 msgr2=0x5593cebf2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 -- 172.21.15.150:0/1635851512 shutdown_connections 2024-05-10T04:39:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 -- 172.21.15.150:0/1635851512 wait complete. 2024-05-10T04:39:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 -- start start 2024-05-10T04:39:33.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.654+0000 7f906b6b0b80 1 Processor -- start 2024-05-10T04:39:33.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec02680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9520 0x5593cec02bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec09b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ceab4510 con 0x5593cebf9520 2024-05-10T04:39:33.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ceab41b0 con 0x5593cebf8650 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ce99b650 con 0x5593cebf9e70 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9520 0x5593cec02bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 10 client.0 _ll_register_callbacks cb 0x5593ceabdae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9520 0x5593cec02bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40920/0 (socket says 172.21.15.150:40920) 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 -- 172.21.15.150:0/2217034085 learned_addr learned my addr 172.21.15.150:0/2217034085 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9059108700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec02680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9059909700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec09b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 -- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 msgr2=0x5593cec09b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec09b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 -- 172.21.15.150:0/2217034085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 msgr2=0x5593cec02680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec02680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 -- 172.21.15.150:0/2217034085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904400fbb0 con 0x5593cebf9520 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9059909700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec09b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f9058907700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9520 0x5593cec02bc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9050097c20 tx=0x7f905009b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0f91a0e007d4b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f903cff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f903cff9700 1 -- 172.21.15.150:0/2217034085 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905009b1b0 con 0x5593cebf9520 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f903cff9700 1 -- 172.21.15.150:0/2217034085 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f905009efe0 con 0x5593cebf9520 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.655+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593cea71220 con 0x5593cebf9520 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f903cff9700 1 -- 172.21.15.150:0/2217034085 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90500a35a0 con 0x5593cebf9520 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 10 client.15471 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5593ce945b10 con 0x5593cebf9520 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f903cff9700 1 -- 172.21.15.150:0/2217034085 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593cea71220 con 0x5593cebf9520 2024-05-10T04:39:33.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f903cff9700 1 -- 172.21.15.150:0/2217034085 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9050022720 con 0x5593cebf9520 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 10 client.15471 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 20 client.15471 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec0d890 0x5593cec125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 4 client.15471 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593cebc7730 con 0x5593cec0d890 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.656+0000 7f9059108700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec0d890 0x5593cec125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.657+0000 7f903cff9700 10 client.15471 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.657+0000 7f9059108700 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec0d890 0x5593cec125a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db0af9e1ef0e5280 server_cookie=c1847dc3bcaabc00 in_seq=0 out_seq=0 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.657+0000 7f903cff9700 1 -- 172.21.15.150:0/2217034085 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593cebc7730 con 0x5593cec0d890 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.657+0000 7f903cff9700 10 client.15471 handle_command_reply: tid=0 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.657+0000 7f906b6b0b80 1 client.15471 shutdown 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9520 msgr2=0x5593cec02bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9520 0x5593cec02bc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9050097c20 tx=0x7f905009b6f0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 shutdown_connections 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec0d890 0x5593cec125a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9e70 0x5593cec09b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f903cff9700 0 client.15471 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec02680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/2217034085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9520 0x5593cec02bc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 >> 172.21.15.150:0/2217034085 conn(0x5593ceb5aa10 msgr2=0x5593cebf2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 shutdown_connections 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 1 -- 172.21.15.150:0/2217034085 wait complete. 2024-05-10T04:39:33.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.658+0000 7f906b6b0b80 20 client.15471 trim_cache size 0 max 16384 2024-05-10T04:39:33.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 Processor -- start 2024-05-10T04:39:33.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 -- start start 2024-05-10T04:39:33.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9520 0x5593cec07ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec08010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ceb15010 con 0x5593cebf9e70 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ceab75a0 con 0x5593cebf8650 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cebefce0 con 0x5593cebf9520 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f9059909700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec08010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f9059909700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec08010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40922/0 (socket says 172.21.15.150:40922) 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f9059909700 1 -- 172.21.15.150:0/3980633132 learned_addr learned my addr 172.21.15.150:0/3980633132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f9059108700 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.676+0000 7f9058907700 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9520 0x5593cec07ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f9059909700 1 -- 172.21.15.150:0/3980633132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9520 msgr2=0x5593cec07ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f9059909700 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9520 0x5593cec07ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f9059909700 1 -- 172.21.15.150:0/3980633132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 msgr2=0x5593cec010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f9059909700 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f9059909700 1 -- 172.21.15.150:0/3980633132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593ceade510 con 0x5593cebf9e70 2024-05-10T04:39:33.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f9059909700 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec08010 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f904c009b10 tx=0x7f904c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9208eb1146e048e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f903ffff700 1 -- 172.21.15.150:0/3980633132 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0079f0 con 0x5593cebf9e70 2024-05-10T04:39:33.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f903ffff700 1 -- 172.21.15.150:0/3980633132 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904c007b50 con 0x5593cebf9e70 2024-05-10T04:39:33.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.677+0000 7f903ffff700 1 -- 172.21.15.150:0/3980633132 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0155c0 con 0x5593cebf9e70 2024-05-10T04:39:33.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- 172.21.15.150:0/3980633132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 msgr2=0x5593cec08010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec08010 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f904c009b10 tx=0x7f904c009040 comp rx=0 tx=0).stop 2024-05-10T04:39:33.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- 172.21.15.150:0/3980633132 shutdown_connections 2024-05-10T04:39:33.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec08010 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf8650 0x5593cec010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/3980633132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf9520 0x5593cec07ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- 172.21.15.150:0/3980633132 >> 172.21.15.150:0/3980633132 conn(0x5593ceb5aa10 msgr2=0x5593cec009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- 172.21.15.150:0/3980633132 shutdown_connections 2024-05-10T04:39:33.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- 172.21.15.150:0/3980633132 wait complete. 2024-05-10T04:39:33.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- start start 2024-05-10T04:39:33.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 Processor -- start 2024-05-10T04:39:33.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf8650 0x5593cec06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cec06dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec07300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cec64260 con 0x5593cebf9e70 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cec643a0 con 0x5593cebf9520 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cec644e0 con 0x5593cebf8650 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.678+0000 7f906b6b0b80 10 client.0 _ll_register_callbacks cb 0x5593ceabdae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec07300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec07300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40926/0 (socket says 172.21.15.150:40926) 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9058907700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cec06dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 -- 172.21.15.150:0/4152000046 learned_addr learned my addr 172.21.15.150:0/4152000046 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059108700 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf8650 0x5593cec06880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 -- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf8650 msgr2=0x5593cec06880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf8650 0x5593cec06880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 -- 172.21.15.150:0/4152000046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 msgr2=0x5593cec06dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cec06dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 -- 172.21.15.150:0/4152000046 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c011040 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f9059909700 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec07300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f904c0098b0 tx=0x7f904c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9f4fd415ea44bf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f903cff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f903cff9700 1 -- 172.21.15.150:0/4152000046 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c016600 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9044011dc0 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f903cff9700 1 -- 172.21.15.150:0/4152000046 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f904c016760 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f903cff9700 1 -- 172.21.15.150:0/4152000046 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f904c02a440 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f906b6b0b80 10 client.15477 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.679+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5593ce99dc80 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f903cff9700 1 -- 172.21.15.150:0/4152000046 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904c033020 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f903cff9700 1 -- 172.21.15.150:0/4152000046 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f904c005280 con 0x5593cebf9e70 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f906b6b0b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f906b6b0b80 10 client.15477 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-10T04:39:33.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f906b6b0b80 20 client.15477 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec675e0 0x5593cec65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f906b6b0b80 4 client.15477 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5593ceb10e70 con 0x5593cec675e0 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f9059108700 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec675e0 0x5593cec65b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f903cff9700 10 client.15477 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.680+0000 7f9059108700 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec675e0 0x5593cec65b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5656e4a56c202780 server_cookie=f7ac4044bcb1b322 in_seq=0 out_seq=0 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f903cff9700 1 -- 172.21.15.150:0/4152000046 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x5593ceb10e70 con 0x5593cec675e0 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f903cff9700 10 client.15477 handle_command_reply: tid=0 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 client.15477 shutdown 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 msgr2=0x5593cec07300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec07300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f904c0098b0 tx=0x7f904c004b50 comp rx=0 tx=0).stop 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 shutdown_connections 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5593cec675e0 0x5593cec65b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593cebf9e70 0x5593cec07300 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5593cebf8650 0x5593cec06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 --2- 172.21.15.150:0/4152000046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5593cebf9520 0x5593cec06dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f903cff9700 0 client.15477 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 >> 172.21.15.150:0/4152000046 conn(0x5593ceb5aa10 msgr2=0x5593cec165c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 shutdown_connections 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 1 -- 172.21.15.150:0/4152000046 wait complete. 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f906b6b0b80 20 client.15477 trim_cache size 0 max 16384 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 4, 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24445, 2024-05-10T04:39:33.689 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 58.723226253, 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 110.08726983 2024-05-10T04:39:33.690 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.681+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90400a1260 msgr2=0x7f90400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90400a1260 0x7f90400a3770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9044005950 tx=0x7f90440058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 msgr2=0x7f9054198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054198410 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9050005b40 tx=0x7f905000f740 comp rx=0 tx=0).stop 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 shutdown_connections 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90400a1260 0x7f90400a3770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90541025b0 0x7f905419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9054100af0 0x7f9054197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 --2- 172.21.15.150:0/889206614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9054101c10 0x7f9054198410 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 >> 172.21.15.150:0/889206614 conn(0x7f90540fc100 msgr2=0x7f9054104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 shutdown_connections 2024-05-10T04:39:33.692 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.682+0000 7f905b36c700 1 -- 172.21.15.150:0/889206614 wait complete. 2024-05-10T04:39:33.694 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/690641177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:33.694 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1170627333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:33.695 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:33 smithi150 ceph-mon[42404]: pgmap v269: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T04:39:33.696 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-05-10T04:39:33.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 -- 172.21.15.150:0/271218710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101a70 msgr2=0x7f3d08101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/271218710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101a70 0x7f3d08101f40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009990 tx=0x7f3cf8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 -- 172.21.15.150:0/271218710 shutdown_connections 2024-05-10T04:39:33.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/271218710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d081025d0 0x7f3d08109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/271218710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d08101090 0x7f3d081014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/271218710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101a70 0x7f3d08101f40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 -- 172.21.15.150:0/271218710 >> 172.21.15.150:0/271218710 conn(0x7f3d080fb8f0 msgr2=0x7f3d080fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 -- 172.21.15.150:0/271218710 shutdown_connections 2024-05-10T04:39:33.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.827+0000 7f3d0cef4700 1 -- 172.21.15.150:0/271218710 wait complete. 2024-05-10T04:39:33.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 Processor -- start 2024-05-10T04:39:33.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 -- start start 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101090 0x7f3d081007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d08101a70 0x7f3d0819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d081025d0 0x7f3d0819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0819f9c0 con 0x7f3d08101090 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0819fb30 con 0x7f3d08101a70 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.828+0000 7f3d0cef4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0819fcf0 con 0x7f3d081025d0 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101090 0x7f3d081007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101090 0x7f3d081007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40946/0 (socket says 172.21.15.150:40946) 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 -- 172.21.15.150:0/1145725276 learned_addr learned my addr 172.21.15.150:0/1145725276 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d08101a70 0x7f3d0819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d081025d0 0x7f3d0819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 -- 172.21.15.150:0/1145725276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d081025d0 msgr2=0x7f3d0819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d081025d0 0x7f3d0819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 -- 172.21.15.150:0/1145725276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d08101a70 msgr2=0x7f3d0819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d08101a70 0x7f3d0819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 -- 172.21.15.150:0/1145725276 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80095f0 con 0x7f3d08101090 2024-05-10T04:39:33.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0659c700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101090 0x7f3d081007b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00d950 tx=0x7f3cfc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3cf77fe700 1 -- 172.21.15.150:0/1145725276 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00f890 con 0x7f3d08101090 2024-05-10T04:39:33.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3cf77fe700 1 -- 172.21.15.150:0/1145725276 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cfc010460 con 0x7f3d08101090 2024-05-10T04:39:33.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0819ffd0 con 0x7f3d08101090 2024-05-10T04:39:33.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3cf77fe700 1 -- 172.21.15.150:0/1145725276 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00e610 con 0x7f3d08101090 2024-05-10T04:39:33.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.829+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d081a0400 con 0x7f3d08101090 2024-05-10T04:39:33.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/690641177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:33.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1170627333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:33.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:33 smithi161 ceph-mon[46912]: pgmap v269: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.832+0000 7f3cf77fe700 1 -- 172.21.15.150:0/1145725276 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3cfc010860 con 0x7f3d08101090 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.832+0000 7f3cf77fe700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cf00a1260 0x7f3cf00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.832+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cf00a1260 0x7f3cf00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.832+0000 7f3cf77fe700 1 -- 172.21.15.150:0/1145725276 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3cfc0cc810 con 0x7f3d08101090 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cf00a1260 0x7f3cf00a3770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8003730 tx=0x7f3cf801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 Processor -- start 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 -- start start 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb0eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb0f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb16770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9ca35100 con 0x561b9cb0fe70 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9ca32ac0 con 0x561b9cb0f520 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.833+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9ca2e3b0 con 0x561b9cb0e650 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb16770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb16770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40952/0 (socket says 172.21.15.150:40952) 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 -- 172.21.15.150:0/1525967460 learned_addr learned my addr 172.21.15.150:0/1525967460 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb0f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d0659c700 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb0eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 -- 172.21.15.150:0/1525967460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 msgr2=0x561b9cb0eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb0eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 -- 172.21.15.150:0/1525967460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 msgr2=0x561b9cb0f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb0f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 -- 172.21.15.150:0/1525967460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b9c8af360 con 0x561b9cb0fe70 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb0f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d0659c700 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb0eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:33.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb16770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d00003f80 tx=0x7f3d00009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132fb3fe74e59866 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3cf4ff9700 1 -- 172.21.15.150:0/1525967460 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00019070 con 0x561b9cb0fe70 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3cf4ff9700 1 -- 172.21.15.150:0/1525967460 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d0000f040 con 0x561b9cb0fe70 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.834+0000 7f3cf4ff9700 1 -- 172.21.15.150:0/1525967460 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00007a70 con 0x561b9cb0fe70 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1525967460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 msgr2=0x561b9cb16770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb16770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d00003f80 tx=0x7f3d00009ba0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1525967460 shutdown_connections 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb16770 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb0eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1525967460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb0f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1525967460 >> 172.21.15.150:0/1525967460 conn(0x561b9ca70a10 msgr2=0x561b9cb08f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1525967460 shutdown_connections 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.835+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1525967460 wait complete. 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 -- start start 2024-05-10T04:39:33.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 Processor -- start 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb18b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb1faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9c9ca510 con 0x561b9cb0fe70 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9c9ca1b0 con 0x561b9cb0f520 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9c8b1650 con 0x561b9cb0e650 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d1d238b80 10 client.0 _ll_register_callbacks cb 0x561b9c9d3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb1faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb1faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40966/0 (socket says 172.21.15.150:40966) 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 -- 172.21.15.150:0/965438416 learned_addr learned my addr 172.21.15.150:0/965438416 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d0659c700 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb18b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 -- 172.21.15.150:0/965438416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 msgr2=0x561b9cb18650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.846 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb18650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 -- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 msgr2=0x561b9cb18b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb18b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.836+0000 7f3d06d9d700 1 -- 172.21.15.150:0/965438416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d00003bb0 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb1faf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004010 tx=0x7f3d000048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb93b9bd70dd746 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3cedffb700 1 -- 172.21.15.150:0/965438416 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00019040 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3cedffb700 1 -- 172.21.15.150:0/965438416 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d00007d40 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b9c987220 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3cedffb700 1 -- 172.21.15.150:0/965438416 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d00015480 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 10 client.15489 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561b9c85bb10 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3cedffb700 1 -- 172.21.15.150:0/965438416 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b9c987220 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3cedffb700 1 -- 172.21.15.150:0/965438416 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3d0001e710 con 0x561b9cb0fe70 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:33.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 10 client.15489 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 20 client.15489 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb23bd0 0x561b9cb28900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 4 client.15489 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.837+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b9cadd730 con 0x561b9cb23bd0 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.838+0000 7f3d0659c700 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb23bd0 0x561b9cb28900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.838+0000 7f3cedffb700 10 client.15489 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.838+0000 7f3d0659c700 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb23bd0 0x561b9cb28900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e04ea2b11420ea6 server_cookie=c3c469e118f81756 in_seq=0 out_seq=0 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3cedffb700 1 -- 172.21.15.150:0/965438416 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b9cadd730 con 0x561b9cb23bd0 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3cedffb700 10 client.15489 handle_command_reply: tid=0 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 client.15489 shutdown 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 msgr2=0x561b9cb1faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb1faf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004010 tx=0x7f3d000048c0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 shutdown_connections 2024-05-10T04:39:33.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb23bd0 0x561b9cb28900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0fe70 0x561b9cb1faf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb18650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3cedffb700 0 client.15489 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/965438416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0f520 0x561b9cb18b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 >> 172.21.15.150:0/965438416 conn(0x561b9ca70a10 msgr2=0x561b9cb08f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 shutdown_connections 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 1 -- 172.21.15.150:0/965438416 wait complete. 2024-05-10T04:39:33.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.839+0000 7f3d1d238b80 20 client.15489 trim_cache size 0 max 16384 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.857+0000 7f3d1d238b80 1 Processor -- start 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.857+0000 7f3d1d238b80 1 -- start start 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.857+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.857+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.857+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0fe70 0x561b9cb16cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.857+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9c9ec2b0 con 0x561b9cb0f520 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9ca27150 con 0x561b9cb0fe70 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9ca2b010 con 0x561b9cb0e650 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40968/0 (socket says 172.21.15.150:40968) 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 -- 172.21.15.150:0/1198882861 learned_addr learned my addr 172.21.15.150:0/1198882861 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d0659c700 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0fe70 0x561b9cb16cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 -- 172.21.15.150:0/1198882861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 msgr2=0x561b9cb2a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb2a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 -- 172.21.15.150:0/1198882861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0fe70 msgr2=0x561b9cb16cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0fe70 0x561b9cb16cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 -- 172.21.15.150:0/1198882861 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b9c9f4510 con 0x561b9cb0f520 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d0659c700 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb2a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0fe70 0x561b9cb16cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8015000 tx=0x7f3cf8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e388a82cb73788 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3cf57fa700 1 -- 172.21.15.150:0/1198882861 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8024aa0 con 0x561b9cb0f520 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3cf57fa700 1 -- 172.21.15.150:0/1198882861 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cf802c070 con 0x561b9cb0f520 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.858+0000 7f3cf57fa700 1 -- 172.21.15.150:0/1198882861 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800fe80 con 0x561b9cb0f520 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1198882861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 msgr2=0x561b9cb2ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8015000 tx=0x7f3cf8022000 comp rx=0 tx=0).stop 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1198882861 shutdown_connections 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0fe70 0x561b9cb16cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0e650 0x561b9cb2a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/1198882861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1198882861 >> 172.21.15.150:0/1198882861 conn(0x561b9ca70a10 msgr2=0x561b9cb05130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1198882861 shutdown_connections 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 -- 172.21.15.150:0/1198882861 wait complete. 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 -- start start 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.859+0000 7f3d1d238b80 1 Processor -- start 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0e650 0x561b9cb2a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0fe70 0x561b9cb290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9cb29750 con 0x561b9cb0f520 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9cb298c0 con 0x561b9cb0e650 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9cb29a80 con 0x561b9cb0fe70 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 10 client.0 _ll_register_callbacks cb 0x561b9c9d3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:33.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40970/0 (socket says 172.21.15.150:40970) 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 -- 172.21.15.150:0/366727898 learned_addr learned my addr 172.21.15.150:0/366727898 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d0659c700 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0e650 0x561b9cb2a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d06d9d700 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0fe70 0x561b9cb290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 -- 172.21.15.150:0/366727898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0fe70 msgr2=0x561b9cb290e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0fe70 0x561b9cb290e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 -- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0e650 msgr2=0x561b9cb2a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0e650 0x561b9cb2a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 -- 172.21.15.150:0/366727898 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d08109940 con 0x561b9cb0f520 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d05d9b700 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80246f0 tx=0x7f3cf8024cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c5a9bc8700d9b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:33.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3cedffb700 1 -- 172.21.15.150:0/366727898 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf803b480 con 0x561b9cb0f520 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3cedffb700 1 -- 172.21.15.150:0/366727898 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3cf803b5e0 con 0x561b9cb0f520 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.860+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d0000f040 con 0x561b9cb0f520 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3cedffb700 1 -- 172.21.15.150:0/366727898 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cf803b7a0 con 0x561b9cb0f520 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 10 client.15495 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561b9c8b3c80 con 0x561b9cb0f520 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3cedffb700 1 -- 172.21.15.150:0/366727898 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d0000f040 con 0x561b9cb0f520 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3cedffb700 1 -- 172.21.15.150:0/366727898 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3cf8032970 con 0x561b9cb0f520 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 10 client.15495 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 20 client.15495 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb7cca0 0x561b9cb7b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 4 client.15495 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:33.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561b9ca26e70 con 0x561b9cb7cca0 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.861+0000 7f3d0659c700 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb7cca0 0x561b9cb7b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3cedffb700 10 client.15495 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d0659c700 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb7cca0 0x561b9cb7b250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95723b83277ba1b4 server_cookie=2ffa868d914601ea in_seq=0 out_seq=0 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3cedffb700 1 -- 172.21.15.150:0/366727898 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x561b9ca26e70 con 0x561b9cb7cca0 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3cedffb700 10 client.15495 handle_command_reply: tid=0 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 client.15495 shutdown 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 msgr2=0x561b9cb2ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80246f0 tx=0x7f3cf8024cf0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 shutdown_connections 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x561b9cb7cca0 0x561b9cb7b250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561b9cb0fe70 0x561b9cb290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561b9cb0e650 0x561b9cb2a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3cedffb700 0 client.15495 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 --2- 172.21.15.150:0/366727898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561b9cb0f520 0x561b9cb2ae00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 >> 172.21.15.150:0/366727898 conn(0x561b9ca70a10 msgr2=0x561b9cb05130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 shutdown_connections 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 1 -- 172.21.15.150:0/366727898 wait complete. 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.862+0000 7f3d1d238b80 20 client.15495 trim_cache size 0 max 16384 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 3, 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24455, 2024-05-10T04:39:33.876 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 59.90906536, 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 108.590278996 2024-05-10T04:39:33.877 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cf00a1260 msgr2=0x7f3cf00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cf00a1260 0x7f3cf00a3770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8003730 tx=0x7f3cf801a040 comp rx=0 tx=0).stop 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101090 msgr2=0x7f3d081007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101090 0x7f3d081007b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00d950 tx=0x7f3cfc00dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 shutdown_connections 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cf00a1260 0x7f3cf00a3770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d081025d0 0x7f3d0819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08101090 0x7f3d081007b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 --2- 172.21.15.150:0/1145725276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d08101a70 0x7f3d0819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 >> 172.21.15.150:0/1145725276 conn(0x7f3d080fb8f0 msgr2=0x7f3d08105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 shutdown_connections 2024-05-10T04:39:33.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:33.863+0000 7f3d0cef4700 1 -- 172.21.15.150:0/1145725276 wait complete. 2024-05-10T04:39:33.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-05-10T04:39:33.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/690641177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:33.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1170627333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:33.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:33 smithi181 ceph-mon[47314]: pgmap v269: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-10T04:39:34.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2874473216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81025d0 msgr2=0x7fe6c8109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2874473216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81025d0 0x7fe6c8109090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b0a0 tx=0x7fe6c000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2874473216 shutdown_connections 2024-05-10T04:39:34.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2874473216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81025d0 0x7fe6c8109090 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2874473216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6c81010d0 0x7fe6c81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2874473216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c8101ab0 0x7fe6c8101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2874473216 >> 172.21.15.150:0/2874473216 conn(0x7fe6c80fb8f0 msgr2=0x7fe6c80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2874473216 shutdown_connections 2024-05-10T04:39:34.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.013+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2874473216 wait complete. 2024-05-10T04:39:34.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 Processor -- start 2024-05-10T04:39:34.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 -- start start 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81010d0 0x7fe6c8196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6c8101ab0 0x7fe6c81974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c81025d0 0x7fe6c819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c819ea60 con 0x7fe6c81010d0 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c819ebd0 con 0x7fe6c8101ab0 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cf53c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c819ed90 con 0x7fe6c81025d0 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cd2d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81010d0 0x7fe6c8196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cd2d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81010d0 0x7fe6c8196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40982/0 (socket says 172.21.15.150:40982) 2024-05-10T04:39:34.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.014+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/2635898929 learned_addr learned my addr 172.21.15.150:0/2635898929 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6c8101ab0 0x7fe6c81974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/2635898929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c81025d0 msgr2=0x7fe6c819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c81025d0 0x7fe6c819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/2635898929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6c8101ab0 msgr2=0x7fe6c81974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6c8101ab0 0x7fe6c81974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/2635898929 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c0009d00 con 0x7fe6c81010d0 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81010d0 0x7fe6c8196f80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e940 tx=0x7fe6b800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6be7fc700 1 -- 172.21.15.150:0/2635898929 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b800cbd0 con 0x7fe6c81010d0 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6be7fc700 1 -- 172.21.15.150:0/2635898929 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b80044e0 con 0x7fe6c81010d0 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c819f070 con 0x7fe6c81010d0 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6be7fc700 1 -- 172.21.15.150:0/2635898929 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8003dc0 con 0x7fe6c81010d0 2024-05-10T04:39:34.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.015+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c819f5d0 con 0x7fe6c81010d0 2024-05-10T04:39:34.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.017+0000 7fe6be7fc700 1 -- 172.21.15.150:0/2635898929 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe6b8003f20 con 0x7fe6c81010d0 2024-05-10T04:39:34.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.018+0000 7fe6be7fc700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6b40a12e0 0x7fe6b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.018+0000 7fe6be7fc700 1 -- 172.21.15.150:0/2635898929 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6b8014070 con 0x7fe6c81010d0 2024-05-10T04:39:34.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.018+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6b40a12e0 0x7fe6b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.018+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6b40a12e0 0x7fe6b40a37f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4005950 tx=0x7fe6c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 Processor -- start 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 -- start start 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb71860 0x56073eb71cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb72730 0x56073eb72b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb73080 0x56073eb79980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea96100 con 0x56073eb73080 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea93ac0 con 0x56073eb71860 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6df880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea8f3b0 con 0x56073eb72730 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cdad9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb73080 0x56073eb79980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cdad9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb73080 0x56073eb79980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40992/0 (socket says 172.21.15.150:40992) 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cdad9700 1 -- 172.21.15.150:0/1992305475 learned_addr learned my addr 172.21.15.150:0/1992305475 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb72730 0x56073eb72b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb71860 0x56073eb71cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cdad9700 1 -- 172.21.15.150:0/1992305475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb72730 msgr2=0x56073eb72b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb72730 0x56073eb72b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cdad9700 1 -- 172.21.15.150:0/1992305475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb71860 msgr2=0x56073eb71cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.019+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb71860 0x56073eb71cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.020+0000 7fe6cdad9700 1 -- 172.21.15.150:0/1992305475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56073e910360 con 0x56073eb73080 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.020+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb72730 0x56073eb72b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.020+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb73080 0x56073eb79980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b780 tx=0x7fe6c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14642de628724d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.020+0000 7fe6b37fe700 1 -- 172.21.15.150:0/1992305475 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000e070 con 0x56073eb73080 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.020+0000 7fe6b37fe700 1 -- 172.21.15.150:0/1992305475 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c00075c0 con 0x56073eb73080 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.020+0000 7fe6b37fe700 1 -- 172.21.15.150:0/1992305475 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c001b6b0 con 0x56073eb73080 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 -- 172.21.15.150:0/1992305475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb73080 msgr2=0x56073eb79980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb73080 0x56073eb79980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000b780 tx=0x7fe6c0009780 comp rx=0 tx=0).stop 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 -- 172.21.15.150:0/1992305475 shutdown_connections 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb73080 0x56073eb79980 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb71860 0x56073eb71cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1992305475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb72730 0x56073eb72b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 -- 172.21.15.150:0/1992305475 >> 172.21.15.150:0/1992305475 conn(0x56073ead1a10 msgr2=0x56073eb69f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 -- 172.21.15.150:0/1992305475 shutdown_connections 2024-05-10T04:39:34.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 -- 172.21.15.150:0/1992305475 wait complete. 2024-05-10T04:39:34.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 -- start start 2024-05-10T04:39:34.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.021+0000 7fe6df880b80 1 Processor -- start 2024-05-10T04:39:34.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb7b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb7bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb82be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6df880b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea2b510 con 0x56073eb72730 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6df880b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea2b1b0 con 0x56073eb73080 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6df880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073e912650 con 0x56073eb71860 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb7bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb7bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40996/0 (socket says 172.21.15.150:40996) 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6df880b80 10 client.0 _ll_register_callbacks cb 0x56073ea34ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 -- 172.21.15.150:0/2178029207 learned_addr learned my addr 172.21.15.150:0/2178029207 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb7b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb82be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 -- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 msgr2=0x56073eb7b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb7b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 -- 172.21.15.150:0/2178029207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 msgr2=0x56073eb82be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb82be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.022+0000 7fe6ccad7700 1 -- 172.21.15.150:0/2178029207 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c8109950 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb7bc80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4011440 tx=0x7fe6c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec158cc5455b868 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/2178029207 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4011ea0 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/2178029207 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6c4004a30 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56073e9e8220 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/2178029207 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c4004bf0 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 10 client.15507 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56073e8bcb10 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/2178029207 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56073e9e8220 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/2178029207 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6c4025030 con 0x56073eb72730 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 10 client.15507 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 20 client.15507 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:34.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073eb85d60 0x56073eb8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 4 client.15507 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.023+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56073eb3e730 con 0x56073eb85d60 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.024+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073eb85d60 0x56073eb8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.024+0000 7fe6b0ff9700 10 client.15507 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.024+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073eb85d60 0x56073eb8bbb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f573c8cb334098f server_cookie=3343cf5c04b6bc7a in_seq=0 out_seq=0 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/2178029207 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56073eb3e730 con 0x56073eb85d60 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6b0ff9700 10 client.15507 handle_command_reply: tid=0 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 client.15507 shutdown 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 msgr2=0x56073eb7bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb7bc80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4011440 tx=0x7fe6c4004750 comp rx=0 tx=0).stop 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 shutdown_connections 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073eb85d60 0x56073eb8bbb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb82be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb7b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 --2- 172.21.15.150:0/2178029207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb7bc80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6b0ff9700 0 client.15507 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:34.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 >> 172.21.15.150:0/2178029207 conn(0x56073ead1a10 msgr2=0x56073eb69f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 shutdown_connections 2024-05-10T04:39:34.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 1 -- 172.21.15.150:0/2178029207 wait complete. 2024-05-10T04:39:34.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.025+0000 7fe6df880b80 20 client.15507 trim_cache size 0 max 16384 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.043+0000 7fe6df880b80 1 Processor -- start 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.043+0000 7fe6df880b80 1 -- start start 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.043+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb75960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.043+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb71ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb80b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6df880b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea4d2b0 con 0x56073eb72730 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6df880b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea88150 con 0x56073eb73080 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6df880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073ea2e5a0 con 0x56073eb71860 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb71ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb71ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41002/0 (socket says 172.21.15.150:41002) 2024-05-10T04:39:34.047 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 -- 172.21.15.150:0/941911572 learned_addr learned my addr 172.21.15.150:0/941911572 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb75960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb80b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 -- 172.21.15.150:0/941911572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 msgr2=0x56073eb75960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb75960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 -- 172.21.15.150:0/941911572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 msgr2=0x56073eb80b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb80b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 -- 172.21.15.150:0/941911572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56073ea55510 con 0x56073eb72730 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb80b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:34.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb71ec0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4010010 tx=0x7fe6c4011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2001545bee8dd913 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6b3fff700 1 -- 172.21.15.150:0/941911572 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c401fb80 con 0x56073eb72730 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6b3fff700 1 -- 172.21.15.150:0/941911572 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c401f4f0 con 0x56073eb72730 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.044+0000 7fe6b3fff700 1 -- 172.21.15.150:0/941911572 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4005450 con 0x56073eb72730 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 -- 172.21.15.150:0/941911572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 msgr2=0x56073eb71ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb71ec0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4010010 tx=0x7fe6c4011ea0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 -- 172.21.15.150:0/941911572 shutdown_connections 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb73080 0x56073eb80b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb71860 0x56073eb75960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 --2- 172.21.15.150:0/941911572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb72730 0x56073eb71ec0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 -- 172.21.15.150:0/941911572 >> 172.21.15.150:0/941911572 conn(0x56073ead1a10 msgr2=0x56073eb6ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 -- 172.21.15.150:0/941911572 shutdown_connections 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 -- 172.21.15.150:0/941911572 wait complete. 2024-05-10T04:39:34.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.045+0000 7fe6df880b80 1 -- start start 2024-05-10T04:39:34.050 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 1 Processor -- start 2024-05-10T04:39:34.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb71860 0x56073eb7ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb72730 0x56073eb7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb73080 0x56073eb80170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073eb806b0 con 0x56073eb71860 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073eb80820 con 0x56073eb72730 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073eb809e0 con 0x56073eb73080 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6df880b80 10 client.0 _ll_register_callbacks cb 0x56073ea34ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6cd2d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb71860 0x56073eb7ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6cd2d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb71860 0x56073eb7ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41008/0 (socket says 172.21.15.150:41008) 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/1942162952 learned_addr learned my addr 172.21.15.150:0/1942162952 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb72730 0x56073eb7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb73080 msgr2=0x56073eb80170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.046+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb73080 0x56073eb80170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb73080 0x56073eb80170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/1942162952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb72730 msgr2=0x56073eb7f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb72730 0x56073eb7f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6cd2d8700 1 -- 172.21.15.150:0/1942162952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56073e8bcb10 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb72730 0x56073eb7f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6cdad9700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb73080 0x56073eb80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6cd2d8700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb71860 0x56073eb7ed00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80ca740 tx=0x7fe6b800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72893f1556f5fa72 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/1942162952 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b809de80 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/1942162952 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6b80a24d0 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6c00075c0 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/1942162952 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b801fdc0 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 10 client.15513 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56073e914c80 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/1942162952 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c00075c0 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/1942162952 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6b8096760 con 0x56073eb71860 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:34.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 10 client.15513 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 20 client.15513 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073ebdfc70 0x56073ebde200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 4 client.15513 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.047+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56073ea87e70 con 0x56073ebdfc70 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073ebdfc70 0x56073ebde200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6b0ff9700 10 client.15513 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6ccad7700 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073ebdfc70 0x56073ebde200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=103a70c498997899 server_cookie=4604ad85222685dd in_seq=0 out_seq=0 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6b0ff9700 1 -- 172.21.15.150:0/1942162952 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56073ea87e70 con 0x56073ebdfc70 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6b0ff9700 10 client.15513 handle_command_reply: tid=0 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 client.15513 shutdown 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb71860 msgr2=0x56073eb7ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb71860 0x56073eb7ed00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80ca740 tx=0x7fe6b800cee0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 shutdown_connections 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x56073ebdfc70 0x56073ebde200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56073eb73080 0x56073eb80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56073eb71860 0x56073eb7ed00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 --2- 172.21.15.150:0/1942162952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56073eb72730 0x56073eb7f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6b0ff9700 0 client.15513 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.048+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 >> 172.21.15.150:0/1942162952 conn(0x56073ead1a10 msgr2=0x56073eb6aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 shutdown_connections 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6df880b80 1 -- 172.21.15.150:0/1942162952 wait complete. 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6df880b80 20 client.15513 trim_cache size 0 max 16384 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 2, 2024-05-10T04:39:34.057 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24484, 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 61.108975874000002, 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 101.88216957 2024-05-10T04:39:34.058 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6b40a12e0 msgr2=0x7fe6b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6b40a12e0 0x7fe6b40a37f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4005950 tx=0x7fe6c40058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81010d0 msgr2=0x7fe6c8196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81010d0 0x7fe6c8196f80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e940 tx=0x7fe6b800ed00 comp rx=0 tx=0).stop 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 shutdown_connections 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6b40a12e0 0x7fe6b40a37f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c81025d0 0x7fe6c819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6c81010d0 0x7fe6c8196f80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 --2- 172.21.15.150:0/2635898929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6c8101ab0 0x7fe6c81974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 >> 172.21.15.150:0/2635898929 conn(0x7fe6c80fb8f0 msgr2=0x7fe6c8105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 shutdown_connections 2024-05-10T04:39:34.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.049+0000 7fe6cf53c700 1 -- 172.21.15.150:0/2635898929 wait complete. 2024-05-10T04:39:34.063 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-10T04:39:34.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 -- 172.21.15.150:0/4273912633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac01025a0 msgr2=0x7f4ac0108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 --2- 172.21.15.150:0/4273912633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac01025a0 0x7f4ac0108e50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b0a0 tx=0x7f4ab800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 -- 172.21.15.150:0/4273912633 shutdown_connections 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 --2- 172.21.15.150:0/4273912633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac01025a0 0x7f4ac0108e50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 --2- 172.21.15.150:0/4273912633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac0100ae0 0x7f4ac0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 --2- 172.21.15.150:0/4273912633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0101c00 0x7f4ac0102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 -- 172.21.15.150:0/4273912633 >> 172.21.15.150:0/4273912633 conn(0x7f4ac00fc0f0 msgr2=0x7f4ac00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 -- 172.21.15.150:0/4273912633 shutdown_connections 2024-05-10T04:39:34.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.196+0000 7f4ac502d700 1 -- 172.21.15.150:0/4273912633 wait complete. 2024-05-10T04:39:34.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 Processor -- start 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 -- start start 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0100ae0 0x7f4ac0196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac0101c00 0x7f4ac0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac01025a0 0x7f4ac019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac019e9f0 con 0x7f4ac0101c00 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac019eb60 con 0x7f4ac0100ae0 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4ac502d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac019ed20 con 0x7f4ac01025a0 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac0101c00 0x7f4ac0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac0101c00 0x7f4ac0197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41024/0 (socket says 172.21.15.150:41024) 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.197+0000 7f4abe59c700 1 -- 172.21.15.150:0/1770754529 learned_addr learned my addr 172.21.15.150:0/1770754529 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abed9d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0100ae0 0x7f4ac0196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abf59e700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac01025a0 0x7f4ac019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abe59c700 1 -- 172.21.15.150:0/1770754529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac01025a0 msgr2=0x7f4ac019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abe59c700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac01025a0 0x7f4ac019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abe59c700 1 -- 172.21.15.150:0/1770754529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0100ae0 msgr2=0x7f4ac0196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abe59c700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0100ae0 0x7f4ac0196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abe59c700 1 -- 172.21.15.150:0/1770754529 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab8009d00 con 0x7f4ac0101c00 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abf59e700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac01025a0 0x7f4ac019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4abe59c700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac0101c00 0x7f4ac0197450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400e9a0 tx=0x7f4ab400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4aa7fff700 1 -- 172.21.15.150:0/1770754529 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4009790 con 0x7f4ac0101c00 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac019efa0 con 0x7f4ac0101c00 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4aa7fff700 1 -- 172.21.15.150:0/1770754529 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ab400cbd0 con 0x7f4ac0101c00 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac019f4a0 con 0x7f4ac0101c00 2024-05-10T04:39:34.202 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.198+0000 7f4aa7fff700 1 -- 172.21.15.150:0/1770754529 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4018620 con 0x7f4ac0101c00 2024-05-10T04:39:34.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.200+0000 7f4aa7fff700 1 -- 172.21.15.150:0/1770754529 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4ab4018780 con 0x7f4ac0101c00 2024-05-10T04:39:34.210 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.201+0000 7f4aa7fff700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4aa80a1260 0x7f4aa80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.201+0000 7f4aa7fff700 1 -- 172.21.15.150:0/1770754529 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4ab4014070 con 0x7f4ac0101c00 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.201+0000 7f4abed9d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4aa80a1260 0x7f4aa80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.201+0000 7f4abed9d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4aa80a1260 0x7f4aa80a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0101b00 tx=0x7f4ab0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 Processor -- start 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 -- start start 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c55650 0x562947c55aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947c56930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c5d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b7c100 con 0x562947c55650 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b79ac0 con 0x562947c56e70 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b753b0 con 0x562947c56520 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4abed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c55650 0x562947c55aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.211 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4abed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c55650 0x562947c55aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41034/0 (socket says 172.21.15.150:41034) 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4abed9d700 1 -- 172.21.15.150:0/1856508838 learned_addr learned my addr 172.21.15.150:0/1856508838 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.202+0000 7f4abe59c700 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947c56930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4abf59e700 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c5d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4abed9d700 1 -- 172.21.15.150:0/1856508838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 msgr2=0x562947c56930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4abed9d700 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947c56930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4abed9d700 1 -- 172.21.15.150:0/1856508838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 msgr2=0x562947c5d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4abed9d700 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c5d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4abed9d700 1 -- 172.21.15.150:0/1856508838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629479f6360 con 0x562947c55650 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4abed9d700 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c55650 0x562947c55aa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0011d20 tx=0x7f4ab0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abc72f5494d48f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4aa57fa700 1 -- 172.21.15.150:0/1856508838 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000edd0 con 0x562947c55650 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4aa57fa700 1 -- 172.21.15.150:0/1856508838 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ab0021070 con 0x562947c55650 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.203+0000 7f4aa57fa700 1 -- 172.21.15.150:0/1856508838 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab001cea0 con 0x562947c55650 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1856508838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c55650 msgr2=0x562947c55aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.212 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c55650 0x562947c55aa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0011d20 tx=0x7f4ab0014000 comp rx=0 tx=0).stop 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1856508838 shutdown_connections 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c5d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c55650 0x562947c55aa0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1856508838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947c56930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1856508838 >> 172.21.15.150:0/1856508838 conn(0x562947bb7a10 msgr2=0x562947c4ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1856508838 shutdown_connections 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1856508838 wait complete. 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 -- start start 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.204+0000 7f4ad5371b80 1 Processor -- start 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c5f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56520 0x562947c5fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c66b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b11510 con 0x562947c56520 2024-05-10T04:39:34.213 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b111b0 con 0x562947c56e70 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629479f8650 con 0x562947c55650 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4ad5371b80 10 client.0 _ll_register_callbacks cb 0x562947b1aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56520 0x562947c5fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56520 0x562947c5fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41040/0 (socket says 172.21.15.150:41040) 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 -- 172.21.15.150:0/3066823074 learned_addr learned my addr 172.21.15.150:0/3066823074 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abed9d700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c5f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abf59e700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c66b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 -- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 msgr2=0x562947c5f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c5f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.214 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 -- 172.21.15.150:0/3066823074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 msgr2=0x562947c66b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c66b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 -- 172.21.15.150:0/3066823074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab0017070 con 0x562947c56520 2024-05-10T04:39:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abf59e700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c66b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4abe59c700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56520 0x562947c5fbd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40ca670 tx=0x7f4ab409a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512258c1875a0641 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4aa2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.205+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/3066823074 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab40b1070 con 0x562947c56520 2024-05-10T04:39:34.215 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/3066823074 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4ab409ddb0 con 0x562947c56520 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562947ace220 con 0x562947c56520 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/3066823074 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ab40a2400 con 0x562947c56520 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 10 client.15525 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5629479a2b10 con 0x562947c56520 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/3066823074 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562947ace220 con 0x562947c56520 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/3066823074 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ab40a1f70 con 0x562947c56520 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 10 client.15525 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 20 client.15525 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947c6acd0 0x562947c6f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 4 client.15525 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562947c24730 con 0x562947c6acd0 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.206+0000 7f4abed9d700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947c6acd0 0x562947c6f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.207+0000 7f4abed9d700 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947c6acd0 0x562947c6f9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=803ecf32b6310b30 server_cookie=c3eb1bc77130779f in_seq=0 out_seq=0 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.207+0000 7f4aa2ff5700 10 client.15525 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.207+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/3066823074 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562947c24730 con 0x562947c6acd0 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.207+0000 7f4aa2ff5700 10 client.15525 handle_command_reply: tid=0 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 client.15525 shutdown 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56520 msgr2=0x562947c5fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56520 0x562947c5fbd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40ca670 tx=0x7f4ab409a4c0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 shutdown_connections 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947c6acd0 0x562947c6f9e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56e70 0x562947c66b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c5f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4aa2ff5700 0 client.15525 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:34.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/3066823074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56520 0x562947c5fbd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 >> 172.21.15.150:0/3066823074 conn(0x562947bb7a10 msgr2=0x562947c4ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 shutdown_connections 2024-05-10T04:39:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 1 -- 172.21.15.150:0/3066823074 wait complete. 2024-05-10T04:39:34.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.208+0000 7f4ad5371b80 20 client.15525 trim_cache size 0 max 16384 2024-05-10T04:39:34.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 Processor -- start 2024-05-10T04:39:34.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 -- start start 2024-05-10T04:39:34.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c56930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56520 0x562947c5ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947c50bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b6e150 con 0x562947c56e70 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b72010 con 0x562947c56520 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b145a0 con 0x562947c55650 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947c50bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947c50bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41056/0 (socket says 172.21.15.150:41056) 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 -- 172.21.15.150:0/1365386565 learned_addr learned my addr 172.21.15.150:0/1365386565 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abed9d700 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c56930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abe59c700 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56520 0x562947c5ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 -- 172.21.15.150:0/1365386565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 msgr2=0x562947c56930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c56930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 -- 172.21.15.150:0/1365386565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56520 msgr2=0x562947c5ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56520 0x562947c5ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 -- 172.21.15.150:0/1365386565 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562947b3b510 con 0x562947c56e70 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abe59c700 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56520 0x562947c5ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4abf59e700 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947c50bd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006100 tx=0x7f4ab8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7b7995a0eb5264 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.226+0000 7f4aa5ffb700 1 -- 172.21.15.150:0/1365386565 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800e040 con 0x562947c56e70 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4aa5ffb700 1 -- 172.21.15.150:0/1365386565 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ab80092e0 con 0x562947c56e70 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4aa5ffb700 1 -- 172.21.15.150:0/1365386565 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab801c440 con 0x562947c56e70 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1365386565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 msgr2=0x562947c50bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947c50bd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006100 tx=0x7f4ab8009b80 comp rx=0 tx=0).stop 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1365386565 shutdown_connections 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947c50bd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c55650 0x562947c56930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/1365386565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c56520 0x562947c5ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1365386565 >> 172.21.15.150:0/1365386565 conn(0x562947bb7a10 msgr2=0x562947c4ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1365386565 shutdown_connections 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.227+0000 7f4ad5371b80 1 -- 172.21.15.150:0/1365386565 wait complete. 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 -- start start 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 Processor -- start 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c55650 0x562947cbd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947cbfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947cc2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947cc2ac0 con 0x562947c56e70 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947cc2c30 con 0x562947c55650 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947cc2df0 con 0x562947c56520 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4ad5371b80 10 client.0 _ll_register_callbacks cb 0x562947b1aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947cc2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947cc2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41060/0 (socket says 172.21.15.150:41060) 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 -- 172.21.15.150:0/216420641 learned_addr learned my addr 172.21.15.150:0/216420641 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.235 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abe59c700 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947cbfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abed9d700 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c55650 0x562947cbd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 -- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 msgr2=0x562947cbfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947cbfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 -- 172.21.15.150:0/216420641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c55650 msgr2=0x562947cbd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c55650 0x562947cbd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 -- 172.21.15.150:0/216420641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac01098e0 con 0x562947c56e70 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4abf59e700 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947cc2420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006130 tx=0x7f4ab8012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a80d0dcd435cde server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.228+0000 7f4aa2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/216420641 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800e040 con 0x562947c56e70 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/216420641 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4ab8012aa0 con 0x562947c56e70 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ab0021070 con 0x562947c56e70 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/216420641 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ab8009450 con 0x562947c56e70 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 10 client.15531 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:34.236 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5629479fac80 con 0x562947c56e70 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/216420641 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ab8033020 con 0x562947c56e70 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/216420641 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ab8019070 con 0x562947c56e70 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 10 client.15531 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 20 client.15531 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947cc5d20 0x562947cc4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 4 client.15531 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562947b6de70 con 0x562947cc5d20 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.229+0000 7f4abed9d700 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947cc5d20 0x562947cc4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4aa2ff5700 10 client.15531 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4abed9d700 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947cc5d20 0x562947cc4280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cde79a89090b30f server_cookie=6c30ede14fe0dee8 in_seq=0 out_seq=0 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4aa2ff5700 1 -- 172.21.15.150:0/216420641 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562947b6de70 con 0x562947cc5d20 2024-05-10T04:39:34.237 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4aa2ff5700 10 client.15531 handle_command_reply: tid=0 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 client.15531 shutdown 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 msgr2=0x562947cc2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947cc2420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006130 tx=0x7f4ab8012620 comp rx=0 tx=0).stop 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 shutdown_connections 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x562947cc5d20 0x562947cc4280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562947c56e70 0x562947cc2420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562947c55650 0x562947cbd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 --2- 172.21.15.150:0/216420641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562947c56520 0x562947cbfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4aa2ff5700 0 client.15531 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:39:34.238 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 >> 172.21.15.150:0/216420641 conn(0x562947bb7a10 msgr2=0x562947c59750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 shutdown_connections 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 1 -- 172.21.15.150:0/216420641 wait complete. 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.230+0000 7f4ad5371b80 20 client.15531 trim_cache size 0 max 16384 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 1, 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24520, 2024-05-10T04:39:34.239 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 61.831805518000003, 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 96.538713938000001 2024-05-10T04:39:34.240 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4aa80a1260 msgr2=0x7f4aa80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4aa80a1260 0x7f4aa80a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0101b00 tx=0x7f4ab0009260 comp rx=0 tx=0).stop 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac0101c00 msgr2=0x7f4ac0197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac0101c00 0x7f4ac0197450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400e9a0 tx=0x7f4ab400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 shutdown_connections 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4aa80a1260 0x7f4aa80a3770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac01025a0 0x7f4ac019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0100ae0 0x7f4ac0196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 --2- 172.21.15.150:0/1770754529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4ac0101c00 0x7f4ac0197450 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 >> 172.21.15.150:0/1770754529 conn(0x7f4ac00fc0f0 msgr2=0x7f4ac0104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 shutdown_connections 2024-05-10T04:39:34.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.231+0000 7f4ac502d700 1 -- 172.21.15.150:0/1770754529 wait complete. 2024-05-10T04:39:34.245 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-10T04:39:34.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.377+0000 7f4d4a151700 1 -- 172.21.15.150:0/3401048922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44100ad0 msgr2=0x7f4d44100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.377+0000 7f4d4a151700 1 --2- 172.21.15.150:0/3401048922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44100ad0 0x7f4d44100ee0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009990 tx=0x7f4d34009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 -- 172.21.15.150:0/3401048922 shutdown_connections 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 --2- 172.21.15.150:0/3401048922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d44102590 0x7f4d44108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 --2- 172.21.15.150:0/3401048922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44100ad0 0x7f4d44100ee0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 --2- 172.21.15.150:0/3401048922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d44101bf0 0x7f4d44102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 -- 172.21.15.150:0/3401048922 >> 172.21.15.150:0/3401048922 conn(0x7f4d440fc0c0 msgr2=0x7f4d440fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 -- 172.21.15.150:0/3401048922 shutdown_connections 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 -- 172.21.15.150:0/3401048922 wait complete. 2024-05-10T04:39:34.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.378+0000 7f4d4a151700 1 Processor -- start 2024-05-10T04:39:34.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d4a151700 1 -- start start 2024-05-10T04:39:34.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d4a151700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d44100ad0 0x7f4d44196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d4a151700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d44101bf0 0x7f4d44197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d4a151700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44102590 0x7f4d4419e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d4a151700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4419e8e0 con 0x7f4d44102590 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d4a151700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4419ea50 con 0x7f4d44100ad0 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d4a151700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4419ec10 con 0x7f4d44101bf0 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44102590 0x7f4d4419e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44102590 0x7f4d4419e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41072/0 (socket says 172.21.15.150:41072) 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 -- 172.21.15.150:0/2571014426 learned_addr learned my addr 172.21.15.150:0/2571014426 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d44101bf0 0x7f4d44197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d437fe700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d44100ad0 0x7f4d44196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 -- 172.21.15.150:0/2571014426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d44101bf0 msgr2=0x7f4d44197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d44101bf0 0x7f4d44197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 -- 172.21.15.150:0/2571014426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d44100ad0 msgr2=0x7f4d44196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d44100ad0 0x7f4d44196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.379+0000 7f4d43fff700 1 -- 172.21.15.150:0/2571014426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340095f0 con 0x7f4d44102590 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.380+0000 7f4d43fff700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44102590 0x7f4d4419e2d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c007ee0 tx=0x7f4d3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.380+0000 7f4d40ff9700 1 -- 172.21.15.150:0/2571014426 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00dd20 con 0x7f4d44102590 2024-05-10T04:39:34.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.380+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4419eef0 con 0x7f4d44102590 2024-05-10T04:39:34.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.380+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4419f310 con 0x7f4d44102590 2024-05-10T04:39:34.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.380+0000 7f4d40ff9700 1 -- 172.21.15.150:0/2571014426 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3c00f040 con 0x7f4d44102590 2024-05-10T04:39:34.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.380+0000 7f4d40ff9700 1 -- 172.21.15.150:0/2571014426 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0046f0 con 0x7f4d44102590 2024-05-10T04:39:34.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.382+0000 7f4d40ff9700 1 -- 172.21.15.150:0/2571014426 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f4d3c015400 con 0x7f4d44102590 2024-05-10T04:39:34.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.383+0000 7f4d40ff9700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4d300a1260 0x7f4d300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.383+0000 7f4d40ff9700 1 -- 172.21.15.150:0/2571014426 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4d3c0cc5e0 con 0x7f4d44102590 2024-05-10T04:39:34.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.383+0000 7f4d437fe700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4d300a1260 0x7f4d300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.383+0000 7f4d437fe700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4d300a1260 0x7f4d300a3770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006010 tx=0x7f4d3401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 Processor -- start 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 -- start start 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc524cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc525b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc52c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc449100 con 0x55a3cc526080 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc446ac0 con 0x55a3cc525730 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc4423b0 con 0x55a3cc524860 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc52c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc52c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41086/0 (socket says 172.21.15.150:41086) 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 -- 172.21.15.150:0/2942425056 learned_addr learned my addr 172.21.15.150:0/2942425056 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d437fe700 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc524cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc525b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 -- 172.21.15.150:0/2942425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 msgr2=0x55a3cc524cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc524cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 -- 172.21.15.150:0/2942425056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 msgr2=0x55a3cc525b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc525b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 -- 172.21.15.150:0/2942425056 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3cc2c3360 con 0x55a3cc526080 2024-05-10T04:39:34.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.384+0000 7f4d43fff700 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc52c980 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0a1510 tx=0x7f4d3c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b0ea914d2da10e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d39ffb700 1 -- 172.21.15.150:0/2942425056 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c099cc0 con 0x55a3cc526080 2024-05-10T04:39:34.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d39ffb700 1 -- 172.21.15.150:0/2942425056 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3c0995c0 con 0x55a3cc526080 2024-05-10T04:39:34.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d39ffb700 1 -- 172.21.15.150:0/2942425056 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c09d120 con 0x55a3cc526080 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d5a495b80 1 -- 172.21.15.150:0/2942425056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 msgr2=0x55a3cc52c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc52c980 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0a1510 tx=0x7f4d3c00de80 comp rx=0 tx=0).stop 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d5a495b80 1 -- 172.21.15.150:0/2942425056 shutdown_connections 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc52c980 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc524cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/2942425056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc525b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.385+0000 7f4d5a495b80 1 -- 172.21.15.150:0/2942425056 >> 172.21.15.150:0/2942425056 conn(0x55a3cc484a10 msgr2=0x55a3cc51cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 -- 172.21.15.150:0/2942425056 shutdown_connections 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 -- 172.21.15.150:0/2942425056 wait complete. 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 -- start start 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 Processor -- start 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc52e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc525730 0x55a3cc52edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc526080 0x55a3cc535d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc3de510 con 0x55a3cc525730 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc3de1b0 con 0x55a3cc526080 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.386+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc2c5650 con 0x55a3cc524860 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d5a495b80 10 client.0 _ll_register_callbacks cb 0x55a3cc3e7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc525730 0x55a3cc52edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc525730 0x55a3cc52edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41088/0 (socket says 172.21.15.150:41088) 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 -- 172.21.15.150:0/4166089835 learned_addr learned my addr 172.21.15.150:0/4166089835 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d437fe700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc52e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d43fff700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc526080 0x55a3cc535d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 -- 172.21.15.150:0/4166089835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 msgr2=0x55a3cc52e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc52e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 -- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc526080 msgr2=0x55a3cc535d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc526080 0x55a3cc535d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 -- 172.21.15.150:0/4166089835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4419f310 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d437fe700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc52e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc525730 0x55a3cc52edc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c012820 tx=0x7f4d2c012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3922580b1725e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4166089835 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c017070 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4166089835 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4d2c00f500 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3cc39b220 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4166089835 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d2c00f6c0 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d5a495b80 10 client.15543 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.387+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a3cc26fb10 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4166089835 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3cc39b220 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4166089835 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4d2c0108a0 con 0x55a3cc525730 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d5a495b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d5a495b80 10 client.15543 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d5a495b80 20 client.15543 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:34.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc53a000 0x55a3cc53ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d5a495b80 4 client.15543 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3cc4f1730 con 0x55a3cc53a000 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.388+0000 7f4d437fe700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc53a000 0x55a3cc53ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d22ffd700 10 client.15543 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d437fe700 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc53a000 0x55a3cc53ed30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9006e36b289d273 server_cookie=93e93679184c0d3d in_seq=0 out_seq=0 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4166089835 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3cc4f1730 con 0x55a3cc53a000 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d22ffd700 10 client.15543 handle_command_reply: tid=0 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 client.15543 shutdown 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc525730 msgr2=0x55a3cc52edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc525730 0x55a3cc52edc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c012820 tx=0x7f4d2c012b30 comp rx=0 tx=0).stop 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 shutdown_connections 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc53a000 0x55a3cc53ed30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc526080 0x55a3cc535d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc52e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.396 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4166089835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc525730 0x55a3cc52edc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d22ffd700 0 client.15543 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:34.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 >> 172.21.15.150:0/4166089835 conn(0x55a3cc484a10 msgr2=0x55a3cc51cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 shutdown_connections 2024-05-10T04:39:34.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.389+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4166089835 wait complete. 2024-05-10T04:39:34.397 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.390+0000 7f4d5a495b80 20 client.15543 trim_cache size 0 max 16384 2024-05-10T04:39:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 Processor -- start 2024-05-10T04:39:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 -- start start 2024-05-10T04:39:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc525b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc528960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc540d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc43b150 con 0x55a3cc526080 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc43f010 con 0x55a3cc525730 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc3e15a0 con 0x55a3cc524860 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc540d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc540d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41096/0 (socket says 172.21.15.150:41096) 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 -- 172.21.15.150:0/1673728492 learned_addr learned my addr 172.21.15.150:0/1673728492 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc528960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d437fe700 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc525b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 -- 172.21.15.150:0/1673728492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 msgr2=0x55a3cc525b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc525b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 -- 172.21.15.150:0/1673728492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 msgr2=0x55a3cc528960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc528960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d43fff700 1 -- 172.21.15.150:0/1673728492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3cc408510 con 0x55a3cc526080 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.408+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc528960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.409+0000 7f4d43fff700 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc540d40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0b91e0 tx=0x7f4d3c084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286b73af2b42aa58 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.409+0000 7f4d3a7fc700 1 -- 172.21.15.150:0/1673728492 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c099cc0 con 0x55a3cc526080 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.409+0000 7f4d3a7fc700 1 -- 172.21.15.150:0/1673728492 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3c0a1a40 con 0x55a3cc526080 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.409+0000 7f4d3a7fc700 1 -- 172.21.15.150:0/1673728492 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c097960 con 0x55a3cc526080 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 -- 172.21.15.150:0/1673728492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 msgr2=0x55a3cc540d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc540d40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0b91e0 tx=0x7f4d3c084040 comp rx=0 tx=0).stop 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 -- 172.21.15.150:0/1673728492 shutdown_connections 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc526080 0x55a3cc540d40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc524860 0x55a3cc525b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/1673728492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc528960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 -- 172.21.15.150:0/1673728492 >> 172.21.15.150:0/1673728492 conn(0x55a3cc484a10 msgr2=0x55a3cc51cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 -- 172.21.15.150:0/1673728492 shutdown_connections 2024-05-10T04:39:34.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 -- 172.21.15.150:0/1673728492 wait complete. 2024-05-10T04:39:34.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 -- start start 2024-05-10T04:39:34.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.410+0000 7f4d5a495b80 1 Processor -- start 2024-05-10T04:39:34.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc524860 0x55a3cc53eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc53f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d5a495b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc526080 0x55a3cc53ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc540450 con 0x55a3cc524860 2024-05-10T04:39:34.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc5405c0 con 0x55a3cc525730 2024-05-10T04:39:34.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d5a495b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3cc540780 con 0x55a3cc526080 2024-05-10T04:39:34.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d5a495b80 10 client.0 _ll_register_callbacks cb 0x55a3cc3e7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc524860 0x55a3cc53eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc524860 0x55a3cc53eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41112/0 (socket says 172.21.15.150:41112) 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 -- 172.21.15.150:0/4098717088 learned_addr learned my addr 172.21.15.150:0/4098717088 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc53f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d43fff700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc526080 0x55a3cc53ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 -- 172.21.15.150:0/4098717088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc526080 msgr2=0x55a3cc53ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc526080 0x55a3cc53ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 -- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 msgr2=0x55a3cc53f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc53f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 -- 172.21.15.150:0/4098717088 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3cc2c3360 con 0x55a3cc524860 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d43fff700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc526080 0x55a3cc53ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d437fe700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc524860 0x55a3cc53eb90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400b2e0 tx=0x7f4d34009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02611fa26378643 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:34.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4098717088 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3402d040 con 0x55a3cc524860 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.411+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4098717088 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4d34035270 con 0x55a3cc524860 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d3c0995c0 con 0x55a3cc524860 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4098717088 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d34035470 con 0x55a3cc524860 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 10 client.15549 fetch_fsmap learned FSMap version 28 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a3cc2c7c80 con 0x55a3cc524860 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4098717088 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3c0995c0 con 0x55a3cc524860 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4098717088 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4d34024a50 con 0x55a3cc524860 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 10 client.15549 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 20 client.15549 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc592920 0x55a3cc590e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 4 client.15549 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3cc43ae70 con 0x55a3cc592920 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.412+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc592920 0x55a3cc590e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:34.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d42ffd700 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc592920 0x55a3cc590e80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48cba7a541daf5d server_cookie=e10282f953c2ffea in_seq=0 out_seq=0 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d22ffd700 10 client.15549 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d22ffd700 1 -- 172.21.15.150:0/4098717088 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a3cc43ae70 con 0x55a3cc592920 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d22ffd700 10 client.15549 handle_command_reply: tid=0 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 client.15549 shutdown 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc524860 msgr2=0x55a3cc53eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc524860 0x55a3cc53eb90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400b2e0 tx=0x7f4d34009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 shutdown_connections 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55a3cc592920 0x55a3cc590e80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a3cc526080 0x55a3cc53ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a3cc524860 0x55a3cc53eb90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 --2- 172.21.15.150:0/4098717088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3cc525730 0x55a3cc53f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d22ffd700 0 client.15549 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:34.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 >> 172.21.15.150:0/4098717088 conn(0x55a3cc484a10 msgr2=0x55a3cc528960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 shutdown_connections 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 1 -- 172.21.15.150:0/4098717088 wait complete. 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.413+0000 7f4d5a495b80 20 client.15549 trim_cache size 0 max 16384 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "cluster_fsid": "28928ad0-0e86-11ef-bc98-c7b262605968", 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "whoami": 0, 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "id": 24548, 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "want_state": "up:active", 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "state": "up:active", 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "fs_name": "cephfs", 2024-05-10T04:39:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "rank_uptime": 85.661149585999993, 2024-05-10T04:39:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "mdsmap_epoch": 28, 2024-05-10T04:39:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch": 88, 2024-05-10T04:39:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_epoch_barrier": 86, 2024-05-10T04:39:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "uptime": 91.635778086000002 2024-05-10T04:39:34.423 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:34.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4d300a1260 msgr2=0x7f4d300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4d300a1260 0x7f4d300a3770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006010 tx=0x7f4d3401a040 comp rx=0 tx=0).stop 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44102590 msgr2=0x7f4d4419e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44102590 0x7f4d4419e2d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c007ee0 tx=0x7f4d3c00d3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 shutdown_connections 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d44102590 0x7f4d4419e2d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4d44100ad0 0x7f4d44196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4d300a1260 0x7f4d300a3770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 --2- 172.21.15.150:0/2571014426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d44101bf0 0x7f4d44197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 >> 172.21.15.150:0/2571014426 conn(0x7f4d440fc0c0 msgr2=0x7f4d44104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 shutdown_connections 2024-05-10T04:39:34.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:34.414+0000 7f4d4a151700 1 -- 172.21.15.150:0/2571014426 wait complete. 2024-05-10T04:39:34.428 INFO:teuthology.run_tasks:Running task sequential... 2024-05-10T04:39:34.438 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-10T04:39:34.438 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-10T04:39:34.438 INFO:teuthology.task.print:Enabling mgr modules 2024-05-10T04:39:34.439 INFO:teuthology.run_tasks:Running task kclient... 2024-05-10T04:39:34.451 INFO:tasks.kclient:Mounting kernel clients... 2024-05-10T04:39:34.452 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-10T04:39:34.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:34.452 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:39:34.469 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-10T04:39:34.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:34.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-10T04:39:34.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:34.548 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-05-10T04:39:34.649 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-10T04:39:34.649 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-10T04:39:34.649 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:34.650 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-10T04:39:34.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:34 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2806714967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:34.873 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:39:34.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:34 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2806714967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:34.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2806714967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:35.606 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:35 smithi150 ceph-mon[42404]: pgmap v270: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T04:39:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.799+0000 7f5a54f92700 1 -- 172.21.15.150:0/943804107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 msgr2=0x7f5a50106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.799+0000 7f5a54f92700 1 --2- 172.21.15.150:0/943804107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50106c80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009990 tx=0x7f5a40009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.800+0000 7f5a54f92700 1 -- 172.21.15.150:0/943804107 shutdown_connections 2024-05-10T04:39:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.800+0000 7f5a54f92700 1 --2- 172.21.15.150:0/943804107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a501071c0 0x7f5a5010eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:35.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.800+0000 7f5a54f92700 1 --2- 172.21.15.150:0/943804107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a50101f40 0x7f5a50102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:35.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.800+0000 7f5a54f92700 1 --2- 172.21.15.150:0/943804107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50106c80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:35.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.800+0000 7f5a54f92700 1 -- 172.21.15.150:0/943804107 >> 172.21.15.150:0/943804107 conn(0x7f5a500fd850 msgr2=0x7f5a500ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:35.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.800+0000 7f5a54f92700 1 -- 172.21.15.150:0/943804107 shutdown_connections 2024-05-10T04:39:35.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.800+0000 7f5a54f92700 1 -- 172.21.15.150:0/943804107 wait complete. 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 Processor -- start 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 -- start start 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a50101f40 0x7f5a50198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a501071c0 0x7f5a5019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5019d550 con 0x7f5a50106870 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5019d6c0 con 0x7f5a50101f40 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a54f92700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5019d880 con 0x7f5a501071c0 2024-05-10T04:39:35.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a4dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a4dd9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41132/0 (socket says 172.21.15.150:41132) 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a4dd9b700 1 -- 172.21.15.150:0/3394536001 learned_addr learned my addr 172.21.15.150:0/3394536001 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.801+0000 7f5a4e59c700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a50101f40 0x7f5a50198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4ed9d700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a501071c0 0x7f5a5019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4dd9b700 1 -- 172.21.15.150:0/3394536001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a501071c0 msgr2=0x7f5a5019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4dd9b700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a501071c0 0x7f5a5019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4dd9b700 1 -- 172.21.15.150:0/3394536001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a50101f40 msgr2=0x7f5a50198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4dd9b700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a50101f40 0x7f5a50198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4dd9b700 1 -- 172.21.15.150:0/3394536001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a400095f0 con 0x7f5a50106870 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4e59c700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a50101f40 0x7f5a50198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a4dd9b700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50198ad0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5a400098e0 tx=0x7f5a4000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a477fe700 1 -- 172.21.15.150:0/3394536001 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4001d070 con 0x7f5a50106870 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a477fe700 1 -- 172.21.15.150:0/3394536001 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a4000fb90 con 0x7f5a50106870 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5019db00 con 0x7f5a50106870 2024-05-10T04:39:35.805 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a477fe700 1 -- 172.21.15.150:0/3394536001 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40017650 con 0x7f5a50106870 2024-05-10T04:39:35.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.802+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a501a5290 con 0x7f5a50106870 2024-05-10T04:39:35.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.803+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5010a270 con 0x7f5a50106870 2024-05-10T04:39:35.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.804+0000 7f5a477fe700 1 -- 172.21.15.150:0/3394536001 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f5a4000fd00 con 0x7f5a50106870 2024-05-10T04:39:35.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.805+0000 7f5a477fe700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5a3c0a1230 0x7f5a3c0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:35.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.805+0000 7f5a4e59c700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5a3c0a1230 0x7f5a3c0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:35.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.805+0000 7f5a477fe700 1 -- 172.21.15.150:0/3394536001 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5a400ce290 con 0x7f5a50106870 2024-05-10T04:39:35.807 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.806+0000 7f5a4e59c700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5a3c0a1230 0x7f5a3c0a3740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005950 tx=0x7f5a380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:35.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:35.808+0000 7f5a477fe700 1 -- 172.21.15.150:0/3394536001 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5a400975e0 con 0x7f5a50106870 2024-05-10T04:39:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:35 smithi161 ceph-mon[46912]: pgmap v270: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T04:39:35.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:35 smithi181 ceph-mon[47314]: pgmap v270: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T04:39:36.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.049+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a501033f0 con 0x7f5a50106870 2024-05-10T04:39:36.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.050+0000 7f5a477fe700 1 -- 172.21.15.150:0/3394536001 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5a4009ac00 con 0x7f5a50106870 2024-05-10T04:39:36.051 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-10T04:39:36.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5a3c0a1230 msgr2=0x7f5a3c0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5a3c0a1230 0x7f5a3c0a3740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005950 tx=0x7f5a380058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 msgr2=0x7f5a50198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50198ad0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5a400098e0 tx=0x7f5a4000f720 comp rx=0 tx=0).stop 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 shutdown_connections 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5a3c0a1230 0x7f5a3c0a3740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a501071c0 0x7f5a5019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a50101f40 0x7f5a50198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 --2- 172.21.15.150:0/3394536001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a50106870 0x7f5a50198ad0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 >> 172.21.15.150:0/3394536001 conn(0x7f5a500fd850 msgr2=0x7f5a50104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 shutdown_connections 2024-05-10T04:39:36.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:36.052+0000 7f5a54f92700 1 -- 172.21.15.150:0/3394536001 wait complete. 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-10T04:39:36.382 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-10T04:39:36.383 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-10T04:39:36.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.383 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2024-05-10T04:39:36.402 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-10T04:39:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-10T04:39:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2024-05-10T04:39:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-10T04:39:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 41792sec preferred_lft 41792sec 2024-05-10T04:39:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::c92:6035:7edc:b85a/64 scope link noprefixroute 2024-05-10T04:39:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2024-05-10T04:39:36.404 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-10T04:39:36.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.404 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:39:36.404 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:39:36.404 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2024-05-10T04:39:36.404 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2024-05-10T04:39:36.404 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2024-05-10T04:39:36.404 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-10T04:39:36.405 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:39:36.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.527 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-10T04:39:36.554 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-05-10T04:39:36.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.555 DEBUG:teuthology.orchestra.run.smithi150:> route 2024-05-10T04:39:36.614 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2024-05-10T04:39:36.614 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-10T04:39:36.614 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-10T04:39:36.615 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-10T04:39:36.615 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-10T04:39:36.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.615 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:39:36.615 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:39:36.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-10T04:39:36.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-10T04:39:36.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-10T04:39:36.616 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:39:36.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:36 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3394536001' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:39:36.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.726 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:39:36.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.781 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2024-05-10T04:39:36.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:36 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3394536001' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:39:36.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.839 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:39:36.839 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:39:36.839 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T04:39:36.839 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-10T04:39:36.839 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:39:36.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:36 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3394536001' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-10T04:39:36.935 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-10T04:39:36.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-10T04:39:36.935 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:39:37.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:37.130 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2024-05-10T04:39:37.130 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2024-05-10T04:39:37.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2024-05-10T04:39:37.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2024-05-10T04:39:37.130 DEBUG:teuthology.orchestra.run.smithi150:> ') 2024-05-10T04:39:37.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:39:37.229 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:37.246 INFO:teuthology.orchestra.run.smithi150.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-10T04:39:37.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-10T04:39:37.246 DEBUG:teuthology.orchestra.run.smithi150:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:37.303 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-10T04:39:37.303 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-10T04:39:37.303 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/013d3174-3ee5-41d3-8986-00bdc9f11cf2 2024-05-10T04:39:37.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:37.303 DEBUG:teuthology.orchestra.run.smithi150:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/013d3174-3ee5-41d3-8986-00bdc9f11cf2 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-10T04:39:37.400 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-10T04:39:37.400 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-10T04:39:37.490 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:37 smithi150 ceph-mon[42404]: pgmap v271: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: resolved to: "172.21.15.150:3300,172.21.15.161:3300,172.21.15.181:3300" 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with new device syntax: 0@28928ad0-0e86-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/013d3174-3ee5-41d3-8986-00bdc9f11cf2 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.150:3300/172.21.15.161:3300/172.21.15.181:3300" will pass to kernel 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: v2 mount support check returned 2 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.150:3300,172.21.15.161:3300,172.21.15.181:3300:/volumes/qa/sv_1/013d3174-3ee5-41d3-8986-00bdc9f11cf2 2024-05-10T04:39:37.530 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mds_namespace=cephfs,fsid=28928ad0-0e86-11ef-bc98-c7b262605968" will pass to kernel 2024-05-10T04:39:37.533 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-10T04:39:37.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:37.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:37.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:37.562 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T04:39:37.655 INFO:teuthology.orchestra.run.smithi150.stdout:client25304 2024-05-10T04:39:37.656 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:39:37.656 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:39:37.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:37.674 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T04:39:37.767 INFO:teuthology.orchestra.run.smithi150.stdout:client25304 2024-05-10T04:39:37.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:37.767 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-10T04:39:37.802 INFO:teuthology.orchestra.run.smithi150.stdout:client25304 2024-05-10T04:39:37.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:39:37.802 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/28928ad0-0e86-11ef-bc98-c7b262605968.client25304/status') 2024-05-10T04:39:37.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:37 smithi161 ceph-mon[46912]: pgmap v271: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T04:39:37.883 INFO:teuthology.orchestra.run.smithi150.stdout:instance: client.25304 (3)172.21.15.150:0/2655352801 2024-05-10T04:39:37.883 INFO:teuthology.orchestra.run.smithi150.stdout:blocklisted: false 2024-05-10T04:39:37.884 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records in 2024-05-10T04:39:37.884 INFO:teuthology.orchestra.run.smithi150.stderr:0+1 records out 2024-05-10T04:39:37.884 INFO:teuthology.orchestra.run.smithi150.stderr:72 bytes copied, 2.1576e-05 s, 3.3 MB/s 2024-05-10T04:39:37.885 INFO:teuthology.run_tasks:Running task exec... 2024-05-10T04:39:37.895 INFO:teuthology.task.exec:Executing custom commands... 2024-05-10T04:39:37.895 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:39:37.896 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-10T04:39:37.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:37 smithi181 ceph-mon[47314]: pgmap v271: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-10T04:39:38.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.058+0000 7f2773159700 1 -- 172.21.15.150:0/3665393097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 msgr2=0x7f276c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.058+0000 7f2773159700 1 --2- 172.21.15.150:0/3665393097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c100ed0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2760009990 tx=0x7f2760009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.058+0000 7f2773159700 1 -- 172.21.15.150:0/3665393097 shutdown_connections 2024-05-10T04:39:38.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.058+0000 7f2773159700 1 --2- 172.21.15.150:0/3665393097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276c102580 0x7f276c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.058+0000 7f2773159700 1 --2- 172.21.15.150:0/3665393097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c100ed0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.058+0000 7f2773159700 1 --2- 172.21.15.150:0/3665393097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f276c101be0 0x7f276c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.058+0000 7f2773159700 1 -- 172.21.15.150:0/3665393097 >> 172.21.15.150:0/3665393097 conn(0x7f276c0fc0f0 msgr2=0x7f276c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 -- 172.21.15.150:0/3665393097 shutdown_connections 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 -- 172.21.15.150:0/3665393097 wait complete. 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 Processor -- start 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 -- start start 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f276c101be0 0x7f276c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276c102580 0x7f276c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c19fa60 con 0x7f276c100ac0 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c19fba0 con 0x7f276c101be0 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2773159700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c19fd30 con 0x7f276c102580 2024-05-10T04:39:38.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2770ef5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2770ef5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47726/0 (socket says 172.21.15.150:47726) 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.059+0000 7f2770ef5700 1 -- 172.21.15.150:0/1639216601 learned_addr learned my addr 172.21.15.150:0/1639216601 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f276bfff700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f276c101be0 0x7f276c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f27716f6700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276c102580 0x7f276c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2770ef5700 1 -- 172.21.15.150:0/1639216601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276c102580 msgr2=0x7f276c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2770ef5700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276c102580 0x7f276c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2770ef5700 1 -- 172.21.15.150:0/1639216601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f276c101be0 msgr2=0x7f276c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2770ef5700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f276c101be0 0x7f276c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2770ef5700 1 -- 172.21.15.150:0/1639216601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600095f0 con 0x7f276c100ac0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f276bfff700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f276c101be0 0x7f276c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f27716f6700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276c102580 0x7f276c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2770ef5700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c197fc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f27600098e0 tx=0x7f276000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2769ffb700 1 -- 172.21.15.150:0/1639216601 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276001d070 con 0x7f276c100ac0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2769ffb700 1 -- 172.21.15.150:0/1639216601 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2760003e20 con 0x7f276c100ac0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c19ffb0 con 0x7f276c100ac0 2024-05-10T04:39:38.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2769ffb700 1 -- 172.21.15.150:0/1639216601 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760005630 con 0x7f276c100ac0 2024-05-10T04:39:38.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.060+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c1a0320 con 0x7f276c100ac0 2024-05-10T04:39:38.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.061+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c0683a0 con 0x7f276c100ac0 2024-05-10T04:39:38.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.063+0000 7f2769ffb700 1 -- 172.21.15.150:0/1639216601 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f276000fbf0 con 0x7f276c100ac0 2024-05-10T04:39:38.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.063+0000 7f2769ffb700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f275c0a11b0 0x7f275c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.064+0000 7f276bfff700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f275c0a11b0 0x7f275c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.064+0000 7f2769ffb700 1 -- 172.21.15.150:0/1639216601 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f27600ce0f0 con 0x7f276c100ac0 2024-05-10T04:39:38.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.064+0000 7f276bfff700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f275c0a11b0 0x7f275c0a36c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2758005950 tx=0x7f27580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:38.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.067+0000 7f2769ffb700 1 -- 172.21.15.150:0/1639216601 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f27600974a0 con 0x7f276c100ac0 2024-05-10T04:39:38.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.310+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f276c10c390 con 0x7f276c100ac0 2024-05-10T04:39:38.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.458+0000 7f2769ffb700 1 -- 172.21.15.150:0/1639216601 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f276009aac0 con 0x7f276c100ac0 2024-05-10T04:39:38.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f275c0a11b0 msgr2=0x7f275c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f275c0a11b0 0x7f275c0a36c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2758005950 tx=0x7f27580058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 msgr2=0x7f276c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c197fc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f27600098e0 tx=0x7f276000f740 comp rx=0 tx=0).stop 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 shutdown_connections 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f275c0a11b0 0x7f275c0a36c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f276c102580 0x7f276c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f276c100ac0 0x7f276c197fc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 --2- 172.21.15.150:0/1639216601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f276c101be0 0x7f276c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 >> 172.21.15.150:0/1639216601 conn(0x7f276c0fc0f0 msgr2=0x7f276c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 shutdown_connections 2024-05-10T04:39:38.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.461+0000 7f2773159700 1 -- 172.21.15.150:0/1639216601 wait complete. 2024-05-10T04:39:38.475 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-10T04:39:38.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 -- 172.21.15.150:0/2866589971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 msgr2=0x7f3518108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 --2- 172.21.15.150:0/2866589971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f3518108e30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f351000b0a0 tx=0x7f351000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 -- 172.21.15.150:0/2866589971 shutdown_connections 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 --2- 172.21.15.150:0/2866589971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f3518108e30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 --2- 172.21.15.150:0/2866589971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3518100ac0 0x7f3518100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 --2- 172.21.15.150:0/2866589971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3518101be0 0x7f3518101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 -- 172.21.15.150:0/2866589971 >> 172.21.15.150:0/2866589971 conn(0x7f35180fc0f0 msgr2=0x7f35180fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 -- 172.21.15.150:0/2866589971 shutdown_connections 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 -- 172.21.15.150:0/2866589971 wait complete. 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.610+0000 7f351dcc4700 1 Processor -- start 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f351dcc4700 1 -- start start 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f351dcc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3518100ac0 0x7f35180690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f351dcc4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3518101be0 0x7f351819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f351dcc4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f351819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f351dcc4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351819fb10 con 0x7f3518102580 2024-05-10T04:39:38.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f351dcc4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351819fc80 con 0x7f3518101be0 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f351dcc4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351819fe40 con 0x7f3518100ac0 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f351819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f351819f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47748/0 (socket says 172.21.15.150:47748) 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f35177fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3518100ac0 0x7f35180690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 -- 172.21.15.150:0/3470515397 learned_addr learned my addr 172.21.15.150:0/3470515397 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3516ffd700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3518101be0 0x7f351819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 -- 172.21.15.150:0/3470515397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3518100ac0 msgr2=0x7f35180690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3518100ac0 0x7f35180690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 -- 172.21.15.150:0/3470515397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3518101be0 msgr2=0x7f351819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3518101be0 0x7f351819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 -- 172.21.15.150:0/3470515397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3510009d00 con 0x7f3518102580 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.611+0000 7f3517fff700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f351819f410 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f351000b070 tx=0x7f35100079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:38.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.612+0000 7f3514ff9700 1 -- 172.21.15.150:0/3470515397 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351000e040 con 0x7f3518102580 2024-05-10T04:39:38.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.612+0000 7f3514ff9700 1 -- 172.21.15.150:0/3470515397 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3510003c00 con 0x7f3518102580 2024-05-10T04:39:38.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.612+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35181a00c0 con 0x7f3518102580 2024-05-10T04:39:38.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.612+0000 7f3514ff9700 1 -- 172.21.15.150:0/3470515397 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351001b6c0 con 0x7f3518102580 2024-05-10T04:39:38.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.612+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35181a04e0 con 0x7f3518102580 2024-05-10T04:39:38.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.613+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3518107cd0 con 0x7f3518102580 2024-05-10T04:39:38.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.614+0000 7f3514ff9700 1 -- 172.21.15.150:0/3470515397 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f351001d470 con 0x7f3518102580 2024-05-10T04:39:38.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.614+0000 7f3514ff9700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f35040a11b0 0x7f35040a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:38.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.615+0000 7f35177fe700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f35040a11b0 0x7f35040a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:38.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.615+0000 7f3514ff9700 1 -- 172.21.15.150:0/3470515397 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f35100cde30 con 0x7f3518102580 2024-05-10T04:39:38.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.615+0000 7f35177fe700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f35040a11b0 0x7f35040a36c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3518101ae0 tx=0x7f3508009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:38.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.619+0000 7f3514ff9700 1 -- 172.21.15.150:0/3470515397 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f35100971e0 con 0x7f3518102580 2024-05-10T04:39:38.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:38 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1639216601' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-10T04:39:38.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.847+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f3518061650 con 0x7f35040a11b0 2024-05-10T04:39:38.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.853+0000 7f3514ff9700 1 -- 172.21.15.150:0/3470515397 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f3518061650 con 0x7f35040a11b0 2024-05-10T04:39:38.854 INFO:teuthology.orchestra.run.smithi150.stdout:{} 2024-05-10T04:39:38.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f35040a11b0 msgr2=0x7f35040a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f35040a11b0 0x7f35040a36c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3518101ae0 tx=0x7f3508009260 comp rx=0 tx=0).stop 2024-05-10T04:39:38.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 msgr2=0x7f351819f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f351819f410 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f351000b070 tx=0x7f35100079b0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 shutdown_connections 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3518102580 0x7f351819f410 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f35040a11b0 0x7f35040a36c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3518100ac0 0x7f35180690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 --2- 172.21.15.150:0/3470515397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3518101be0 0x7f351819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 >> 172.21.15.150:0/3470515397 conn(0x7f35180fc0f0 msgr2=0x7f3518104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 shutdown_connections 2024-05-10T04:39:38.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:38.855+0000 7f351dcc4700 1 -- 172.21.15.150:0/3470515397 wait complete. 2024-05-10T04:39:38.868 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:38 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1639216601' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-10T04:39:38.869 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-10T04:39:38.880 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-10T04:39:38.951 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:38.952 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:38 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1639216601' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 -- 172.21.15.150:0/2196780236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c1013a0 msgr2=0x7fe07c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 --2- 172.21.15.150:0/2196780236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c1013a0 0x7fe07c1017b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009990 tx=0x7fe06c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 -- 172.21.15.150:0/2196780236 shutdown_connections 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 --2- 172.21.15.150:0/2196780236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c101d40 0x7fe07c1085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 --2- 172.21.15.150:0/2196780236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c100280 0x7fe07c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 --2- 172.21.15.150:0/2196780236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c1013a0 0x7fe07c1017b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 -- 172.21.15.150:0/2196780236 >> 172.21.15.150:0/2196780236 conn(0x7fe07c0fb8b0 msgr2=0x7fe07c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:39.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 -- 172.21.15.150:0/2196780236 shutdown_connections 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.085+0000 7fe0814cc700 1 -- 172.21.15.150:0/2196780236 wait complete. 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 Processor -- start 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 -- start start 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c100280 0x7fe07c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c1013a0 0x7fe07c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c101d40 0x7fe07c19d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19d890 con 0x7fe07c100280 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19da00 con 0x7fe07c101d40 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe0814cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19dbc0 con 0x7fe07c1013a0 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe07affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c100280 0x7fe07c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe07affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c100280 0x7fe07c195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47768/0 (socket says 172.21.15.150:47768) 2024-05-10T04:39:39.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe07affd700 1 -- 172.21.15.150:0/3180056490 learned_addr learned my addr 172.21.15.150:0/3180056490 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe07a7fc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c1013a0 0x7fe07c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.086+0000 7fe07b7fe700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c101d40 0x7fe07c19d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07affd700 1 -- 172.21.15.150:0/3180056490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c1013a0 msgr2=0x7fe07c1962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07affd700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c1013a0 0x7fe07c1962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07affd700 1 -- 172.21.15.150:0/3180056490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c101d40 msgr2=0x7fe07c19d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07affd700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c101d40 0x7fe07c19d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07affd700 1 -- 172.21.15.150:0/3180056490 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c0095f0 con 0x7fe07c100280 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07a7fc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c1013a0 0x7fe07c1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07b7fe700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c101d40 0x7fe07c19d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe07affd700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c100280 0x7fe07c195db0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe07000e9f0 tx=0x7fe07000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe063fff700 1 -- 172.21.15.150:0/3180056490 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07000cbd0 con 0x7fe07c100280 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c19dea0 con 0x7fe07c100280 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe063fff700 1 -- 172.21.15.150:0/3180056490 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0700044f0 con 0x7fe07c100280 2024-05-10T04:39:39.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe063fff700 1 -- 172.21.15.150:0/3180056490 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070003e30 con 0x7fe07c100280 2024-05-10T04:39:39.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.087+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c19e2d0 con 0x7fe07c100280 2024-05-10T04:39:39.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.088+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07c107490 con 0x7fe07c100280 2024-05-10T04:39:39.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.089+0000 7fe063fff700 1 -- 172.21.15.150:0/3180056490 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe070003f90 con 0x7fe07c100280 2024-05-10T04:39:39.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.090+0000 7fe063fff700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe0640a11b0 0x7fe0640a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.090+0000 7fe07a7fc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe0640a11b0 0x7fe0640a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.090+0000 7fe063fff700 1 -- 172.21.15.150:0/3180056490 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe070014070 con 0x7fe07c100280 2024-05-10T04:39:39.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.091+0000 7fe07a7fc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe0640a11b0 0x7fe0640a36c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b2e0 tx=0x7fe06c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:39.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.094+0000 7fe063fff700 1 -- 172.21.15.150:0/3180056490 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe070096490 con 0x7fe07c100280 2024-05-10T04:39:39.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.293+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe07c068730 con 0x7fe07c100280 2024-05-10T04:39:39.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.295+0000 7fe063fff700 1 -- 172.21.15.150:0/3180056490 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7fe070099ab0 con 0x7fe07c100280 2024-05-10T04:39:39.296 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:39.297 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe0640a11b0 msgr2=0x7fe0640a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe0640a11b0 0x7fe0640a36c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b2e0 tx=0x7fe06c01a040 comp rx=0 tx=0).stop 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c100280 msgr2=0x7fe07c195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c100280 0x7fe07c195db0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe07000e9f0 tx=0x7fe07000edb0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 shutdown_connections 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe07c101d40 0x7fe07c19d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe07c100280 0x7fe07c195db0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe0640a11b0 0x7fe0640a36c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 --2- 172.21.15.150:0/3180056490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c1013a0 0x7fe07c1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 >> 172.21.15.150:0/3180056490 conn(0x7fe07c0fb8b0 msgr2=0x7fe07c1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 shutdown_connections 2024-05-10T04:39:39.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.297+0000 7fe0814cc700 1 -- 172.21.15.150:0/3180056490 wait complete. 2024-05-10T04:39:39.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:39:39.313 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:39:39.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 -- 172.21.15.150:0/684897229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 msgr2=0x7f1758109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 --2- 172.21.15.150:0/684897229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f1758109070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f175000b0a0 tx=0x7f175000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 -- 172.21.15.150:0/684897229 shutdown_connections 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 --2- 172.21.15.150:0/684897229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f1758109070 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 --2- 172.21.15.150:0/684897229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17581010b0 0x7f17581014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 --2- 172.21.15.150:0/684897229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1758101a90 0x7f1758101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 -- 172.21.15.150:0/684897229 >> 172.21.15.150:0/684897229 conn(0x7f17580fb8f0 msgr2=0x7f17580fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 -- 172.21.15.150:0/684897229 shutdown_connections 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.449+0000 7f175f975700 1 -- 172.21.15.150:0/684897229 wait complete. 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 Processor -- start 2024-05-10T04:39:39.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 -- start start 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17581010b0 0x7f1758196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1758101a90 0x7f1758197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f175819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175819e960 con 0x7f17581025f0 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175819eaa0 con 0x7f17581010b0 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175f975700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175819ec30 con 0x7f1758101a90 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f175819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f175819e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47782/0 (socket says 172.21.15.150:47782) 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 -- 172.21.15.150:0/1215822062 learned_addr learned my addr 172.21.15.150:0/1215822062 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175d711700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17581010b0 0x7f1758196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175cf10700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1758101a90 0x7f1758197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 -- 172.21.15.150:0/1215822062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1758101a90 msgr2=0x7f1758197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1758101a90 0x7f1758197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 -- 172.21.15.150:0/1215822062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17581010b0 msgr2=0x7f1758196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17581010b0 0x7f1758196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 -- 172.21.15.150:0/1215822062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1750009d00 con 0x7f17581025f0 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175cf10700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1758101a90 0x7f1758197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.450+0000 7f175df12700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f175819e390 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f175000b070 tx=0x7f1750009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.451+0000 7f174e7fc700 1 -- 172.21.15.150:0/1215822062 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000e040 con 0x7f17581025f0 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.451+0000 7f174e7fc700 1 -- 172.21.15.150:0/1215822062 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1750004d60 con 0x7f17581025f0 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.451+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175819eeb0 con 0x7f17581025f0 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.451+0000 7f174e7fc700 1 -- 172.21.15.150:0/1215822062 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175001d480 con 0x7f17581025f0 2024-05-10T04:39:39.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.451+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175819f2d0 con 0x7f17581025f0 2024-05-10T04:39:39.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.452+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1758107f10 con 0x7f17581025f0 2024-05-10T04:39:39.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.454+0000 7f174e7fc700 1 -- 172.21.15.150:0/1215822062 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1750004000 con 0x7f17581025f0 2024-05-10T04:39:39.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.454+0000 7f174e7fc700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17440a12e0 0x7f17440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.454+0000 7f174e7fc700 1 -- 172.21.15.150:0/1215822062 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f17500ce720 con 0x7f17581025f0 2024-05-10T04:39:39.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.454+0000 7f175d711700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17440a12e0 0x7f17440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.455+0000 7f175d711700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17440a12e0 0x7f17440a37f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1754005950 tx=0x7f175400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:39.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.458+0000 7f174e7fc700 1 -- 172.21.15.150:0/1215822062 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1750097ad0 con 0x7f17581025f0 2024-05-10T04:39:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.657+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f175810c450 con 0x7f17581025f0 2024-05-10T04:39:39.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.658+0000 7f174e7fc700 1 -- 172.21.15.150:0/1215822062 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f175009b0f0 con 0x7f17581025f0 2024-05-10T04:39:39.659 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:39:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:39 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1639216601' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-10T04:39:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:39 smithi150 ceph-mon[42404]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T04:39:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:39 smithi150 ceph-mon[42404]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:39 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3180056490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:39.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:39 smithi150 ceph-mon[42404]: pgmap v272: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-10T04:39:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17440a12e0 msgr2=0x7f17440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17440a12e0 0x7f17440a37f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1754005950 tx=0x7f175400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 msgr2=0x7f175819e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f175819e390 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f175000b070 tx=0x7f1750009440 comp rx=0 tx=0).stop 2024-05-10T04:39:39.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 shutdown_connections 2024-05-10T04:39:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17581025f0 0x7f175819e390 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17581010b0 0x7f1758196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17440a12e0 0x7f17440a37f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 --2- 172.21.15.150:0/1215822062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1758101a90 0x7f1758197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 >> 172.21.15.150:0/1215822062 conn(0x7f17580fb8f0 msgr2=0x7f1758105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 shutdown_connections 2024-05-10T04:39:39.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.660+0000 7f175f975700 1 -- 172.21.15.150:0/1215822062 wait complete. 2024-05-10T04:39:39.674 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:39:39.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 -- 172.21.15.150:0/2254014547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0102580 msgr2=0x7ff3b0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/2254014547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0102580 0x7ff3b0108e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b0a0 tx=0x7ff3a800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 -- 172.21.15.150:0/2254014547 shutdown_connections 2024-05-10T04:39:39.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/2254014547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0102580 0x7ff3b0108e30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/2254014547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3b0100ac0 0x7ff3b0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/2254014547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3b0101be0 0x7ff3b0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 -- 172.21.15.150:0/2254014547 >> 172.21.15.150:0/2254014547 conn(0x7ff3b00fc0d0 msgr2=0x7ff3b00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 -- 172.21.15.150:0/2254014547 shutdown_connections 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 -- 172.21.15.150:0/2254014547 wait complete. 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.809+0000 7ff3b5e67700 1 Processor -- start 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3b5e67700 1 -- start start 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3b5e67700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3b0100ac0 0x7ff3b0196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3b5e67700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0101be0 0x7ff3b0197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3b5e67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3b0102580 0x7ff3b019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3b5e67700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b019e9e0 con 0x7ff3b0101be0 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3b5e67700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b019eb50 con 0x7ff3b0100ac0 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3b5e67700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b019ed10 con 0x7ff3b0102580 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0101be0 0x7ff3b0197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0101be0 0x7ff3b0197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47796/0 (socket says 172.21.15.150:47796) 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 -- 172.21.15.150:0/106392770 learned_addr learned my addr 172.21.15.150:0/106392770 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3af7fe700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3b0100ac0 0x7ff3b0196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3affff700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3b0102580 0x7ff3b019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 -- 172.21.15.150:0/106392770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3b0102580 msgr2=0x7ff3b019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3b0102580 0x7ff3b019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 -- 172.21.15.150:0/106392770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3b0100ac0 msgr2=0x7ff3b0196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3b0100ac0 0x7ff3b0196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.810+0000 7ff3aeffd700 1 -- 172.21.15.150:0/106392770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8009d00 con 0x7ff3b0101be0 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.811+0000 7ff3aeffd700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0101be0 0x7ff3b0197440 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c920 tx=0x7ff3a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.811+0000 7ff3acff9700 1 -- 172.21.15.150:0/106392770 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0004a30 con 0x7ff3b0101be0 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.811+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b019eff0 con 0x7ff3b0101be0 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.811+0000 7ff3acff9700 1 -- 172.21.15.150:0/106392770 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a00054e0 con 0x7ff3b0101be0 2024-05-10T04:39:39.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.811+0000 7ff3acff9700 1 -- 172.21.15.150:0/106392770 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0007c10 con 0x7ff3b0101be0 2024-05-10T04:39:39.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.811+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b019f420 con 0x7ff3b0101be0 2024-05-10T04:39:39.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.812+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0107cd0 con 0x7ff3b0101be0 2024-05-10T04:39:39.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.813+0000 7ff3acff9700 1 -- 172.21.15.150:0/106392770 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7ff3a0005000 con 0x7ff3b0101be0 2024-05-10T04:39:39.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.814+0000 7ff3acff9700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3980a1260 0x7ff3980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:39.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.814+0000 7ff3af7fe700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3980a1260 0x7ff3980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:39.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.814+0000 7ff3acff9700 1 -- 172.21.15.150:0/106392770 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3a00cccd0 con 0x7ff3b0101be0 2024-05-10T04:39:39.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.814+0000 7ff3af7fe700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3980a1260 0x7ff3980a3770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0101ae0 tx=0x7ff3a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:39.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:39.817+0000 7ff3acff9700 1 -- 172.21.15.150:0/106392770 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff3a0095f90 con 0x7ff3b0101be0 2024-05-10T04:39:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:39 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1639216601' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-10T04:39:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:39 smithi161 ceph-mon[46912]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T04:39:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:39 smithi161 ceph-mon[46912]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:39 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3180056490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:39.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:39 smithi161 ceph-mon[46912]: pgmap v272: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-10T04:39:39.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:39 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1639216601' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-10T04:39:39.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:39 smithi181 ceph-mon[47314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-10T04:39:39.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:39 smithi181 ceph-mon[47314]: from='client.15567 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-10T04:39:39.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:39 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3180056490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:39.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:39 smithi181 ceph-mon[47314]: pgmap v272: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-10T04:39:40.016 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.015+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3b010c320 con 0x7ff3b0101be0 2024-05-10T04:39:40.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.016+0000 7ff3acff9700 1 -- 172.21.15.150:0/106392770 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3a00995b0 con 0x7ff3b0101be0 2024-05-10T04:39:40.017 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3980a1260 msgr2=0x7ff3980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3980a1260 0x7ff3980a3770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0101ae0 tx=0x7ff3a400b410 comp rx=0 tx=0).stop 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0101be0 msgr2=0x7ff3b0197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0101be0 0x7ff3b0197440 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c920 tx=0x7ff3a000cc30 comp rx=0 tx=0).stop 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 shutdown_connections 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3980a1260 0x7ff3980a3770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3b0102580 0x7ff3b019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3b0100ac0 0x7ff3b0196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 --2- 172.21.15.150:0/106392770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3b0101be0 0x7ff3b0197440 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 >> 172.21.15.150:0/106392770 conn(0x7ff3b00fc0d0 msgr2=0x7ff3b0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:40.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 shutdown_connections 2024-05-10T04:39:40.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.018+0000 7ff3b5e67700 1 -- 172.21.15.150:0/106392770 wait complete. 2024-05-10T04:39:40.033 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:39:40.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 -- 172.21.15.150:0/2116915492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc14101d80 msgr2=0x7fdc14108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/2116915492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc14101d80 0x7fdc14108630 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c210 tx=0x7fdc0c00c520 comp rx=0 tx=0).stop 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 -- 172.21.15.150:0/2116915492 shutdown_connections 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/2116915492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc14101d80 0x7fdc14108630 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/2116915492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc141002c0 0x7fdc141006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/2116915492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc141013e0 0x7fdc141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 -- 172.21.15.150:0/2116915492 >> 172.21.15.150:0/2116915492 conn(0x7fdc140fb8f0 msgr2=0x7fdc140fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.167+0000 7fdc1acfc700 1 -- 172.21.15.150:0/2116915492 shutdown_connections 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.168+0000 7fdc1acfc700 1 -- 172.21.15.150:0/2116915492 wait complete. 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.168+0000 7fdc1acfc700 1 Processor -- start 2024-05-10T04:39:40.170 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.168+0000 7fdc1acfc700 1 -- start start 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc1acfc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc141002c0 0x7fdc14198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc1acfc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc141013e0 0x7fdc14198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc1acfc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc14101d80 0x7fdc1419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc1acfc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1419faf0 con 0x7fdc141002c0 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc1acfc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1419fc60 con 0x7fdc141013e0 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc1acfc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1419fe20 con 0x7fdc14101d80 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc141002c0 0x7fdc14198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.172 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc141002c0 0x7fdc14198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47812/0 (socket says 172.21.15.150:47812) 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 -- 172.21.15.150:0/3991496407 learned_addr learned my addr 172.21.15.150:0/3991496407 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc19299700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc14101d80 0x7fdc1419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc13fff700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc141013e0 0x7fdc14198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 -- 172.21.15.150:0/3991496407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc14101d80 msgr2=0x7fdc1419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc14101d80 0x7fdc1419f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 -- 172.21.15.150:0/3991496407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc141013e0 msgr2=0x7fdc14198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc141013e0 0x7fdc14198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 -- 172.21.15.150:0/3991496407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c00ae10 con 0x7fdc141002c0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc18a98700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc141002c0 0x7fdc14198010 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e940 tx=0x7fdc0800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc11ffb700 1 -- 172.21.15.150:0/3991496407 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800cbd0 con 0x7fdc141002c0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc141a0100 con 0x7fdc141002c0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc11ffb700 1 -- 172.21.15.150:0/3991496407 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0800ee90 con 0x7fdc141002c0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.169+0000 7fdc11ffb700 1 -- 172.21.15.150:0/3991496407 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08003cc0 con 0x7fdc141002c0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.170+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc141a0660 con 0x7fdc141002c0 2024-05-10T04:39:40.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.171+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc141074d0 con 0x7fdc141002c0 2024-05-10T04:39:40.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.172+0000 7fdc11ffb700 1 -- 172.21.15.150:0/3991496407 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fdc0800cd30 con 0x7fdc141002c0 2024-05-10T04:39:40.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.172+0000 7fdc11ffb700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdc040a1260 0x7fdc040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.172+0000 7fdc11ffb700 1 -- 172.21.15.150:0/3991496407 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdc08014070 con 0x7fdc141002c0 2024-05-10T04:39:40.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.172+0000 7fdc13fff700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdc040a1260 0x7fdc040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.175 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.173+0000 7fdc13fff700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdc040a1260 0x7fdc040a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdc141990b0 tx=0x7fdc00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:40.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.176+0000 7fdc11ffb700 1 -- 172.21.15.150:0/3991496407 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdc08096420 con 0x7fdc141002c0 2024-05-10T04:39:40.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.372+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc1410b240 con 0x7fdc141002c0 2024-05-10T04:39:40.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.372+0000 7fdc11ffb700 1 -- 172.21.15.150:0/3991496407 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc08099a40 con 0x7fdc141002c0 2024-05-10T04:39:40.374 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:39:40.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.374+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdc040a1260 msgr2=0x7fdc040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.374+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdc040a1260 0x7fdc040a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdc141990b0 tx=0x7fdc00009260 comp rx=0 tx=0).stop 2024-05-10T04:39:40.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.374+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc141002c0 msgr2=0x7fdc14198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.374+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc141002c0 0x7fdc14198010 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e940 tx=0x7fdc0800ec50 comp rx=0 tx=0).stop 2024-05-10T04:39:40.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 shutdown_connections 2024-05-10T04:39:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fdc040a1260 0x7fdc040a3770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc14101d80 0x7fdc1419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc141002c0 0x7fdc14198010 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 --2- 172.21.15.150:0/3991496407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc141013e0 0x7fdc14198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 >> 172.21.15.150:0/3991496407 conn(0x7fdc140fb8f0 msgr2=0x7fdc14104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 shutdown_connections 2024-05-10T04:39:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.375+0000 7fdc1acfc700 1 -- 172.21.15.150:0/3991496407 wait complete. 2024-05-10T04:39:40.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 -- 172.21.15.150:0/800862053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 msgr2=0x7fe628101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/800862053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe628101ff0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe614009990 tx=0x7fe614009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 -- 172.21.15.150:0/800862053 shutdown_connections 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/800862053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe628102580 0x7fe628108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/800862053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe628100ac0 0x7fe628100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/800862053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe628101ff0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 -- 172.21.15.150:0/800862053 >> 172.21.15.150:0/800862053 conn(0x7fe6280fc0d0 msgr2=0x7fe6280fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 -- 172.21.15.150:0/800862053 shutdown_connections 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.524+0000 7fe62eaa6700 1 -- 172.21.15.150:0/800862053 wait complete. 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 Processor -- start 2024-05-10T04:39:40.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 -- start start 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe628100ac0 0x7fe628068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe62819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe628102580 0x7fe6281a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281a1ba0 con 0x7fe628101be0 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281a1d10 con 0x7fe628102580 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62eaa6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281a1ed0 con 0x7fe628100ac0 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe62819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe62819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47840/0 (socket says 172.21.15.150:47840) 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 -- 172.21.15.150:0/171004410 learned_addr learned my addr 172.21.15.150:0/171004410 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62d043700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe628102580 0x7fe6281a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62c842700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe628100ac0 0x7fe628068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 -- 172.21.15.150:0/171004410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe628100ac0 msgr2=0x7fe628068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe628100ac0 0x7fe628068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 -- 172.21.15.150:0/171004410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe628102580 msgr2=0x7fe6281a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe628102580 0x7fe6281a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe627fff700 1 -- 172.21.15.150:0/171004410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140095f0 con 0x7fe628101be0 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.525+0000 7fe62c842700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe628100ac0 0x7fe628068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.526+0000 7fe627fff700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe62819f080 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe614005b40 tx=0x7fe61400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.526+0000 7fe625ffb700 1 -- 172.21.15.150:0/171004410 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61401d070 con 0x7fe628101be0 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.526+0000 7fe625ffb700 1 -- 172.21.15.150:0/171004410 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe61400fb80 con 0x7fe628101be0 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.526+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6281a2150 con 0x7fe628101be0 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.526+0000 7fe625ffb700 1 -- 172.21.15.150:0/171004410 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140173f0 con 0x7fe628101be0 2024-05-10T04:39:40.530 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.526+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6281a25d0 con 0x7fe628101be0 2024-05-10T04:39:40.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.527+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe628107cd0 con 0x7fe628101be0 2024-05-10T04:39:40.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.528+0000 7fe625ffb700 1 -- 172.21.15.150:0/171004410 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe61400fcf0 con 0x7fe628101be0 2024-05-10T04:39:40.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.529+0000 7fe625ffb700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6180a12e0 0x7fe6180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.529+0000 7fe62c842700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6180a12e0 0x7fe6180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.529+0000 7fe625ffb700 1 -- 172.21.15.150:0/171004410 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6140cdef0 con 0x7fe628101be0 2024-05-10T04:39:40.531 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.529+0000 7fe62c842700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6180a12e0 0x7fe6180a37f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005950 tx=0x7fe61c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:40.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.534+0000 7fe625ffb700 1 -- 172.21.15.150:0/171004410 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe61405dd60 con 0x7fe628101be0 2024-05-10T04:39:40.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:40 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1215822062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:40 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/106392770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:40 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3991496407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.733+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe62810c2c0 con 0x7fe628101be0 2024-05-10T04:39:40.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.734+0000 7fe625ffb700 1 -- 172.21.15.150:0/171004410 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe614097240 con 0x7fe628101be0 2024-05-10T04:39:40.735 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:40.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6180a12e0 msgr2=0x7fe6180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.738 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6180a12e0 0x7fe6180a37f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005950 tx=0x7fe61c0058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 msgr2=0x7fe62819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe62819f080 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe614005b40 tx=0x7fe61400f690 comp rx=0 tx=0).stop 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 shutdown_connections 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe628102580 0x7fe6281a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe6180a12e0 0x7fe6180a37f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe628100ac0 0x7fe628068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 --2- 172.21.15.150:0/171004410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe628101be0 0x7fe62819f080 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 >> 172.21.15.150:0/171004410 conn(0x7fe6280fc0d0 msgr2=0x7fe628104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 shutdown_connections 2024-05-10T04:39:40.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.736+0000 7fe62eaa6700 1 -- 172.21.15.150:0/171004410 wait complete. 2024-05-10T04:39:40.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:40 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1215822062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:40 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/106392770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:40 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3991496407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 -- 172.21.15.150:0/3112852824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc100ac0 msgr2=0x7f8ecc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 --2- 172.21.15.150:0/3112852824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc100ac0 0x7f8ecc100ed0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009990 tx=0x7f8ec0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 -- 172.21.15.150:0/3112852824 shutdown_connections 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 --2- 172.21.15.150:0/3112852824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ecc102580 0x7f8ecc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 --2- 172.21.15.150:0/3112852824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc100ac0 0x7f8ecc100ed0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 --2- 172.21.15.150:0/3112852824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc101be0 0x7f8ecc101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 -- 172.21.15.150:0/3112852824 >> 172.21.15.150:0/3112852824 conn(0x7f8ecc0fc0d0 msgr2=0x7f8ecc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 -- 172.21.15.150:0/3112852824 shutdown_connections 2024-05-10T04:39:40.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.885+0000 7f8ed3075700 1 -- 172.21.15.150:0/3112852824 wait complete. 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 Processor -- start 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 -- start start 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc100ac0 0x7f8ecc068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc101be0 0x7f8ecc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ecc102580 0x7f8ecc19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc19fab0 con 0x7f8ecc101be0 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc19fc20 con 0x7f8ecc102580 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed3075700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc19fde0 con 0x7f8ecc100ac0 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc101be0 0x7f8ecc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ecbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc101be0 0x7f8ecc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47864/0 (socket says 172.21.15.150:47864) 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ecbfff700 1 -- 172.21.15.150:0/1482105656 learned_addr learned my addr 172.21.15.150:0/1482105656 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:40.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed0e11700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc100ac0 0x7f8ecc068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ed1612700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ecc102580 0x7f8ecc19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.886+0000 7f8ecbfff700 1 -- 172.21.15.150:0/1482105656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc100ac0 msgr2=0x7f8ecc068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ecbfff700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc100ac0 0x7f8ecc068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ecbfff700 1 -- 172.21.15.150:0/1482105656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ecc102580 msgr2=0x7f8ecc19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ecbfff700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ecc102580 0x7f8ecc19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ecbfff700 1 -- 172.21.15.150:0/1482105656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec00095f0 con 0x7f8ecc101be0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ecbfff700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc101be0 0x7f8ecc19cda0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800d950 tx=0x7f8eb800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ec9ffb700 1 -- 172.21.15.150:0/1482105656 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8009940 con 0x7f8ecc101be0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ec9ffb700 1 -- 172.21.15.150:0/1482105656 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8eb8010460 con 0x7f8ecc101be0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ec9ffb700 1 -- 172.21.15.150:0/1482105656 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800f620 con 0x7f8ecc101be0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc1a00c0 con 0x7f8ecc101be0 2024-05-10T04:39:40.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.887+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc1a0580 con 0x7f8ecc101be0 2024-05-10T04:39:40.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.888+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc107cd0 con 0x7f8ecc101be0 2024-05-10T04:39:40.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.889+0000 7f8ec9ffb700 1 -- 172.21.15.150:0/1482105656 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8eb80108d0 con 0x7f8ecc101be0 2024-05-10T04:39:40.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.890+0000 7f8ec9ffb700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ebc0a11b0 0x7f8ebc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:40.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.890+0000 7f8ec9ffb700 1 -- 172.21.15.150:0/1482105656 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8eb80ccc20 con 0x7f8ecc101be0 2024-05-10T04:39:40.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.890+0000 7f8ed0e11700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ebc0a11b0 0x7f8ebc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:40.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.890+0000 7f8ed0e11700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ebc0a11b0 0x7f8ebc0a36c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00038c0 tx=0x7f8ec001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:40.895 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:40.894+0000 7f8ec9ffb700 1 -- 172.21.15.150:0/1482105656 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8eb8095f40 con 0x7f8ecc101be0 2024-05-10T04:39:40.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:40 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1215822062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:40 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/106392770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:40.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:40 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3991496407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:41.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.096+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ecc10c2f0 con 0x7f8ecc101be0 2024-05-10T04:39:41.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.097+0000 7f8ec9ffb700 1 -- 172.21.15.150:0/1482105656 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f8eb8016090 con 0x7f8ecc101be0 2024-05-10T04:39:41.099 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:41.099 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:41.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ebc0a11b0 msgr2=0x7f8ebc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.101 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ebc0a11b0 0x7f8ebc0a36c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00038c0 tx=0x7f8ec001a040 comp rx=0 tx=0).stop 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc101be0 msgr2=0x7f8ecc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc101be0 0x7f8ecc19cda0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800d950 tx=0x7f8eb800dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 shutdown_connections 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ecc102580 0x7f8ecc19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8ebc0a11b0 0x7f8ebc0a36c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ecc100ac0 0x7f8ecc068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 --2- 172.21.15.150:0/1482105656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ecc101be0 0x7f8ecc19cda0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 >> 172.21.15.150:0/1482105656 conn(0x7f8ecc0fc0d0 msgr2=0x7f8ecc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 shutdown_connections 2024-05-10T04:39:41.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.100+0000 7f8ed3075700 1 -- 172.21.15.150:0/1482105656 wait complete. 2024-05-10T04:39:41.115 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:39:41.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 -- 172.21.15.150:0/3439888784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 msgr2=0x7fe80c109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 --2- 172.21.15.150:0/3439888784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c109f40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b0a0 tx=0x7fe80400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 -- 172.21.15.150:0/3439888784 shutdown_connections 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 --2- 172.21.15.150:0/3439888784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c109f40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 --2- 172.21.15.150:0/3439888784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c101bd0 0x7fe80c101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 --2- 172.21.15.150:0/3439888784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80c102cf0 0x7fe80c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 -- 172.21.15.150:0/3439888784 >> 172.21.15.150:0/3439888784 conn(0x7fe80c0fd1e0 msgr2=0x7fe80c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 -- 172.21.15.150:0/3439888784 shutdown_connections 2024-05-10T04:39:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 -- 172.21.15.150:0/3439888784 wait complete. 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 Processor -- start 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.250+0000 7fe813d59700 1 -- start start 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe813d59700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c101bd0 0x7fe80c074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe813d59700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80c102cf0 0x7fe80c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe813d59700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe813d59700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c074180 con 0x7fe80c103690 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe813d59700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0742f0 con 0x7fe80c102cf0 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe813d59700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c1a20d0 con 0x7fe80c101bd0 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47878/0 (socket says 172.21.15.150:47878) 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 -- 172.21.15.150:0/844708058 learned_addr learned my addr 172.21.15.150:0/844708058 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8112f4700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80c102cf0 0x7fe80c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe811af5700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c101bd0 0x7fe80c074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 -- 172.21.15.150:0/844708058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c101bd0 msgr2=0x7fe80c074ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c101bd0 0x7fe80c074ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 -- 172.21.15.150:0/844708058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80c102cf0 msgr2=0x7fe80c073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80c102cf0 0x7fe80c073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 -- 172.21.15.150:0/844708058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804009d00 con 0x7fe80c103690 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe8122f6700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c073aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe804004c80 tx=0x7fe804004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.251+0000 7fe7feffd700 1 -- 172.21.15.150:0/844708058 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400e040 con 0x7fe80c103690 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.252+0000 7fe7feffd700 1 -- 172.21.15.150:0/844708058 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe80400ce50 con 0x7fe80c103690 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.252+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c1a22d0 con 0x7fe80c103690 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.252+0000 7fe7feffd700 1 -- 172.21.15.150:0/844708058 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80401b9b0 con 0x7fe80c103690 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.252+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80c1a2750 con 0x7fe80c103690 2024-05-10T04:39:41.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.252+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80c108de0 con 0x7fe80c103690 2024-05-10T04:39:41.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.254+0000 7fe7feffd700 1 -- 172.21.15.150:0/844708058 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fe80401dc50 con 0x7fe80c103690 2024-05-10T04:39:41.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.255+0000 7fe7feffd700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe7f80a1260 0x7fe7f80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.255+0000 7fe811af5700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe7f80a1260 0x7fe7f80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.255+0000 7fe7feffd700 1 -- 172.21.15.150:0/844708058 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe8040cf1d0 con 0x7fe80c103690 2024-05-10T04:39:41.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.255+0000 7fe811af5700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe7f80a1260 0x7fe7f80a3770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe800005950 tx=0x7fe8000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:41.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.258+0000 7fe7feffd700 1 -- 172.21.15.150:0/844708058 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe804098520 con 0x7fe80c103690 2024-05-10T04:39:41.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.457+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe80c10d430 con 0x7fe80c103690 2024-05-10T04:39:41.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.457+0000 7fe7feffd700 1 -- 172.21.15.150:0/844708058 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe80409bb40 con 0x7fe80c103690 2024-05-10T04:39:41.459 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:39:41.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe7f80a1260 msgr2=0x7fe7f80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe7f80a1260 0x7fe7f80a3770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe800005950 tx=0x7fe8000058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 msgr2=0x7fe80c073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c073aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe804004c80 tx=0x7fe804004cb0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 shutdown_connections 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe80c103690 0x7fe80c073aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe7f80a1260 0x7fe7f80a3770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe80c101bd0 0x7fe80c074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 --2- 172.21.15.150:0/844708058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe80c102cf0 0x7fe80c073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 >> 172.21.15.150:0/844708058 conn(0x7fe80c0fd1e0 msgr2=0x7fe80c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 shutdown_connections 2024-05-10T04:39:41.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.460+0000 7fe813d59700 1 -- 172.21.15.150:0/844708058 wait complete. 2024-05-10T04:39:41.475 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:39:41.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 -- 172.21.15.150:0/809514904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 msgr2=0x7fa324104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 --2- 172.21.15.150:0/809514904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324104ae0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa314009990 tx=0x7fa314009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 -- 172.21.15.150:0/809514904 shutdown_connections 2024-05-10T04:39:41.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 --2- 172.21.15.150:0/809514904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa324107980 0x7fa324109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 --2- 172.21.15.150:0/809514904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324104ae0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 --2- 172.21.15.150:0/809514904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa324105020 0x7fa324107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 -- 172.21.15.150:0/809514904 >> 172.21.15.150:0/809514904 conn(0x7fa3240fc0e0 msgr2=0x7fa3240fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:41.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 -- 172.21.15.150:0/809514904 shutdown_connections 2024-05-10T04:39:41.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.612+0000 7fa329cda700 1 -- 172.21.15.150:0/809514904 wait complete. 2024-05-10T04:39:41.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 Processor -- start 2024-05-10T04:39:41.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 -- start start 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa324105020 0x7fa3241984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa324107980 0x7fa32419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32419f9b0 con 0x7fa3241026c0 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32419faf0 con 0x7fa324105020 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa329cda700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32419fc80 con 0x7fa324107980 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa3237fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa3237fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47892/0 (socket says 172.21.15.150:47892) 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa3237fe700 1 -- 172.21.15.150:0/2368666037 learned_addr learned my addr 172.21.15.150:0/2368666037 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.613+0000 7fa323fff700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa324107980 0x7fa32419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa3237fe700 1 -- 172.21.15.150:0/2368666037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa324107980 msgr2=0x7fa32419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa3237fe700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa324107980 0x7fa32419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa3237fe700 1 -- 172.21.15.150:0/2368666037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa324105020 msgr2=0x7fa3241984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa3237fe700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa324105020 0x7fa3241984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa3237fe700 1 -- 172.21.15.150:0/2368666037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3140095f0 con 0x7fa3241026c0 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa3237fe700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324197fa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa31400c010 tx=0x7fa31400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa320ff9700 1 -- 172.21.15.150:0/2368666037 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31401d070 con 0x7fa3241026c0 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa320ff9700 1 -- 172.21.15.150:0/2368666037 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa31400fc20 con 0x7fa3241026c0 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32419ff00 con 0x7fa3241026c0 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa320ff9700 1 -- 172.21.15.150:0/2368666037 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314017650 con 0x7fa3241026c0 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.614+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3241a02c0 con 0x7fa3241026c0 2024-05-10T04:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.615+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3240fdc90 con 0x7fa3241026c0 2024-05-10T04:39:41.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.616+0000 7fa320ff9700 1 -- 172.21.15.150:0/2368666037 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fa314022470 con 0x7fa3241026c0 2024-05-10T04:39:41.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.616+0000 7fa320ff9700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa30c0a1260 0x7fa30c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.617+0000 7fa320ff9700 1 -- 172.21.15.150:0/2368666037 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa3140ce2d0 con 0x7fa3241026c0 2024-05-10T04:39:41.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.617+0000 7fa322ffd700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa30c0a1260 0x7fa30c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.617+0000 7fa322ffd700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa30c0a1260 0x7fa30c0a3770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa318005fd0 tx=0x7fa318005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:41.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.621+0000 7fa320ff9700 1 -- 172.21.15.150:0/2368666037 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa31409b110 con 0x7fa3241026c0 2024-05-10T04:39:41.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.815+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa32410d560 con 0x7fa3241026c0 2024-05-10T04:39:41.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.815+0000 7fa320ff9700 1 -- 172.21.15.150:0/2368666037 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa31409aca0 con 0x7fa3241026c0 2024-05-10T04:39:41.816 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.817+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa30c0a1260 msgr2=0x7fa30c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.817+0000 7fa329cda700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa30c0a1260 0x7fa30c0a3770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa318005fd0 tx=0x7fa318005d00 comp rx=0 tx=0).stop 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.817+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 msgr2=0x7fa324197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.817+0000 7fa329cda700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324197fa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa31400c010 tx=0x7fa31400f740 comp rx=0 tx=0).stop 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 shutdown_connections 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa30c0a1260 0x7fa30c0a3770 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa324107980 0x7fa32419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241026c0 0x7fa324197fa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 --2- 172.21.15.150:0/2368666037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa324105020 0x7fa3241984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 >> 172.21.15.150:0/2368666037 conn(0x7fa3240fc0e0 msgr2=0x7fa3241035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:41.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 shutdown_connections 2024-05-10T04:39:41.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.818+0000 7fa329cda700 1 -- 172.21.15.150:0/2368666037 wait complete. 2024-05-10T04:39:41.830 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:41 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/171004410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:41.830 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:41 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1482105656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:41.830 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:41 smithi150 ceph-mon[42404]: pgmap v273: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T04:39:41.830 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:41 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/844708058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:41.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:39:41.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:41 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/171004410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:41.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:41 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1482105656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:41.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:41 smithi161 ceph-mon[46912]: pgmap v273: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T04:39:41.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:41 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/844708058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:41.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:41 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/171004410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:41.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:41 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1482105656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:41.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:41 smithi181 ceph-mon[47314]: pgmap v273: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-10T04:39:41.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:41 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/844708058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:41.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.966+0000 7f36e98af700 1 -- 172.21.15.150:0/3237156946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 msgr2=0x7f36e4100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.966+0000 7f36e98af700 1 --2- 172.21.15.150:0/3237156946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4100690 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009990 tx=0x7f36d4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 -- 172.21.15.150:0/3237156946 shutdown_connections 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 --2- 172.21.15.150:0/3237156946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4101d40 0x7f36e41085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 --2- 172.21.15.150:0/3237156946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4100690 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 --2- 172.21.15.150:0/3237156946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e41013a0 0x7f36e41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 -- 172.21.15.150:0/3237156946 >> 172.21.15.150:0/3237156946 conn(0x7f36e40fb8b0 msgr2=0x7f36e40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 -- 172.21.15.150:0/3237156946 shutdown_connections 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 -- 172.21.15.150:0/3237156946 wait complete. 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 Processor -- start 2024-05-10T04:39:41.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.967+0000 7f36e98af700 1 -- start start 2024-05-10T04:39:41.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e98af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e98af700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e41013a0 0x7f36e4196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e98af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4101d40 0x7f36e419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e98af700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e419d7d0 con 0x7f36e4100280 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e98af700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e419d940 con 0x7f36e41013a0 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e98af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e419db00 con 0x7f36e4101d40 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47908/0 (socket says 172.21.15.150:47908) 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 -- 172.21.15.150:0/3579874865 learned_addr learned my addr 172.21.15.150:0/3579874865 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e27fc700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e41013a0 0x7f36e4196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e37fe700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4101d40 0x7f36e419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 -- 172.21.15.150:0/3579874865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4101d40 msgr2=0x7f36e419d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4101d40 0x7f36e419d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 -- 172.21.15.150:0/3579874865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e41013a0 msgr2=0x7f36e4196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:41.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e41013a0 0x7f36e4196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 -- 172.21.15.150:0/3579874865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40095f0 con 0x7f36e4100280 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e2ffd700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4195cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f36d40098e0 tx=0x7f36d4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e88ad700 1 -- 172.21.15.150:0/3579874865 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d401d070 con 0x7f36e4100280 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e419dd80 con 0x7f36e4100280 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e88ad700 1 -- 172.21.15.150:0/3579874865 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d4021410 con 0x7f36e4100280 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.968+0000 7f36e88ad700 1 -- 172.21.15.150:0/3579874865 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4017650 con 0x7f36e4100280 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.969+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e419e1b0 con 0x7f36e4100280 2024-05-10T04:39:41.972 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.969+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4107490 con 0x7f36e4100280 2024-05-10T04:39:41.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.971+0000 7f36e88ad700 1 -- 172.21.15.150:0/3579874865 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f36d4021580 con 0x7f36e4100280 2024-05-10T04:39:41.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.971+0000 7f36e88ad700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36cc0a1260 0x7f36cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:41.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.972+0000 7f36e88ad700 1 -- 172.21.15.150:0/3579874865 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f36d40ce650 con 0x7f36e4100280 2024-05-10T04:39:41.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.972+0000 7f36e27fc700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36cc0a1260 0x7f36cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:41.973 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.972+0000 7f36e27fc700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36cc0a1260 0x7f36cc0a3770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f36e4196d70 tx=0x7f36d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:41.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:41.975+0000 7f36e88ad700 1 -- 172.21.15.150:0/3579874865 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f36d40979a0 con 0x7f36e4100280 2024-05-10T04:39:42.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.173+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36e410b480 con 0x7f36e4100280 2024-05-10T04:39:42.174 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.174+0000 7f36e88ad700 1 -- 172.21.15.150:0/3579874865 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36d409afc0 con 0x7f36e4100280 2024-05-10T04:39:42.174 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36cc0a1260 msgr2=0x7f36cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36cc0a1260 0x7f36cc0a3770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f36e4196d70 tx=0x7f36d8009260 comp rx=0 tx=0).stop 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 msgr2=0x7f36e4195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4195cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f36d40098e0 tx=0x7f36d4005b00 comp rx=0 tx=0).stop 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 shutdown_connections 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36cc0a1260 0x7f36cc0a3770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e4101d40 0x7f36e419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36e4100280 0x7f36e4195cd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 --2- 172.21.15.150:0/3579874865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e41013a0 0x7f36e4196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 >> 172.21.15.150:0/3579874865 conn(0x7f36e40fb8b0 msgr2=0x7f36e41045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:42.177 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 shutdown_connections 2024-05-10T04:39:42.178 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.176+0000 7f36e98af700 1 -- 172.21.15.150:0/3579874865 wait complete. 2024-05-10T04:39:42.190 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:39:42.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2587021040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f01026c0 msgr2=0x7f25f0104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2587021040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f01026c0 0x7f25f0104ae0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009990 tx=0x7f25e0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2587021040 shutdown_connections 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2587021040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f0107980 0x7f25f0109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2587021040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f01026c0 0x7f25f0104ae0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2587021040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f0105020 0x7f25f0107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2587021040 >> 172.21.15.150:0/2587021040 conn(0x7f25f00fc0e0 msgr2=0x7f25f00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2587021040 shutdown_connections 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2587021040 wait complete. 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.327+0000 7f25f5fb5700 1 Processor -- start 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25f5fb5700 1 -- start start 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25f5fb5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f01026c0 0x7f25f0101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25f5fb5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f0105020 0x7f25f01005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25f5fb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f0107980 0x7f25f0100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25f5fb5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f01011f0 con 0x7f25f0105020 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25f5fb5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0101360 con 0x7f25f01026c0 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25f5fb5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f01a2080 con 0x7f25f0107980 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f0105020 0x7f25f01005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f0105020 0x7f25f01005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47924/0 (socket says 172.21.15.150:47924) 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 -- 172.21.15.150:0/2502446992 learned_addr learned my addr 172.21.15.150:0/2502446992 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25effff700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f0107980 0x7f25f0100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25ef7fe700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f01026c0 0x7f25f0101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 -- 172.21.15.150:0/2502446992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f0107980 msgr2=0x7f25f0100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f0107980 0x7f25f0100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 -- 172.21.15.150:0/2502446992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f01026c0 msgr2=0x7f25f0101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f01026c0 0x7f25f0101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 -- 172.21.15.150:0/2502446992 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00095f0 con 0x7f25f0105020 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.328+0000 7f25eeffd700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f0105020 0x7f25f01005d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d980 tx=0x7f25e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:42.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.329+0000 7f25ecff9700 1 -- 172.21.15.150:0/2502446992 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4004220 con 0x7f25f0105020 2024-05-10T04:39:42.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.329+0000 7f25ecff9700 1 -- 172.21.15.150:0/2502446992 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e4007640 con 0x7f25f0105020 2024-05-10T04:39:42.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.329+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f01a22e0 con 0x7f25f0105020 2024-05-10T04:39:42.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.329+0000 7f25ecff9700 1 -- 172.21.15.150:0/2502446992 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4005610 con 0x7f25f0105020 2024-05-10T04:39:42.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.329+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f01a2810 con 0x7f25f0105020 2024-05-10T04:39:42.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.330+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f00fdc90 con 0x7f25f0105020 2024-05-10T04:39:42.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.332+0000 7f25ecff9700 1 -- 172.21.15.150:0/2502446992 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f25e4004750 con 0x7f25f0105020 2024-05-10T04:39:42.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.332+0000 7f25ecff9700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25d80a12e0 0x7f25d80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.333+0000 7f25ef7fe700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25d80a12e0 0x7f25d80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.333+0000 7f25ecff9700 1 -- 172.21.15.150:0/2502446992 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f25e40cd0e0 con 0x7f25f0105020 2024-05-10T04:39:42.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.333+0000 7f25ef7fe700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25d80a12e0 0x7f25d80a37f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f25e0006010 tx=0x7f25e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:42.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.336+0000 7f25ecff9700 1 -- 172.21.15.150:0/2502446992 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f25e40963a0 con 0x7f25f0105020 2024-05-10T04:39:42.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.536+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f25f010d420 con 0x7f25f0105020 2024-05-10T04:39:42.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.536+0000 7f25ecff9700 1 -- 172.21.15.150:0/2502446992 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f25e4016020 con 0x7f25f0105020 2024-05-10T04:39:42.537 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25d80a12e0 msgr2=0x7f25d80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25d80a12e0 0x7f25d80a37f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f25e0006010 tx=0x7f25e001a040 comp rx=0 tx=0).stop 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f0105020 msgr2=0x7f25f01005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f0105020 0x7f25f01005d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d980 tx=0x7f25e400dc90 comp rx=0 tx=0).stop 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 shutdown_connections 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f25d80a12e0 0x7f25d80a37f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25f0107980 0x7f25f0100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25f01026c0 0x7f25f0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 --2- 172.21.15.150:0/2502446992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25f0105020 0x7f25f01005d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.538+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 >> 172.21.15.150:0/2502446992 conn(0x7f25f00fc0e0 msgr2=0x7f25f0103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:42.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.539+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 shutdown_connections 2024-05-10T04:39:42.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.539+0000 7f25f5fb5700 1 -- 172.21.15.150:0/2502446992 wait complete. 2024-05-10T04:39:42.552 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-10T04:39:42.552 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:39:42.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 -- 172.21.15.150:0/4282730079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90102580 msgr2=0x7f7c90108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 --2- 172.21.15.150:0/4282730079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90102580 0x7f7c90108e30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b0a0 tx=0x7f7c8800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 -- 172.21.15.150:0/4282730079 shutdown_connections 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 --2- 172.21.15.150:0/4282730079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90102580 0x7f7c90108e30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 --2- 172.21.15.150:0/4282730079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c90100ac0 0x7f7c90100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 --2- 172.21.15.150:0/4282730079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c90101be0 0x7f7c90101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 -- 172.21.15.150:0/4282730079 >> 172.21.15.150:0/4282730079 conn(0x7f7c900fc0d0 msgr2=0x7f7c900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 -- 172.21.15.150:0/4282730079 shutdown_connections 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 -- 172.21.15.150:0/4282730079 wait complete. 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 Processor -- start 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.685+0000 7f7c98232700 1 -- start start 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c90100ac0 0x7f7c90196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90101be0 0x7f7c901974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c90102580 0x7f7c9019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9019ea50 con 0x7f7c90101be0 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9019ebc0 con 0x7f7c90102580 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9019ed80 con 0x7f7c90100ac0 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90101be0 0x7f7c901974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90101be0 0x7f7c901974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47942/0 (socket says 172.21.15.150:47942) 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 -- 172.21.15.150:0/3521362722 learned_addr learned my addr 172.21.15.150:0/3521362722 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c967cf700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c90102580 0x7f7c9019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c95fce700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c90100ac0 0x7f7c90196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 -- 172.21.15.150:0/3521362722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c90100ac0 msgr2=0x7f7c90196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c90100ac0 0x7f7c90196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 -- 172.21.15.150:0/3521362722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c90102580 msgr2=0x7f7c9019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.689 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c90102580 0x7f7c9019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 -- 172.21.15.150:0/3521362722 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88009d00 con 0x7f7c90101be0 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c967cf700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c90102580 0x7f7c9019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c957cd700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90101be0 0x7f7c901974b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7c84007910 tx=0x7f7c84007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c82ffd700 1 -- 172.21.15.150:0/3521362722 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84012070 con 0x7f7c90101be0 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c82ffd700 1 -- 172.21.15.150:0/3521362722 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c8400d810 con 0x7f7c90101be0 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c90068420 con 0x7f7c90101be0 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c82ffd700 1 -- 172.21.15.150:0/3521362722 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84005280 con 0x7f7c90101be0 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.686+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c90068930 con 0x7f7c90101be0 2024-05-10T04:39:42.690 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.687+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c90069320 con 0x7f7c90101be0 2024-05-10T04:39:42.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.688+0000 7f7c82ffd700 1 -- 172.21.15.150:0/3521362722 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7c84016070 con 0x7f7c90101be0 2024-05-10T04:39:42.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.689+0000 7f7c82ffd700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c7c0a11b0 0x7f7c7c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:42.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.689+0000 7f7c95fce700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c7c0a11b0 0x7f7c7c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:42.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.689+0000 7f7c82ffd700 1 -- 172.21.15.150:0/3521362722 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c840ce870 con 0x7f7c90101be0 2024-05-10T04:39:42.691 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.690+0000 7f7c95fce700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c7c0a11b0 0x7f7c7c0a36c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7c90101ae0 tx=0x7f7c8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:42.693 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.693+0000 7f7c82ffd700 1 -- 172.21.15.150:0/3521362722 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c84097b90 con 0x7f7c90101be0 2024-05-10T04:39:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:42 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2368666037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:42 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3579874865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:42.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:42 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2368666037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:42.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:42 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3579874865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:42.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:42 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2368666037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:42.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:42 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3579874865' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:42.940 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.940+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c9010c580 con 0x7f7c90101be0 2024-05-10T04:39:42.941 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.941+0000 7f7c82ffd700 1 -- 172.21.15.150:0/3521362722 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f7c8401c090 con 0x7f7c90101be0 2024-05-10T04:39:42.941 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:42.941 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:39:42.943 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c7c0a11b0 msgr2=0x7f7c7c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c7c0a11b0 0x7f7c7c0a36c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7c90101ae0 tx=0x7f7c8c009260 comp rx=0 tx=0).stop 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90101be0 msgr2=0x7f7c901974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90101be0 0x7f7c901974b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7c84007910 tx=0x7f7c84007cd0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 shutdown_connections 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c90102580 0x7f7c9019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7c7c0a11b0 0x7f7c7c0a36c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c90100ac0 0x7f7c90196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 --2- 172.21.15.150:0/3521362722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c90101be0 0x7f7c901974b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 >> 172.21.15.150:0/3521362722 conn(0x7f7c900fc0d0 msgr2=0x7f7c90104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 shutdown_connections 2024-05-10T04:39:42.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:42.943+0000 7f7c98232700 1 -- 172.21.15.150:0/3521362722 wait complete. 2024-05-10T04:39:42.945 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:39:42.957 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-10T04:39:42.957 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-10T04:39:42.957 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 -- 172.21.15.150:0/740706613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0104d10 msgr2=0x7f90f01071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 --2- 172.21.15.150:0/740706613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0104d10 0x7f90f01071f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009990 tx=0x7f90e0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 -- 172.21.15.150:0/740706613 shutdown_connections 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 --2- 172.21.15.150:0/740706613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90f0107880 0x7f90f0109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 --2- 172.21.15.150:0/740706613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f0069090 0x7f90f0104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 --2- 172.21.15.150:0/740706613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0104d10 0x7f90f01071f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 -- 172.21.15.150:0/740706613 >> 172.21.15.150:0/740706613 conn(0x7f90f00fc0d0 msgr2=0x7f90f00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:43.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 -- 172.21.15.150:0/740706613 shutdown_connections 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.091+0000 7f90f6713700 1 -- 172.21.15.150:0/740706613 wait complete. 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 Processor -- start 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 -- start start 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0069090 0x7f90f0101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f0104d10 0x7f90f0100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90f0107880 0x7f90f0100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01011c0 con 0x7f90f0069090 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f0101330 con 0x7f90f0104d10 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90f6713700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01014f0 con 0x7f90f0107880 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90effff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0069090 0x7f90f0101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90effff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0069090 0x7f90f0101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47964/0 (socket says 172.21.15.150:47964) 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90effff700 1 -- 172.21.15.150:0/2603290213 learned_addr learned my addr 172.21.15.150:0/2603290213 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:43.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.092+0000 7f90ef7fe700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f0104d10 0x7f90f0100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90f4cb0700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90f0107880 0x7f90f0100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90effff700 1 -- 172.21.15.150:0/2603290213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90f0107880 msgr2=0x7f90f0100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90effff700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90f0107880 0x7f90f0100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90effff700 1 -- 172.21.15.150:0/2603290213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f0104d10 msgr2=0x7f90f0100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90effff700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f0104d10 0x7f90f0100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90effff700 1 -- 172.21.15.150:0/2603290213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e00095f0 con 0x7f90f0069090 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90effff700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0069090 0x7f90f0101f80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f90d800c8c0 tx=0x7f90d800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90ed7fa700 1 -- 172.21.15.150:0/2603290213 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8004230 con 0x7f90f0069090 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90ed7fa700 1 -- 172.21.15.150:0/2603290213 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d8004cf0 con 0x7f90f0069090 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f01a2250 con 0x7f90f0069090 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90ed7fa700 1 -- 172.21.15.150:0/2603290213 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8005700 con 0x7f90f0069090 2024-05-10T04:39:43.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.093+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f01a2710 con 0x7f90f0069090 2024-05-10T04:39:43.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.094+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f00fdc80 con 0x7f90f0069090 2024-05-10T04:39:43.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.095+0000 7f90ed7fa700 1 -- 172.21.15.150:0/2603290213 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f90d8004390 con 0x7f90f0069090 2024-05-10T04:39:43.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.096+0000 7f90ed7fa700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90dc0a11b0 0x7f90dc0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.096+0000 7f90ed7fa700 1 -- 172.21.15.150:0/2603290213 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f90d80ccda0 con 0x7f90f0069090 2024-05-10T04:39:43.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.096+0000 7f90ef7fe700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90dc0a11b0 0x7f90dc0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.098 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.096+0000 7f90ef7fe700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90dc0a11b0 0x7f90dc0a36c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90e0003730 tx=0x7f90e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:43.100 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.100+0000 7f90ed7fa700 1 -- 172.21.15.150:0/2603290213 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f90d80960c0 con 0x7f90f0069090 2024-05-10T04:39:43.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.295+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90f0061650 con 0x7f90dc0a11b0 2024-05-10T04:39:43.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.307+0000 7f90ed7fa700 1 -- 172.21.15.150:0/2603290213 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427114 (secure 0 0 0) 0x7f90f0061650 con 0x7f90dc0a11b0 2024-05-10T04:39:43.313 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90dc0a11b0 msgr2=0x7f90dc0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90dc0a11b0 0x7f90dc0a36c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90e0003730 tx=0x7f90e001a040 comp rx=0 tx=0).stop 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0069090 msgr2=0x7f90f0101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0069090 0x7f90f0101f80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f90d800c8c0 tx=0x7f90d800cbd0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 shutdown_connections 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f90dc0a11b0 0x7f90dc0a36c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90f0107880 0x7f90f0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90f0069090 0x7f90f0101f80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 --2- 172.21.15.150:0/2603290213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90f0104d10 0x7f90f0100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 >> 172.21.15.150:0/2603290213 conn(0x7f90f00fc0d0 msgr2=0x7f90f0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:43.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 shutdown_connections 2024-05-10T04:39:43.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.311+0000 7f90f6713700 1 -- 172.21.15.150:0/2603290213 wait complete. 2024-05-10T04:39:43.318 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:39:43.325 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-05-10T04:39:41.329730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660851,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":765,"num_read_kb":520,"num_write":419,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182363,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360692,"kb_used_data":11868,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581132,"statfs":{"total":1151940427776,"available":1151571079168,"internally_reserved":0,"allocated":12152832,"data_stored":7774854,"data_compressed":323004,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19127,"internal_metadata":357021001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":314,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":56,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003836"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445850+0000","last_change":"2024-05-10T04:38:08.784019+0000","last_active":"2024-05-10T04:39:11.445850+0000","last_peered":"2024-05-10T04:39:11.445850+0000","last_clean":"2024-05-10T04:39:11.445850+0000","last_became_active":"2024-05-10T04:38:08.783585+0000","last_became_peered":"2024-05-10T04:38:08.783585+0000","last_unstale":"2024-05-10T04:39:11.445850+0000","last_undegraded":"2024-05-10T04:39:11.445850+0000","last_fullsized":"2024-05-10T04:39:11.445850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:56.963068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439378+0000","last_change":"2024-05-10T04:38:08.777364+0000","last_active":"2024-05-10T04:39:11.439378+0000","last_peered":"2024-05-10T04:39:11.439378+0000","last_clean":"2024-05-10T04:39:11.439378+0000","last_became_active":"2024-05-10T04:38:08.776488+0000","last_became_peered":"2024-05-10T04:38:08.776488+0000","last_unstale":"2024-05-10T04:39:11.439378+0000","last_undegraded":"2024-05-10T04:39:11.439378+0000","last_fullsized":"2024-05-10T04:39:11.439378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:40.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439228+0000","last_change":"2024-05-10T04:38:08.777169+0000","last_active":"2024-05-10T04:39:11.439228+0000","last_peered":"2024-05-10T04:39:11.439228+0000","last_clean":"2024-05-10T04:39:11.439228+0000","last_became_active":"2024-05-10T04:38:08.776713+0000","last_became_peered":"2024-05-10T04:38:08.776713+0000","last_unstale":"2024-05-10T04:39:11.439228+0000","last_undegraded":"2024-05-10T04:39:11.439228+0000","last_fullsized":"2024-05-10T04:39:11.439228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:08:41.947577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443030+0000","last_change":"2024-05-10T04:38:08.788464+0000","last_active":"2024-05-10T04:39:11.443030+0000","last_peered":"2024-05-10T04:39:11.443030+0000","last_clean":"2024-05-10T04:39:11.443030+0000","last_became_active":"2024-05-10T04:38:08.787811+0000","last_became_peered":"2024-05-10T04:38:08.787811+0000","last_unstale":"2024-05-10T04:39:11.443030+0000","last_undegraded":"2024-05-10T04:39:11.443030+0000","last_fullsized":"2024-05-10T04:39:11.443030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:22.574882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437946+0000","last_change":"2024-05-10T04:38:08.772904+0000","last_active":"2024-05-10T04:39:11.437946+0000","last_peered":"2024-05-10T04:39:11.437946+0000","last_clean":"2024-05-10T04:39:11.437946+0000","last_became_active":"2024-05-10T04:38:08.772503+0000","last_became_peered":"2024-05-10T04:38:08.772503+0000","last_unstale":"2024-05-10T04:39:11.437946+0000","last_undegraded":"2024-05-10T04:39:11.437946+0000","last_fullsized":"2024-05-10T04:39:11.437946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:58.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442107+0000","last_change":"2024-05-10T04:38:08.779442+0000","last_active":"2024-05-10T04:39:11.442107+0000","last_peered":"2024-05-10T04:39:11.442107+0000","last_clean":"2024-05-10T04:39:11.442107+0000","last_became_active":"2024-05-10T04:38:08.777348+0000","last_became_peered":"2024-05-10T04:38:08.777348+0000","last_unstale":"2024-05-10T04:39:11.442107+0000","last_undegraded":"2024-05-10T04:39:11.442107+0000","last_fullsized":"2024-05-10T04:39:11.442107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:25.619031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446022+0000","last_change":"2024-05-10T04:38:08.775646+0000","last_active":"2024-05-10T04:39:11.446022+0000","last_peered":"2024-05-10T04:39:11.446022+0000","last_clean":"2024-05-10T04:39:11.446022+0000","last_became_active":"2024-05-10T04:38:08.768229+0000","last_became_peered":"2024-05-10T04:38:08.768229+0000","last_unstale":"2024-05-10T04:39:11.446022+0000","last_undegraded":"2024-05-10T04:39:11.446022+0000","last_fullsized":"2024-05-10T04:39:11.446022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:14.653065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450030+0000","last_change":"2024-05-10T04:38:08.771012+0000","last_active":"2024-05-10T04:39:11.450030+0000","last_peered":"2024-05-10T04:39:11.450030+0000","last_clean":"2024-05-10T04:39:11.450030+0000","last_became_active":"2024-05-10T04:38:08.770674+0000","last_became_peered":"2024-05-10T04:38:08.770674+0000","last_unstale":"2024-05-10T04:39:11.450030+0000","last_undegraded":"2024-05-10T04:39:11.450030+0000","last_fullsized":"2024-05-10T04:39:11.450030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:14.389263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450011+0000","last_change":"2024-05-10T04:38:08.782109+0000","last_active":"2024-05-10T04:39:11.450011+0000","last_peered":"2024-05-10T04:39:11.450011+0000","last_clean":"2024-05-10T04:39:11.450011+0000","last_became_active":"2024-05-10T04:38:08.781717+0000","last_became_peered":"2024-05-10T04:38:08.781717+0000","last_unstale":"2024-05-10T04:39:11.450011+0000","last_undegraded":"2024-05-10T04:39:11.450011+0000","last_fullsized":"2024-05-10T04:39:11.450011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:17.505448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442749+0000","last_change":"2024-05-10T04:38:08.778429+0000","last_active":"2024-05-10T04:39:11.442749+0000","last_peered":"2024-05-10T04:39:11.442749+0000","last_clean":"2024-05-10T04:39:11.442749+0000","last_became_active":"2024-05-10T04:38:08.777874+0000","last_became_peered":"2024-05-10T04:38:08.777874+0000","last_unstale":"2024-05-10T04:39:11.442749+0000","last_undegraded":"2024-05-10T04:39:11.442749+0000","last_fullsized":"2024-05-10T04:39:11.442749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:20:19.704678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446510+0000","last_change":"2024-05-10T04:38:08.780104+0000","last_active":"2024-05-10T04:39:11.446510+0000","last_peered":"2024-05-10T04:39:11.446510+0000","last_clean":"2024-05-10T04:39:11.446510+0000","last_became_active":"2024-05-10T04:38:08.779298+0000","last_became_peered":"2024-05-10T04:38:08.779298+0000","last_unstale":"2024-05-10T04:39:11.446510+0000","last_undegraded":"2024-05-10T04:39:11.446510+0000","last_fullsized":"2024-05-10T04:39:11.446510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:41.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443351+0000","last_change":"2024-05-10T04:38:08.788605+0000","last_active":"2024-05-10T04:39:11.443351+0000","last_peered":"2024-05-10T04:39:11.443351+0000","last_clean":"2024-05-10T04:39:11.443351+0000","last_became_active":"2024-05-10T04:38:08.788207+0000","last_became_peered":"2024-05-10T04:38:08.788207+0000","last_unstale":"2024-05-10T04:39:11.443351+0000","last_undegraded":"2024-05-10T04:39:11.443351+0000","last_fullsized":"2024-05-10T04:39:11.443351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:35.000980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446423+0000","last_change":"2024-05-10T04:38:08.761953+0000","last_active":"2024-05-10T04:39:11.446423+0000","last_peered":"2024-05-10T04:39:11.446423+0000","last_clean":"2024-05-10T04:39:11.446423+0000","last_became_active":"2024-05-10T04:38:08.761585+0000","last_became_peered":"2024-05-10T04:38:08.761585+0000","last_unstale":"2024-05-10T04:39:11.446423+0000","last_undegraded":"2024-05-10T04:39:11.446423+0000","last_fullsized":"2024-05-10T04:39:11.446423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:25.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439022+0000","last_change":"2024-05-10T04:38:08.777656+0000","last_active":"2024-05-10T04:39:11.439022+0000","last_peered":"2024-05-10T04:39:11.439022+0000","last_clean":"2024-05-10T04:39:11.439022+0000","last_became_active":"2024-05-10T04:38:08.777332+0000","last_became_peered":"2024-05-10T04:38:08.777332+0000","last_unstale":"2024-05-10T04:39:11.439022+0000","last_undegraded":"2024-05-10T04:39:11.439022+0000","last_fullsized":"2024-05-10T04:39:11.439022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:17:52.878943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438927+0000","last_change":"2024-05-10T04:38:08.784934+0000","last_active":"2024-05-10T04:39:11.438927+0000","last_peered":"2024-05-10T04:39:11.438927+0000","last_clean":"2024-05-10T04:39:11.438927+0000","last_became_active":"2024-05-10T04:38:08.784644+0000","last_became_peered":"2024-05-10T04:38:08.784644+0000","last_unstale":"2024-05-10T04:39:11.438927+0000","last_undegraded":"2024-05-10T04:39:11.438927+0000","last_fullsized":"2024-05-10T04:39:11.438927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:43.889305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448828+0000","last_change":"2024-05-10T04:38:08.784110+0000","last_active":"2024-05-10T04:39:11.448828+0000","last_peered":"2024-05-10T04:39:11.448828+0000","last_clean":"2024-05-10T04:39:11.448828+0000","last_became_active":"2024-05-10T04:38:08.783779+0000","last_became_peered":"2024-05-10T04:38:08.783779+0000","last_unstale":"2024-05-10T04:39:11.448828+0000","last_undegraded":"2024-05-10T04:39:11.448828+0000","last_fullsized":"2024-05-10T04:39:11.448828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:24.233024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445229+0000","last_change":"2024-05-10T04:38:08.788685+0000","last_active":"2024-05-10T04:39:11.445229+0000","last_peered":"2024-05-10T04:39:11.445229+0000","last_clean":"2024-05-10T04:39:11.445229+0000","last_became_active":"2024-05-10T04:38:08.788122+0000","last_became_peered":"2024-05-10T04:38:08.788122+0000","last_unstale":"2024-05-10T04:39:11.445229+0000","last_undegraded":"2024-05-10T04:39:11.445229+0000","last_fullsized":"2024-05-10T04:39:11.445229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:21:01.098784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444179+0000","last_change":"2024-05-10T04:38:08.776330+0000","last_active":"2024-05-10T04:39:11.444179+0000","last_peered":"2024-05-10T04:39:11.444179+0000","last_clean":"2024-05-10T04:39:11.444179+0000","last_became_active":"2024-05-10T04:38:08.776005+0000","last_became_peered":"2024-05-10T04:38:08.776005+0000","last_unstale":"2024-05-10T04:39:11.444179+0000","last_undegraded":"2024-05-10T04:39:11.444179+0000","last_fullsized":"2024-05-10T04:39:11.444179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:46:14.407766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449082+0000","last_change":"2024-05-10T04:38:08.772811+0000","last_active":"2024-05-10T04:39:11.449082+0000","last_peered":"2024-05-10T04:39:11.449082+0000","last_clean":"2024-05-10T04:39:11.449082+0000","last_became_active":"2024-05-10T04:38:08.772462+0000","last_became_peered":"2024-05-10T04:38:08.772462+0000","last_unstale":"2024-05-10T04:39:11.449082+0000","last_undegraded":"2024-05-10T04:39:11.449082+0000","last_fullsized":"2024-05-10T04:39:11.449082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:39.703137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447478+0000","last_change":"2024-05-10T04:38:12.799483+0000","last_active":"2024-05-10T04:39:11.447478+0000","last_peered":"2024-05-10T04:39:11.447478+0000","last_clean":"2024-05-10T04:39:11.447478+0000","last_became_active":"2024-05-10T04:38:12.799049+0000","last_became_peered":"2024-05-10T04:38:12.799049+0000","last_unstale":"2024-05-10T04:39:11.447478+0000","last_undegraded":"2024-05-10T04:39:11.447478+0000","last_fullsized":"2024-05-10T04:39:11.447478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:00.380645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:39.065705+0000","last_change":"2024-05-10T04:38:08.345935+0000","last_active":"2024-05-10T04:39:39.065705+0000","last_peered":"2024-05-10T04:39:39.065705+0000","last_clean":"2024-05-10T04:39:39.065705+0000","last_became_active":"2024-05-10T04:38:08.345580+0000","last_became_peered":"2024-05-10T04:38:08.345580+0000","last_unstale":"2024-05-10T04:39:39.065705+0000","last_undegraded":"2024-05-10T04:39:39.065705+0000","last_fullsized":"2024-05-10T04:39:39.065705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:11.527710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449675+0000","last_change":"2024-05-10T04:38:08.771072+0000","last_active":"2024-05-10T04:39:11.449675+0000","last_peered":"2024-05-10T04:39:11.449675+0000","last_clean":"2024-05-10T04:39:11.449675+0000","last_became_active":"2024-05-10T04:38:08.770711+0000","last_became_peered":"2024-05-10T04:38:08.770711+0000","last_unstale":"2024-05-10T04:39:11.449675+0000","last_undegraded":"2024-05-10T04:39:11.449675+0000","last_fullsized":"2024-05-10T04:39:11.449675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:44.803219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.452968+0000","last_change":"2024-05-10T04:39:12.452968+0000","last_active":"2024-05-10T04:39:12.452968+0000","last_peered":"2024-05-10T04:39:12.452968+0000","last_clean":"2024-05-10T04:39:12.452968+0000","last_became_active":"2024-05-10T04:39:12.452621+0000","last_became_peered":"2024-05-10T04:39:12.452621+0000","last_unstale":"2024-05-10T04:39:12.452968+0000","last_undegraded":"2024-05-10T04:39:12.452968+0000","last_fullsized":"2024-05-10T04:39:12.452968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:29.312810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449726+0000","last_change":"2024-05-10T04:38:08.348806+0000","last_active":"2024-05-10T04:39:11.449726+0000","last_peered":"2024-05-10T04:39:11.449726+0000","last_clean":"2024-05-10T04:39:11.449726+0000","last_became_active":"2024-05-10T04:38:08.348513+0000","last_became_peered":"2024-05-10T04:38:08.348513+0000","last_unstale":"2024-05-10T04:39:11.449726+0000","last_undegraded":"2024-05-10T04:39:11.449726+0000","last_fullsized":"2024-05-10T04:39:11.449726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:45.474403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439225+0000","last_change":"2024-05-10T04:38:08.782738+0000","last_active":"2024-05-10T04:39:11.439225+0000","last_peered":"2024-05-10T04:39:11.439225+0000","last_clean":"2024-05-10T04:39:11.439225+0000","last_became_active":"2024-05-10T04:38:08.782375+0000","last_became_peered":"2024-05-10T04:38:08.782375+0000","last_unstale":"2024-05-10T04:39:11.439225+0000","last_undegraded":"2024-05-10T04:39:11.439225+0000","last_fullsized":"2024-05-10T04:39:11.439225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:16.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442600+0000","last_change":"2024-05-10T04:38:12.801322+0000","last_active":"2024-05-10T04:39:11.442600+0000","last_peered":"2024-05-10T04:39:11.442600+0000","last_clean":"2024-05-10T04:39:11.442600+0000","last_became_active":"2024-05-10T04:38:12.800388+0000","last_became_peered":"2024-05-10T04:38:12.800388+0000","last_unstale":"2024-05-10T04:39:11.442600+0000","last_undegraded":"2024-05-10T04:39:11.442600+0000","last_fullsized":"2024-05-10T04:39:11.442600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:57.678691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438911+0000","last_change":"2024-05-10T04:38:08.347997+0000","last_active":"2024-05-10T04:39:11.438911+0000","last_peered":"2024-05-10T04:39:11.438911+0000","last_clean":"2024-05-10T04:39:11.438911+0000","last_became_active":"2024-05-10T04:38:08.347469+0000","last_became_peered":"2024-05-10T04:38:08.347469+0000","last_unstale":"2024-05-10T04:39:11.438911+0000","last_undegraded":"2024-05-10T04:39:11.438911+0000","last_fullsized":"2024-05-10T04:39:11.438911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:39.122148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449614+0000","last_change":"2024-05-10T04:38:08.781068+0000","last_active":"2024-05-10T04:39:11.449614+0000","last_peered":"2024-05-10T04:39:11.449614+0000","last_clean":"2024-05-10T04:39:11.449614+0000","last_became_active":"2024-05-10T04:38:08.780671+0000","last_became_peered":"2024-05-10T04:38:08.780671+0000","last_unstale":"2024-05-10T04:39:11.449614+0000","last_undegraded":"2024-05-10T04:39:11.449614+0000","last_fullsized":"2024-05-10T04:39:11.449614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442827+0000","last_change":"2024-05-10T04:38:12.801702+0000","last_active":"2024-05-10T04:39:11.442827+0000","last_peered":"2024-05-10T04:39:11.442827+0000","last_clean":"2024-05-10T04:39:11.442827+0000","last_became_active":"2024-05-10T04:38:12.801031+0000","last_became_peered":"2024-05-10T04:38:12.801031+0000","last_unstale":"2024-05-10T04:39:11.442827+0000","last_undegraded":"2024-05-10T04:39:11.442827+0000","last_fullsized":"2024-05-10T04:39:11.442827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:05:07.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437865+0000","last_change":"2024-05-10T04:38:08.347981+0000","last_active":"2024-05-10T04:39:11.437865+0000","last_peered":"2024-05-10T04:39:11.437865+0000","last_clean":"2024-05-10T04:39:11.437865+0000","last_became_active":"2024-05-10T04:38:08.346915+0000","last_became_peered":"2024-05-10T04:38:08.346915+0000","last_unstale":"2024-05-10T04:39:11.437865+0000","last_undegraded":"2024-05-10T04:39:11.437865+0000","last_fullsized":"2024-05-10T04:39:11.437865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:52.383734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444542+0000","last_change":"2024-05-10T04:38:08.779164+0000","last_active":"2024-05-10T04:39:11.444542+0000","last_peered":"2024-05-10T04:39:11.444542+0000","last_clean":"2024-05-10T04:39:11.444542+0000","last_became_active":"2024-05-10T04:38:08.778851+0000","last_became_peered":"2024-05-10T04:38:08.778851+0000","last_unstale":"2024-05-10T04:39:11.444542+0000","last_undegraded":"2024-05-10T04:39:11.444542+0000","last_fullsized":"2024-05-10T04:39:11.444542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:01.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.451673+0000","last_change":"2024-05-10T04:39:12.451673+0000","last_active":"2024-05-10T04:39:12.451673+0000","last_peered":"2024-05-10T04:39:12.451673+0000","last_clean":"2024-05-10T04:39:12.451673+0000","last_became_active":"2024-05-10T04:39:12.451181+0000","last_became_peered":"2024-05-10T04:39:12.451181+0000","last_unstale":"2024-05-10T04:39:12.451673+0000","last_undegraded":"2024-05-10T04:39:12.451673+0000","last_fullsized":"2024-05-10T04:39:12.451673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:46:30.325584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:39.041629+0000","last_change":"2024-05-10T04:38:08.341633+0000","last_active":"2024-05-10T04:39:39.041629+0000","last_peered":"2024-05-10T04:39:39.041629+0000","last_clean":"2024-05-10T04:39:39.041629+0000","last_became_active":"2024-05-10T04:38:08.341220+0000","last_became_peered":"2024-05-10T04:38:08.341220+0000","last_unstale":"2024-05-10T04:39:39.041629+0000","last_undegraded":"2024-05-10T04:39:39.041629+0000","last_fullsized":"2024-05-10T04:39:39.041629+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:36.861191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437912+0000","last_change":"2024-05-10T04:38:08.790969+0000","last_active":"2024-05-10T04:39:11.437912+0000","last_peered":"2024-05-10T04:39:11.437912+0000","last_clean":"2024-05-10T04:39:11.437912+0000","last_became_active":"2024-05-10T04:38:08.790539+0000","last_became_peered":"2024-05-10T04:38:08.790539+0000","last_unstale":"2024-05-10T04:39:11.437912+0000","last_undegraded":"2024-05-10T04:39:11.437912+0000","last_fullsized":"2024-05-10T04:39:11.437912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:50.174798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.453441+0000","last_change":"2024-05-10T04:39:12.453441+0000","last_active":"2024-05-10T04:39:12.453441+0000","last_peered":"2024-05-10T04:39:12.453441+0000","last_clean":"2024-05-10T04:39:12.453441+0000","last_became_active":"2024-05-10T04:39:12.452879+0000","last_became_peered":"2024-05-10T04:39:12.452879+0000","last_unstale":"2024-05-10T04:39:12.453441+0000","last_undegraded":"2024-05-10T04:39:12.453441+0000","last_fullsized":"2024-05-10T04:39:12.453441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:06.118264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442029+0000","last_change":"2024-05-10T04:38:08.342449+0000","last_active":"2024-05-10T04:39:11.442029+0000","last_peered":"2024-05-10T04:39:11.442029+0000","last_clean":"2024-05-10T04:39:11.442029+0000","last_became_active":"2024-05-10T04:38:08.342043+0000","last_became_peered":"2024-05-10T04:38:08.342043+0000","last_unstale":"2024-05-10T04:39:11.442029+0000","last_undegraded":"2024-05-10T04:39:11.442029+0000","last_fullsized":"2024-05-10T04:39:11.442029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:31.161203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444956+0000","last_change":"2024-05-10T04:38:08.783759+0000","last_active":"2024-05-10T04:39:11.444956+0000","last_peered":"2024-05-10T04:39:11.444956+0000","last_clean":"2024-05-10T04:39:11.444956+0000","last_became_active":"2024-05-10T04:38:08.783419+0000","last_became_peered":"2024-05-10T04:38:08.783419+0000","last_unstale":"2024-05-10T04:39:11.444956+0000","last_undegraded":"2024-05-10T04:39:11.444956+0000","last_fullsized":"2024-05-10T04:39:11.444956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:16.114017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441404+0000","last_change":"2024-05-10T04:38:12.803018+0000","last_active":"2024-05-10T04:39:11.441404+0000","last_peered":"2024-05-10T04:39:11.441404+0000","last_clean":"2024-05-10T04:39:11.441404+0000","last_became_active":"2024-05-10T04:38:12.802546+0000","last_became_peered":"2024-05-10T04:38:12.802546+0000","last_unstale":"2024-05-10T04:39:11.441404+0000","last_undegraded":"2024-05-10T04:39:11.441404+0000","last_fullsized":"2024-05-10T04:39:11.441404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:40.676335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446042+0000","last_change":"2024-05-10T04:38:08.341404+0000","last_active":"2024-05-10T04:39:11.446042+0000","last_peered":"2024-05-10T04:39:11.446042+0000","last_clean":"2024-05-10T04:39:11.446042+0000","last_became_active":"2024-05-10T04:38:08.341013+0000","last_became_peered":"2024-05-10T04:38:08.341013+0000","last_unstale":"2024-05-10T04:39:11.446042+0000","last_undegraded":"2024-05-10T04:39:11.446042+0000","last_fullsized":"2024-05-10T04:39:11.446042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:43.513989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450121+0000","last_change":"2024-05-10T04:38:08.775650+0000","last_active":"2024-05-10T04:39:11.450121+0000","last_peered":"2024-05-10T04:39:11.450121+0000","last_clean":"2024-05-10T04:39:11.450121+0000","last_became_active":"2024-05-10T04:38:08.775320+0000","last_became_peered":"2024-05-10T04:38:08.775320+0000","last_unstale":"2024-05-10T04:39:11.450121+0000","last_undegraded":"2024-05-10T04:39:11.450121+0000","last_fullsized":"2024-05-10T04:39:11.450121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:49:07.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439300+0000","last_change":"2024-05-10T04:38:12.797168+0000","last_active":"2024-05-10T04:39:11.439300+0000","last_peered":"2024-05-10T04:39:11.439300+0000","last_clean":"2024-05-10T04:39:11.439300+0000","last_became_active":"2024-05-10T04:38:12.796776+0000","last_became_peered":"2024-05-10T04:38:12.796776+0000","last_unstale":"2024-05-10T04:39:11.439300+0000","last_undegraded":"2024-05-10T04:39:11.439300+0000","last_fullsized":"2024-05-10T04:39:11.439300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:59:02.363398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444198+0000","last_change":"2024-05-10T04:38:08.344086+0000","last_active":"2024-05-10T04:39:11.444198+0000","last_peered":"2024-05-10T04:39:11.444198+0000","last_clean":"2024-05-10T04:39:11.444198+0000","last_became_active":"2024-05-10T04:38:08.343744+0000","last_became_peered":"2024-05-10T04:38:08.343744+0000","last_unstale":"2024-05-10T04:39:11.444198+0000","last_undegraded":"2024-05-10T04:39:11.444198+0000","last_fullsized":"2024-05-10T04:39:11.444198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:01.800590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443135+0000","last_change":"2024-05-10T04:38:08.773638+0000","last_active":"2024-05-10T04:39:11.443135+0000","last_peered":"2024-05-10T04:39:11.443135+0000","last_clean":"2024-05-10T04:39:11.443135+0000","last_became_active":"2024-05-10T04:38:08.773218+0000","last_became_peered":"2024-05-10T04:38:08.773218+0000","last_unstale":"2024-05-10T04:39:11.443135+0000","last_undegraded":"2024-05-10T04:39:11.443135+0000","last_fullsized":"2024-05-10T04:39:11.443135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:54:38.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448821+0000","last_change":"2024-05-10T04:38:12.796344+0000","last_active":"2024-05-10T04:39:11.448821+0000","last_peered":"2024-05-10T04:39:11.448821+0000","last_clean":"2024-05-10T04:39:11.448821+0000","last_became_active":"2024-05-10T04:38:12.795958+0000","last_became_peered":"2024-05-10T04:38:12.795958+0000","last_unstale":"2024-05-10T04:39:11.448821+0000","last_undegraded":"2024-05-10T04:39:11.448821+0000","last_fullsized":"2024-05-10T04:39:11.448821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:45.708078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":148,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:36.487014+0000","last_change":"2024-05-10T04:38:08.341327+0000","last_active":"2024-05-10T04:39:36.487014+0000","last_peered":"2024-05-10T04:39:36.487014+0000","last_clean":"2024-05-10T04:39:36.487014+0000","last_became_active":"2024-05-10T04:38:08.339606+0000","last_became_peered":"2024-05-10T04:38:08.339606+0000","last_unstale":"2024-05-10T04:39:36.487014+0000","last_undegraded":"2024-05-10T04:39:36.487014+0000","last_fullsized":"2024-05-10T04:39:36.487014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:22:40.212212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449986+0000","last_change":"2024-05-10T04:38:08.781195+0000","last_active":"2024-05-10T04:39:11.449986+0000","last_peered":"2024-05-10T04:39:11.449986+0000","last_clean":"2024-05-10T04:39:11.449986+0000","last_became_active":"2024-05-10T04:38:08.780779+0000","last_became_peered":"2024-05-10T04:38:08.780779+0000","last_unstale":"2024-05-10T04:39:11.449986+0000","last_undegraded":"2024-05-10T04:39:11.449986+0000","last_fullsized":"2024-05-10T04:39:11.449986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:24.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442936+0000","last_change":"2024-05-10T04:38:12.799494+0000","last_active":"2024-05-10T04:39:11.442936+0000","last_peered":"2024-05-10T04:39:11.442936+0000","last_clean":"2024-05-10T04:39:11.442936+0000","last_became_active":"2024-05-10T04:38:12.799071+0000","last_became_peered":"2024-05-10T04:38:12.799071+0000","last_unstale":"2024-05-10T04:39:11.442936+0000","last_undegraded":"2024-05-10T04:39:11.442936+0000","last_fullsized":"2024-05-10T04:39:11.442936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:00.814875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445725+0000","last_change":"2024-05-10T04:38:08.348569+0000","last_active":"2024-05-10T04:39:11.445725+0000","last_peered":"2024-05-10T04:39:11.445725+0000","last_clean":"2024-05-10T04:39:11.445725+0000","last_became_active":"2024-05-10T04:38:08.347942+0000","last_became_peered":"2024-05-10T04:38:08.347942+0000","last_unstale":"2024-05-10T04:39:11.445725+0000","last_undegraded":"2024-05-10T04:39:11.445725+0000","last_fullsized":"2024-05-10T04:39:11.445725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.865369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439473+0000","last_change":"2024-05-10T04:38:08.784447+0000","last_active":"2024-05-10T04:39:11.439473+0000","last_peered":"2024-05-10T04:39:11.439473+0000","last_clean":"2024-05-10T04:39:11.439473+0000","last_became_active":"2024-05-10T04:38:08.783989+0000","last_became_peered":"2024-05-10T04:38:08.783989+0000","last_unstale":"2024-05-10T04:39:11.439473+0000","last_undegraded":"2024-05-10T04:39:11.439473+0000","last_fullsized":"2024-05-10T04:39:11.439473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:37.511400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442966+0000","last_change":"2024-05-10T04:38:12.807478+0000","last_active":"2024-05-10T04:39:11.442966+0000","last_peered":"2024-05-10T04:39:11.442966+0000","last_clean":"2024-05-10T04:39:11.442966+0000","last_became_active":"2024-05-10T04:38:12.806801+0000","last_became_peered":"2024-05-10T04:38:12.806801+0000","last_unstale":"2024-05-10T04:39:11.442966+0000","last_undegraded":"2024-05-10T04:39:11.442966+0000","last_fullsized":"2024-05-10T04:39:11.442966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:32.168836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444446+0000","last_change":"2024-05-10T04:38:08.343610+0000","last_active":"2024-05-10T04:39:11.444446+0000","last_peered":"2024-05-10T04:39:11.444446+0000","last_clean":"2024-05-10T04:39:11.444446+0000","last_became_active":"2024-05-10T04:38:08.342880+0000","last_became_peered":"2024-05-10T04:38:08.342880+0000","last_unstale":"2024-05-10T04:39:11.444446+0000","last_undegraded":"2024-05-10T04:39:11.444446+0000","last_fullsized":"2024-05-10T04:39:11.444446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:45:11.502258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449795+0000","last_change":"2024-05-10T04:38:08.780354+0000","last_active":"2024-05-10T04:39:11.449795+0000","last_peered":"2024-05-10T04:39:11.449795+0000","last_clean":"2024-05-10T04:39:11.449795+0000","last_became_active":"2024-05-10T04:38:08.779777+0000","last_became_peered":"2024-05-10T04:38:08.779777+0000","last_unstale":"2024-05-10T04:39:11.449795+0000","last_undegraded":"2024-05-10T04:39:11.449795+0000","last_fullsized":"2024-05-10T04:39:11.449795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:28.243474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.450211+0000","last_change":"2024-05-10T04:39:12.450211+0000","last_active":"2024-05-10T04:39:12.450211+0000","last_peered":"2024-05-10T04:39:12.450211+0000","last_clean":"2024-05-10T04:39:12.450211+0000","last_became_active":"2024-05-10T04:39:12.449843+0000","last_became_peered":"2024-05-10T04:39:12.449843+0000","last_unstale":"2024-05-10T04:39:12.450211+0000","last_undegraded":"2024-05-10T04:39:12.450211+0000","last_fullsized":"2024-05-10T04:39:12.450211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:07.725273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"86'35","reported_seq":191,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:37.037748+0000","last_change":"2024-05-10T04:38:08.343873+0000","last_active":"2024-05-10T04:39:37.037748+0000","last_peered":"2024-05-10T04:39:37.037748+0000","last_clean":"2024-05-10T04:39:37.037748+0000","last_became_active":"2024-05-10T04:38:08.343360+0000","last_became_peered":"2024-05-10T04:38:08.343360+0000","last_unstale":"2024-05-10T04:39:37.037748+0000","last_undegraded":"2024-05-10T04:39:37.037748+0000","last_fullsized":"2024-05-10T04:39:37.037748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:06:10.816343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63312,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":131,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437968+0000","last_change":"2024-05-10T04:38:08.783955+0000","last_active":"2024-05-10T04:39:11.437968+0000","last_peered":"2024-05-10T04:39:11.437968+0000","last_clean":"2024-05-10T04:39:11.437968+0000","last_became_active":"2024-05-10T04:38:08.779131+0000","last_became_peered":"2024-05-10T04:38:08.779131+0000","last_unstale":"2024-05-10T04:39:11.437968+0000","last_undegraded":"2024-05-10T04:39:11.437968+0000","last_fullsized":"2024-05-10T04:39:11.437968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:58.619571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445044+0000","last_change":"2024-05-10T04:38:12.795140+0000","last_active":"2024-05-10T04:39:11.445044+0000","last_peered":"2024-05-10T04:39:11.445044+0000","last_clean":"2024-05-10T04:39:11.445044+0000","last_became_active":"2024-05-10T04:38:12.794666+0000","last_became_peered":"2024-05-10T04:38:12.794666+0000","last_unstale":"2024-05-10T04:39:11.445044+0000","last_undegraded":"2024-05-10T04:39:11.445044+0000","last_fullsized":"2024-05-10T04:39:11.445044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:44.468321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447995+0000","last_change":"2024-05-10T04:38:08.338866+0000","last_active":"2024-05-10T04:39:11.447995+0000","last_peered":"2024-05-10T04:39:11.447995+0000","last_clean":"2024-05-10T04:39:11.447995+0000","last_became_active":"2024-05-10T04:38:08.338434+0000","last_became_peered":"2024-05-10T04:38:08.338434+0000","last_unstale":"2024-05-10T04:39:11.447995+0000","last_undegraded":"2024-05-10T04:39:11.447995+0000","last_fullsized":"2024-05-10T04:39:11.447995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:49.858034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446144+0000","last_change":"2024-05-10T04:38:08.768998+0000","last_active":"2024-05-10T04:39:11.446144+0000","last_peered":"2024-05-10T04:39:11.446144+0000","last_clean":"2024-05-10T04:39:11.446144+0000","last_became_active":"2024-05-10T04:38:08.768687+0000","last_became_peered":"2024-05-10T04:38:08.768687+0000","last_unstale":"2024-05-10T04:39:11.446144+0000","last_undegraded":"2024-05-10T04:39:11.446144+0000","last_fullsized":"2024-05-10T04:39:11.446144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:19.390746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442289+0000","last_change":"2024-05-10T04:38:12.801223+0000","last_active":"2024-05-10T04:39:11.442289+0000","last_peered":"2024-05-10T04:39:11.442289+0000","last_clean":"2024-05-10T04:39:11.442289+0000","last_became_active":"2024-05-10T04:38:12.800750+0000","last_became_peered":"2024-05-10T04:38:12.800750+0000","last_unstale":"2024-05-10T04:39:11.442289+0000","last_undegraded":"2024-05-10T04:39:11.442289+0000","last_fullsized":"2024-05-10T04:39:11.442289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:33.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445239+0000","last_change":"2024-05-10T04:38:08.344579+0000","last_active":"2024-05-10T04:39:11.445239+0000","last_peered":"2024-05-10T04:39:11.445239+0000","last_clean":"2024-05-10T04:39:11.445239+0000","last_became_active":"2024-05-10T04:38:08.344250+0000","last_became_peered":"2024-05-10T04:38:08.344250+0000","last_unstale":"2024-05-10T04:39:11.445239+0000","last_undegraded":"2024-05-10T04:39:11.445239+0000","last_fullsized":"2024-05-10T04:39:11.445239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:56.491732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445406+0000","last_change":"2024-05-10T04:38:08.784556+0000","last_active":"2024-05-10T04:39:11.445406+0000","last_peered":"2024-05-10T04:39:11.445406+0000","last_clean":"2024-05-10T04:39:11.445406+0000","last_became_active":"2024-05-10T04:38:08.784229+0000","last_became_peered":"2024-05-10T04:38:08.784229+0000","last_unstale":"2024-05-10T04:39:11.445406+0000","last_undegraded":"2024-05-10T04:39:11.445406+0000","last_fullsized":"2024-05-10T04:39:11.445406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:48:46.519837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443931+0000","last_change":"2024-05-10T04:38:12.805795+0000","last_active":"2024-05-10T04:39:11.443931+0000","last_peered":"2024-05-10T04:39:11.443931+0000","last_clean":"2024-05-10T04:39:11.443931+0000","last_became_active":"2024-05-10T04:38:12.805366+0000","last_became_peered":"2024-05-10T04:38:12.805366+0000","last_unstale":"2024-05-10T04:39:11.443931+0000","last_undegraded":"2024-05-10T04:39:11.443931+0000","last_fullsized":"2024-05-10T04:39:11.443931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:38:11.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:38.064666+0000","last_change":"2024-05-10T04:38:08.345396+0000","last_active":"2024-05-10T04:39:38.064666+0000","last_peered":"2024-05-10T04:39:38.064666+0000","last_clean":"2024-05-10T04:39:38.064666+0000","last_became_active":"2024-05-10T04:38:08.345076+0000","last_became_peered":"2024-05-10T04:38:08.345076+0000","last_unstale":"2024-05-10T04:39:38.064666+0000","last_undegraded":"2024-05-10T04:39:38.064666+0000","last_fullsized":"2024-05-10T04:39:38.064666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:38.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":854,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446201+0000","last_change":"2024-05-10T04:38:08.791089+0000","last_active":"2024-05-10T04:39:11.446201+0000","last_peered":"2024-05-10T04:39:11.446201+0000","last_clean":"2024-05-10T04:39:11.446201+0000","last_became_active":"2024-05-10T04:38:08.790813+0000","last_became_peered":"2024-05-10T04:38:08.790813+0000","last_unstale":"2024-05-10T04:39:11.446201+0000","last_undegraded":"2024-05-10T04:39:11.446201+0000","last_fullsized":"2024-05-10T04:39:11.446201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:52:45.753196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446807+0000","last_change":"2024-05-10T04:38:12.803896+0000","last_active":"2024-05-10T04:39:11.446807+0000","last_peered":"2024-05-10T04:39:11.446807+0000","last_clean":"2024-05-10T04:39:11.446807+0000","last_became_active":"2024-05-10T04:38:12.803512+0000","last_became_peered":"2024-05-10T04:38:12.803512+0000","last_unstale":"2024-05-10T04:39:11.446807+0000","last_undegraded":"2024-05-10T04:39:11.446807+0000","last_fullsized":"2024-05-10T04:39:11.446807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:01:23.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.467762+0000","last_change":"2024-05-10T04:39:12.467304+0000","last_active":"2024-05-10T04:39:12.467762+0000","last_peered":"2024-05-10T04:39:12.467762+0000","last_clean":"2024-05-10T04:39:12.467762+0000","last_became_active":"2024-05-10T04:39:12.448415+0000","last_became_peered":"2024-05-10T04:39:12.448415+0000","last_unstale":"2024-05-10T04:39:12.467762+0000","last_undegraded":"2024-05-10T04:39:12.467762+0000","last_fullsized":"2024-05-10T04:39:12.467762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:20.063125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.762052+0000","last_change":"2024-05-10T04:39:12.762052+0000","last_active":"2024-05-10T04:39:12.762052+0000","last_peered":"2024-05-10T04:39:12.762052+0000","last_clean":"2024-05-10T04:39:12.762052+0000","last_became_active":"2024-05-10T04:39:12.761658+0000","last_became_peered":"2024-05-10T04:39:12.761658+0000","last_unstale":"2024-05-10T04:39:12.762052+0000","last_undegraded":"2024-05-10T04:39:12.762052+0000","last_fullsized":"2024-05-10T04:39:12.762052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:05:07.565757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443495+0000","last_change":"2024-05-10T04:38:12.796046+0000","last_active":"2024-05-10T04:39:11.443495+0000","last_peered":"2024-05-10T04:39:11.443495+0000","last_clean":"2024-05-10T04:39:11.443495+0000","last_became_active":"2024-05-10T04:38:12.795635+0000","last_became_peered":"2024-05-10T04:38:12.795635+0000","last_unstale":"2024-05-10T04:39:11.443495+0000","last_undegraded":"2024-05-10T04:39:11.443495+0000","last_fullsized":"2024-05-10T04:39:11.443495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:50.822404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438595+0000","last_change":"2024-05-10T04:38:08.348701+0000","last_active":"2024-05-10T04:39:11.438595+0000","last_peered":"2024-05-10T04:39:11.438595+0000","last_clean":"2024-05-10T04:39:11.438595+0000","last_became_active":"2024-05-10T04:38:08.348345+0000","last_became_peered":"2024-05-10T04:38:08.348345+0000","last_unstale":"2024-05-10T04:39:11.438595+0000","last_undegraded":"2024-05-10T04:39:11.438595+0000","last_fullsized":"2024-05-10T04:39:11.438595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:31.505996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.450401+0000","last_change":"2024-05-10T04:39:12.450401+0000","last_active":"2024-05-10T04:39:12.450401+0000","last_peered":"2024-05-10T04:39:12.450401+0000","last_clean":"2024-05-10T04:39:12.450401+0000","last_became_active":"2024-05-10T04:39:12.449999+0000","last_became_peered":"2024-05-10T04:39:12.449999+0000","last_unstale":"2024-05-10T04:39:12.450401+0000","last_undegraded":"2024-05-10T04:39:12.450401+0000","last_fullsized":"2024-05-10T04:39:12.450401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:46.589660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437683+0000","last_change":"2024-05-10T04:38:12.809215+0000","last_active":"2024-05-10T04:39:11.437683+0000","last_peered":"2024-05-10T04:39:11.437683+0000","last_clean":"2024-05-10T04:39:11.437683+0000","last_became_active":"2024-05-10T04:38:12.808717+0000","last_became_peered":"2024-05-10T04:38:12.808717+0000","last_unstale":"2024-05-10T04:39:11.437683+0000","last_undegraded":"2024-05-10T04:39:11.437683+0000","last_fullsized":"2024-05-10T04:39:11.437683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:40.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443605+0000","last_change":"2024-05-10T04:38:08.347770+0000","last_active":"2024-05-10T04:39:11.443605+0000","last_peered":"2024-05-10T04:39:11.443605+0000","last_clean":"2024-05-10T04:39:11.443605+0000","last_became_active":"2024-05-10T04:38:08.347444+0000","last_became_peered":"2024-05-10T04:38:08.347444+0000","last_unstale":"2024-05-10T04:39:11.443605+0000","last_undegraded":"2024-05-10T04:39:11.443605+0000","last_fullsized":"2024-05-10T04:39:11.443605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:49.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442442+0000","last_change":"2024-05-10T04:38:08.765358+0000","last_active":"2024-05-10T04:39:11.442442+0000","last_peered":"2024-05-10T04:39:11.442442+0000","last_clean":"2024-05-10T04:39:11.442442+0000","last_became_active":"2024-05-10T04:38:08.765090+0000","last_became_peered":"2024-05-10T04:38:08.765090+0000","last_unstale":"2024-05-10T04:39:11.442442+0000","last_undegraded":"2024-05-10T04:39:11.442442+0000","last_fullsized":"2024-05-10T04:39:11.442442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:55:25.722331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446434+0000","last_change":"2024-05-10T04:38:12.794765+0000","last_active":"2024-05-10T04:39:11.446434+0000","last_peered":"2024-05-10T04:39:11.446434+0000","last_clean":"2024-05-10T04:39:11.446434+0000","last_became_active":"2024-05-10T04:38:12.794400+0000","last_became_peered":"2024-05-10T04:38:12.794400+0000","last_unstale":"2024-05-10T04:39:11.446434+0000","last_undegraded":"2024-05-10T04:39:11.446434+0000","last_fullsized":"2024-05-10T04:39:11.446434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:15:16.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:38.065103+0000","last_change":"2024-05-10T04:38:08.348812+0000","last_active":"2024-05-10T04:39:38.065103+0000","last_peered":"2024-05-10T04:39:38.065103+0000","last_clean":"2024-05-10T04:39:38.065103+0000","last_became_active":"2024-05-10T04:38:08.348207+0000","last_became_peered":"2024-05-10T04:38:08.348207+0000","last_unstale":"2024-05-10T04:39:38.065103+0000","last_undegraded":"2024-05-10T04:39:38.065103+0000","last_fullsized":"2024-05-10T04:39:38.065103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:17.350094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442205+0000","last_change":"2024-05-10T04:38:08.785502+0000","last_active":"2024-05-10T04:39:11.442205+0000","last_peered":"2024-05-10T04:39:11.442205+0000","last_clean":"2024-05-10T04:39:11.442205+0000","last_became_active":"2024-05-10T04:38:08.785073+0000","last_became_peered":"2024-05-10T04:38:08.785073+0000","last_unstale":"2024-05-10T04:39:11.442205+0000","last_undegraded":"2024-05-10T04:39:11.442205+0000","last_fullsized":"2024-05-10T04:39:11.442205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:08:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446700+0000","last_change":"2024-05-10T04:38:12.809599+0000","last_active":"2024-05-10T04:39:11.446700+0000","last_peered":"2024-05-10T04:39:11.446700+0000","last_clean":"2024-05-10T04:39:11.446700+0000","last_became_active":"2024-05-10T04:38:12.809200+0000","last_became_peered":"2024-05-10T04:38:12.809200+0000","last_unstale":"2024-05-10T04:39:11.446700+0000","last_undegraded":"2024-05-10T04:39:11.446700+0000","last_fullsized":"2024-05-10T04:39:11.446700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:54.940517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444413+0000","last_change":"2024-05-10T04:38:08.779837+0000","last_active":"2024-05-10T04:39:11.444413+0000","last_peered":"2024-05-10T04:39:11.444413+0000","last_clean":"2024-05-10T04:39:11.444413+0000","last_became_active":"2024-05-10T04:38:08.779501+0000","last_became_peered":"2024-05-10T04:38:08.779501+0000","last_unstale":"2024-05-10T04:39:11.444413+0000","last_undegraded":"2024-05-10T04:39:11.444413+0000","last_fullsized":"2024-05-10T04:39:11.444413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:00.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":85,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:35.060773+0000","last_change":"2024-05-10T04:38:08.343649+0000","last_active":"2024-05-10T04:39:35.060773+0000","last_peered":"2024-05-10T04:39:35.060773+0000","last_clean":"2024-05-10T04:39:35.060773+0000","last_became_active":"2024-05-10T04:38:08.343292+0000","last_became_peered":"2024-05-10T04:38:08.343292+0000","last_unstale":"2024-05-10T04:39:35.060773+0000","last_undegraded":"2024-05-10T04:39:35.060773+0000","last_fullsized":"2024-05-10T04:39:35.060773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:53.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446345+0000","last_change":"2024-05-10T04:38:12.796552+0000","last_active":"2024-05-10T04:39:11.446345+0000","last_peered":"2024-05-10T04:39:11.446345+0000","last_clean":"2024-05-10T04:39:11.446345+0000","last_became_active":"2024-05-10T04:38:12.796151+0000","last_became_peered":"2024-05-10T04:38:12.796151+0000","last_unstale":"2024-05-10T04:39:11.446345+0000","last_undegraded":"2024-05-10T04:39:11.446345+0000","last_fullsized":"2024-05-10T04:39:11.446345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:05.520859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448787+0000","last_change":"2024-05-10T04:38:08.770957+0000","last_active":"2024-05-10T04:39:11.448787+0000","last_peered":"2024-05-10T04:39:11.448787+0000","last_clean":"2024-05-10T04:39:11.448787+0000","last_became_active":"2024-05-10T04:38:08.770572+0000","last_became_peered":"2024-05-10T04:38:08.770572+0000","last_unstale":"2024-05-10T04:39:11.448787+0000","last_undegraded":"2024-05-10T04:39:11.448787+0000","last_fullsized":"2024-05-10T04:39:11.448787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:21.421033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442857+0000","last_change":"2024-05-10T04:38:08.340788+0000","last_active":"2024-05-10T04:39:11.442857+0000","last_peered":"2024-05-10T04:39:11.442857+0000","last_clean":"2024-05-10T04:39:11.442857+0000","last_became_active":"2024-05-10T04:38:08.340407+0000","last_became_peered":"2024-05-10T04:38:08.340407+0000","last_unstale":"2024-05-10T04:39:11.442857+0000","last_undegraded":"2024-05-10T04:39:11.442857+0000","last_fullsized":"2024-05-10T04:39:11.442857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:30:05.276951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":329,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446409+0000","last_change":"2024-05-10T04:37:20.961282+0000","last_active":"2024-05-10T04:39:11.446409+0000","last_peered":"2024-05-10T04:39:11.446409+0000","last_clean":"2024-05-10T04:39:11.446409+0000","last_became_active":"2024-05-10T04:37:20.960828+0000","last_became_peered":"2024-05-10T04:37:20.960828+0000","last_unstale":"2024-05-10T04:39:11.446409+0000","last_undegraded":"2024-05-10T04:39:11.446409+0000","last_fullsized":"2024-05-10T04:39:11.446409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_clean_scrub_stamp":"2024-05-10T04:35:00.248380+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:08:04.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446236+0000","last_change":"2024-05-10T04:38:12.809899+0000","last_active":"2024-05-10T04:39:11.446236+0000","last_peered":"2024-05-10T04:39:11.446236+0000","last_clean":"2024-05-10T04:39:11.446236+0000","last_became_active":"2024-05-10T04:38:12.809441+0000","last_became_peered":"2024-05-10T04:38:12.809441+0000","last_unstale":"2024-05-10T04:39:11.446236+0000","last_undegraded":"2024-05-10T04:39:11.446236+0000","last_fullsized":"2024-05-10T04:39:11.446236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:48.378890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445906+0000","last_change":"2024-05-10T04:38:08.784838+0000","last_active":"2024-05-10T04:39:11.445906+0000","last_peered":"2024-05-10T04:39:11.445906+0000","last_clean":"2024-05-10T04:39:11.445906+0000","last_became_active":"2024-05-10T04:38:08.784464+0000","last_became_peered":"2024-05-10T04:38:08.784464+0000","last_unstale":"2024-05-10T04:39:11.445906+0000","last_undegraded":"2024-05-10T04:39:11.445906+0000","last_fullsized":"2024-05-10T04:39:11.445906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:00.335350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.470361+0000","last_change":"2024-05-10T04:39:12.469901+0000","last_active":"2024-05-10T04:39:12.470361+0000","last_peered":"2024-05-10T04:39:12.470361+0000","last_clean":"2024-05-10T04:39:12.470361+0000","last_became_active":"2024-05-10T04:39:12.454202+0000","last_became_peered":"2024-05-10T04:39:12.454202+0000","last_unstale":"2024-05-10T04:39:12.470361+0000","last_undegraded":"2024-05-10T04:39:12.470361+0000","last_fullsized":"2024-05-10T04:39:12.470361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:18.646199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441308+0000","last_change":"2024-05-10T04:38:12.801079+0000","last_active":"2024-05-10T04:39:11.441308+0000","last_peered":"2024-05-10T04:39:11.441308+0000","last_clean":"2024-05-10T04:39:11.441308+0000","last_became_active":"2024-05-10T04:38:12.800630+0000","last_became_peered":"2024-05-10T04:38:12.800630+0000","last_unstale":"2024-05-10T04:39:11.441308+0000","last_undegraded":"2024-05-10T04:39:11.441308+0000","last_fullsized":"2024-05-10T04:39:11.441308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:05.213620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438080+0000","last_change":"2024-05-10T04:38:08.348764+0000","last_active":"2024-05-10T04:39:11.438080+0000","last_peered":"2024-05-10T04:39:11.438080+0000","last_clean":"2024-05-10T04:39:11.438080+0000","last_became_active":"2024-05-10T04:38:08.347939+0000","last_became_peered":"2024-05-10T04:38:08.347939+0000","last_unstale":"2024-05-10T04:39:11.438080+0000","last_undegraded":"2024-05-10T04:39:11.438080+0000","last_fullsized":"2024-05-10T04:39:11.438080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:01.308255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446594+0000","last_change":"2024-05-10T04:38:08.771382+0000","last_active":"2024-05-10T04:39:11.446594+0000","last_peered":"2024-05-10T04:39:11.446594+0000","last_clean":"2024-05-10T04:39:11.446594+0000","last_became_active":"2024-05-10T04:38:08.771108+0000","last_became_peered":"2024-05-10T04:38:08.771108+0000","last_unstale":"2024-05-10T04:39:11.446594+0000","last_undegraded":"2024-05-10T04:39:11.446594+0000","last_fullsized":"2024-05-10T04:39:11.446594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:32.664640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450061+0000","last_change":"2024-05-10T04:38:12.809582+0000","last_active":"2024-05-10T04:39:11.450061+0000","last_peered":"2024-05-10T04:39:11.450061+0000","last_clean":"2024-05-10T04:39:11.450061+0000","last_became_active":"2024-05-10T04:38:12.809134+0000","last_became_peered":"2024-05-10T04:38:12.809134+0000","last_unstale":"2024-05-10T04:39:11.450061+0000","last_undegraded":"2024-05-10T04:39:11.450061+0000","last_fullsized":"2024-05-10T04:39:11.450061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:31:07.694368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444107+0000","last_change":"2024-05-10T04:38:08.345903+0000","last_active":"2024-05-10T04:39:11.444107+0000","last_peered":"2024-05-10T04:39:11.444107+0000","last_clean":"2024-05-10T04:39:11.444107+0000","last_became_active":"2024-05-10T04:38:08.345371+0000","last_became_peered":"2024-05-10T04:38:08.345371+0000","last_unstale":"2024-05-10T04:39:11.444107+0000","last_undegraded":"2024-05-10T04:39:11.444107+0000","last_fullsized":"2024-05-10T04:39:11.444107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:03.581112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449208+0000","last_change":"2024-05-10T04:38:08.779853+0000","last_active":"2024-05-10T04:39:11.449208+0000","last_peered":"2024-05-10T04:39:11.449208+0000","last_clean":"2024-05-10T04:39:11.449208+0000","last_became_active":"2024-05-10T04:38:08.779531+0000","last_became_peered":"2024-05-10T04:38:08.779531+0000","last_unstale":"2024-05-10T04:39:11.449208+0000","last_undegraded":"2024-05-10T04:39:11.449208+0000","last_fullsized":"2024-05-10T04:39:11.449208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:05.079540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447323+0000","last_change":"2024-05-10T04:38:12.795970+0000","last_active":"2024-05-10T04:39:11.447323+0000","last_peered":"2024-05-10T04:39:11.447323+0000","last_clean":"2024-05-10T04:39:11.447323+0000","last_became_active":"2024-05-10T04:38:12.795497+0000","last_became_peered":"2024-05-10T04:38:12.795497+0000","last_unstale":"2024-05-10T04:39:11.447323+0000","last_undegraded":"2024-05-10T04:39:11.447323+0000","last_fullsized":"2024-05-10T04:39:11.447323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:11.174672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443466+0000","last_change":"2024-05-10T04:38:08.343268+0000","last_active":"2024-05-10T04:39:11.443466+0000","last_peered":"2024-05-10T04:39:11.443466+0000","last_clean":"2024-05-10T04:39:11.443466+0000","last_became_active":"2024-05-10T04:38:08.342952+0000","last_became_peered":"2024-05-10T04:38:08.342952+0000","last_unstale":"2024-05-10T04:39:11.443466+0000","last_undegraded":"2024-05-10T04:39:11.443466+0000","last_fullsized":"2024-05-10T04:39:11.443466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:49.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439365+0000","last_change":"2024-05-10T04:38:08.784276+0000","last_active":"2024-05-10T04:39:11.439365+0000","last_peered":"2024-05-10T04:39:11.439365+0000","last_clean":"2024-05-10T04:39:11.439365+0000","last_became_active":"2024-05-10T04:38:08.783951+0000","last_became_peered":"2024-05-10T04:38:08.783951+0000","last_unstale":"2024-05-10T04:39:11.439365+0000","last_undegraded":"2024-05-10T04:39:11.439365+0000","last_fullsized":"2024-05-10T04:39:11.439365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:45:43.992567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443129+0000","last_change":"2024-05-10T04:38:12.811689+0000","last_active":"2024-05-10T04:39:11.443129+0000","last_peered":"2024-05-10T04:39:11.443129+0000","last_clean":"2024-05-10T04:39:11.443129+0000","last_became_active":"2024-05-10T04:38:12.811138+0000","last_became_peered":"2024-05-10T04:38:12.811138+0000","last_unstale":"2024-05-10T04:39:11.443129+0000","last_undegraded":"2024-05-10T04:39:11.443129+0000","last_fullsized":"2024-05-10T04:39:11.443129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:09.291137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448922+0000","last_change":"2024-05-10T04:38:08.348738+0000","last_active":"2024-05-10T04:39:11.448922+0000","last_peered":"2024-05-10T04:39:11.448922+0000","last_clean":"2024-05-10T04:39:11.448922+0000","last_became_active":"2024-05-10T04:38:08.348382+0000","last_became_peered":"2024-05-10T04:38:08.348382+0000","last_unstale":"2024-05-10T04:39:11.448922+0000","last_undegraded":"2024-05-10T04:39:11.448922+0000","last_fullsized":"2024-05-10T04:39:11.448922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:02.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437662+0000","last_change":"2024-05-10T04:38:08.781483+0000","last_active":"2024-05-10T04:39:11.437662+0000","last_peered":"2024-05-10T04:39:11.437662+0000","last_clean":"2024-05-10T04:39:11.437662+0000","last_became_active":"2024-05-10T04:38:08.777713+0000","last_became_peered":"2024-05-10T04:38:08.777713+0000","last_unstale":"2024-05-10T04:39:11.437662+0000","last_undegraded":"2024-05-10T04:39:11.437662+0000","last_fullsized":"2024-05-10T04:39:11.437662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:41.050901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444353+0000","last_change":"2024-05-10T04:38:12.800656+0000","last_active":"2024-05-10T04:39:11.444353+0000","last_peered":"2024-05-10T04:39:11.444353+0000","last_clean":"2024-05-10T04:39:11.444353+0000","last_became_active":"2024-05-10T04:38:12.800191+0000","last_became_peered":"2024-05-10T04:38:12.800191+0000","last_unstale":"2024-05-10T04:39:11.444353+0000","last_undegraded":"2024-05-10T04:39:11.444353+0000","last_fullsized":"2024-05-10T04:39:11.444353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:47.017642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448670+0000","last_change":"2024-05-10T04:38:08.344017+0000","last_active":"2024-05-10T04:39:11.448670+0000","last_peered":"2024-05-10T04:39:11.448670+0000","last_clean":"2024-05-10T04:39:11.448670+0000","last_became_active":"2024-05-10T04:38:08.343646+0000","last_became_peered":"2024-05-10T04:38:08.343646+0000","last_unstale":"2024-05-10T04:39:11.448670+0000","last_undegraded":"2024-05-10T04:39:11.448670+0000","last_fullsized":"2024-05-10T04:39:11.448670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:02:30.455019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441892+0000","last_change":"2024-05-10T04:38:08.779065+0000","last_active":"2024-05-10T04:39:11.441892+0000","last_peered":"2024-05-10T04:39:11.441892+0000","last_clean":"2024-05-10T04:39:11.441892+0000","last_became_active":"2024-05-10T04:38:08.778685+0000","last_became_peered":"2024-05-10T04:38:08.778685+0000","last_unstale":"2024-05-10T04:39:11.441892+0000","last_undegraded":"2024-05-10T04:39:11.441892+0000","last_fullsized":"2024-05-10T04:39:11.441892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:25:58.780080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444508+0000","last_change":"2024-05-10T04:38:12.801524+0000","last_active":"2024-05-10T04:39:11.444508+0000","last_peered":"2024-05-10T04:39:11.444508+0000","last_clean":"2024-05-10T04:39:11.444508+0000","last_became_active":"2024-05-10T04:38:12.800777+0000","last_became_peered":"2024-05-10T04:38:12.800777+0000","last_unstale":"2024-05-10T04:39:11.444508+0000","last_undegraded":"2024-05-10T04:39:11.444508+0000","last_fullsized":"2024-05-10T04:39:11.444508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:14.139236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447769+0000","last_change":"2024-05-10T04:38:08.344785+0000","last_active":"2024-05-10T04:39:11.447769+0000","last_peered":"2024-05-10T04:39:11.447769+0000","last_clean":"2024-05-10T04:39:11.447769+0000","last_became_active":"2024-05-10T04:38:08.344337+0000","last_became_peered":"2024-05-10T04:38:08.344337+0000","last_unstale":"2024-05-10T04:39:11.447769+0000","last_undegraded":"2024-05-10T04:39:11.447769+0000","last_fullsized":"2024-05-10T04:39:11.447769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:47.682532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448971+0000","last_change":"2024-05-10T04:38:08.768686+0000","last_active":"2024-05-10T04:39:11.448971+0000","last_peered":"2024-05-10T04:39:11.448971+0000","last_clean":"2024-05-10T04:39:11.448971+0000","last_became_active":"2024-05-10T04:38:08.768317+0000","last_became_peered":"2024-05-10T04:38:08.768317+0000","last_unstale":"2024-05-10T04:39:11.448971+0000","last_undegraded":"2024-05-10T04:39:11.448971+0000","last_fullsized":"2024-05-10T04:39:11.448971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:42:33.670912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443063+0000","last_change":"2024-05-10T04:38:12.809825+0000","last_active":"2024-05-10T04:39:11.443063+0000","last_peered":"2024-05-10T04:39:11.443063+0000","last_clean":"2024-05-10T04:39:11.443063+0000","last_became_active":"2024-05-10T04:38:12.809378+0000","last_became_peered":"2024-05-10T04:38:12.809378+0000","last_unstale":"2024-05-10T04:39:11.443063+0000","last_undegraded":"2024-05-10T04:39:11.443063+0000","last_fullsized":"2024-05-10T04:39:11.443063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:28.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438131+0000","last_change":"2024-05-10T04:38:08.347871+0000","last_active":"2024-05-10T04:39:11.438131+0000","last_peered":"2024-05-10T04:39:11.438131+0000","last_clean":"2024-05-10T04:39:11.438131+0000","last_became_active":"2024-05-10T04:38:08.347618+0000","last_became_peered":"2024-05-10T04:38:08.347618+0000","last_unstale":"2024-05-10T04:39:11.438131+0000","last_undegraded":"2024-05-10T04:39:11.438131+0000","last_fullsized":"2024-05-10T04:39:11.438131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:09:42.060082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445616+0000","last_change":"2024-05-10T04:38:08.779191+0000","last_active":"2024-05-10T04:39:11.445616+0000","last_peered":"2024-05-10T04:39:11.445616+0000","last_clean":"2024-05-10T04:39:11.445616+0000","last_became_active":"2024-05-10T04:38:08.778829+0000","last_became_peered":"2024-05-10T04:38:08.778829+0000","last_unstale":"2024-05-10T04:39:11.445616+0000","last_undegraded":"2024-05-10T04:39:11.445616+0000","last_fullsized":"2024-05-10T04:39:11.445616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:07.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448922+0000","last_change":"2024-05-10T04:38:12.788693+0000","last_active":"2024-05-10T04:39:11.448922+0000","last_peered":"2024-05-10T04:39:11.448922+0000","last_clean":"2024-05-10T04:39:11.448922+0000","last_became_active":"2024-05-10T04:38:12.788156+0000","last_became_peered":"2024-05-10T04:38:12.788156+0000","last_unstale":"2024-05-10T04:39:11.448922+0000","last_undegraded":"2024-05-10T04:39:11.448922+0000","last_fullsized":"2024-05-10T04:39:11.448922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:30.119523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:36.487598+0000","last_change":"2024-05-10T04:39:12.455797+0000","last_active":"2024-05-10T04:39:36.487598+0000","last_peered":"2024-05-10T04:39:36.487598+0000","last_clean":"2024-05-10T04:39:36.487598+0000","last_became_active":"2024-05-10T04:39:12.449843+0000","last_became_peered":"2024-05-10T04:39:12.449843+0000","last_unstale":"2024-05-10T04:39:36.487598+0000","last_undegraded":"2024-05-10T04:39:36.487598+0000","last_fullsized":"2024-05-10T04:39:36.487598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:32:17.432455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1515,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446668+0000","last_change":"2024-05-10T04:38:08.780352+0000","last_active":"2024-05-10T04:39:11.446668+0000","last_peered":"2024-05-10T04:39:11.446668+0000","last_clean":"2024-05-10T04:39:11.446668+0000","last_became_active":"2024-05-10T04:38:08.779779+0000","last_became_peered":"2024-05-10T04:38:08.779779+0000","last_unstale":"2024-05-10T04:39:11.446668+0000","last_undegraded":"2024-05-10T04:39:11.446668+0000","last_fullsized":"2024-05-10T04:39:11.446668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:54:31.973896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439035+0000","last_change":"2024-05-10T04:38:12.805310+0000","last_active":"2024-05-10T04:39:11.439035+0000","last_peered":"2024-05-10T04:39:11.439035+0000","last_clean":"2024-05-10T04:39:11.439035+0000","last_became_active":"2024-05-10T04:38:12.804827+0000","last_became_peered":"2024-05-10T04:38:12.804827+0000","last_unstale":"2024-05-10T04:39:11.439035+0000","last_undegraded":"2024-05-10T04:39:11.439035+0000","last_fullsized":"2024-05-10T04:39:11.439035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:01.655788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445754+0000","last_change":"2024-05-10T04:38:08.343813+0000","last_active":"2024-05-10T04:39:11.445754+0000","last_peered":"2024-05-10T04:39:11.445754+0000","last_clean":"2024-05-10T04:39:11.445754+0000","last_became_active":"2024-05-10T04:38:08.343309+0000","last_became_peered":"2024-05-10T04:38:08.343309+0000","last_unstale":"2024-05-10T04:39:11.445754+0000","last_undegraded":"2024-05-10T04:39:11.445754+0000","last_fullsized":"2024-05-10T04:39:11.445754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:07.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445944+0000","last_change":"2024-05-10T04:38:08.780598+0000","last_active":"2024-05-10T04:39:11.445944+0000","last_peered":"2024-05-10T04:39:11.445944+0000","last_clean":"2024-05-10T04:39:11.445944+0000","last_became_active":"2024-05-10T04:38:08.775367+0000","last_became_peered":"2024-05-10T04:38:08.775367+0000","last_unstale":"2024-05-10T04:39:11.445944+0000","last_undegraded":"2024-05-10T04:39:11.445944+0000","last_fullsized":"2024-05-10T04:39:11.445944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:33.826116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444007+0000","last_change":"2024-05-10T04:38:12.795975+0000","last_active":"2024-05-10T04:39:11.444007+0000","last_peered":"2024-05-10T04:39:11.444007+0000","last_clean":"2024-05-10T04:39:11.444007+0000","last_became_active":"2024-05-10T04:38:12.795582+0000","last_became_peered":"2024-05-10T04:38:12.795582+0000","last_unstale":"2024-05-10T04:39:11.444007+0000","last_undegraded":"2024-05-10T04:39:11.444007+0000","last_fullsized":"2024-05-10T04:39:11.444007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:42:30.734833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.766085+0000","last_change":"2024-05-10T04:39:12.766085+0000","last_active":"2024-05-10T04:39:12.766085+0000","last_peered":"2024-05-10T04:39:12.766085+0000","last_clean":"2024-05-10T04:39:12.766085+0000","last_became_active":"2024-05-10T04:39:12.761991+0000","last_became_peered":"2024-05-10T04:39:12.761991+0000","last_unstale":"2024-05-10T04:39:12.766085+0000","last_undegraded":"2024-05-10T04:39:12.766085+0000","last_fullsized":"2024-05-10T04:39:12.766085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:00:52.188593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446261+0000","last_change":"2024-05-10T04:38:08.780925+0000","last_active":"2024-05-10T04:39:11.446261+0000","last_peered":"2024-05-10T04:39:11.446261+0000","last_clean":"2024-05-10T04:39:11.446261+0000","last_became_active":"2024-05-10T04:38:08.778777+0000","last_became_peered":"2024-05-10T04:38:08.778777+0000","last_unstale":"2024-05-10T04:39:11.446261+0000","last_undegraded":"2024-05-10T04:39:11.446261+0000","last_fullsized":"2024-05-10T04:39:11.446261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:57.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442028+0000","last_change":"2024-05-10T04:38:08.780281+0000","last_active":"2024-05-10T04:39:11.442028+0000","last_peered":"2024-05-10T04:39:11.442028+0000","last_clean":"2024-05-10T04:39:11.442028+0000","last_became_active":"2024-05-10T04:38:08.779927+0000","last_became_peered":"2024-05-10T04:38:08.779927+0000","last_unstale":"2024-05-10T04:39:11.442028+0000","last_undegraded":"2024-05-10T04:39:11.442028+0000","last_fullsized":"2024-05-10T04:39:11.442028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:33.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448040+0000","last_change":"2024-05-10T04:38:08.784916+0000","last_active":"2024-05-10T04:39:11.448040+0000","last_peered":"2024-05-10T04:39:11.448040+0000","last_clean":"2024-05-10T04:39:11.448040+0000","last_became_active":"2024-05-10T04:38:08.784544+0000","last_became_peered":"2024-05-10T04:38:08.784544+0000","last_unstale":"2024-05-10T04:39:11.448040+0000","last_undegraded":"2024-05-10T04:39:11.448040+0000","last_fullsized":"2024-05-10T04:39:11.448040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:03.669897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447644+0000","last_change":"2024-05-10T04:38:08.769149+0000","last_active":"2024-05-10T04:39:11.447644+0000","last_peered":"2024-05-10T04:39:11.447644+0000","last_clean":"2024-05-10T04:39:11.447644+0000","last_became_active":"2024-05-10T04:38:08.768808+0000","last_became_peered":"2024-05-10T04:38:08.768808+0000","last_unstale":"2024-05-10T04:39:11.447644+0000","last_undegraded":"2024-05-10T04:39:11.447644+0000","last_fullsized":"2024-05-10T04:39:11.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:42:46.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438965+0000","last_change":"2024-05-10T04:38:08.784411+0000","last_active":"2024-05-10T04:39:11.438965+0000","last_peered":"2024-05-10T04:39:11.438965+0000","last_clean":"2024-05-10T04:39:11.438965+0000","last_became_active":"2024-05-10T04:38:08.784064+0000","last_became_peered":"2024-05-10T04:38:08.784064+0000","last_unstale":"2024-05-10T04:39:11.438965+0000","last_undegraded":"2024-05-10T04:39:11.438965+0000","last_fullsized":"2024-05-10T04:39:11.438965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:32:46.493888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446647+0000","last_change":"2024-05-10T04:38:08.779643+0000","last_active":"2024-05-10T04:39:11.446647+0000","last_peered":"2024-05-10T04:39:11.446647+0000","last_clean":"2024-05-10T04:39:11.446647+0000","last_became_active":"2024-05-10T04:38:08.779291+0000","last_became_peered":"2024-05-10T04:38:08.779291+0000","last_unstale":"2024-05-10T04:39:11.446647+0000","last_undegraded":"2024-05-10T04:39:11.446647+0000","last_fullsized":"2024-05-10T04:39:11.446647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:24.535515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443918+0000","last_change":"2024-05-10T04:38:08.789382+0000","last_active":"2024-05-10T04:39:11.443918+0000","last_peered":"2024-05-10T04:39:11.443918+0000","last_clean":"2024-05-10T04:39:11.443918+0000","last_became_active":"2024-05-10T04:38:08.789003+0000","last_became_peered":"2024-05-10T04:38:08.789003+0000","last_unstale":"2024-05-10T04:39:11.443918+0000","last_undegraded":"2024-05-10T04:39:11.443918+0000","last_fullsized":"2024-05-10T04:39:11.443918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:03.031319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448507+0000","last_change":"2024-05-10T04:38:08.777290+0000","last_active":"2024-05-10T04:39:11.448507+0000","last_peered":"2024-05-10T04:39:11.448507+0000","last_clean":"2024-05-10T04:39:11.448507+0000","last_became_active":"2024-05-10T04:38:08.776331+0000","last_became_peered":"2024-05-10T04:38:08.776331+0000","last_unstale":"2024-05-10T04:39:11.448507+0000","last_undegraded":"2024-05-10T04:39:11.448507+0000","last_fullsized":"2024-05-10T04:39:11.448507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:51.074991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449405+0000","last_change":"2024-05-10T04:38:08.780937+0000","last_active":"2024-05-10T04:39:11.449405+0000","last_peered":"2024-05-10T04:39:11.449405+0000","last_clean":"2024-05-10T04:39:11.449405+0000","last_became_active":"2024-05-10T04:38:08.780620+0000","last_became_peered":"2024-05-10T04:38:08.780620+0000","last_unstale":"2024-05-10T04:39:11.449405+0000","last_undegraded":"2024-05-10T04:39:11.449405+0000","last_fullsized":"2024-05-10T04:39:11.449405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:27.697571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444313+0000","last_change":"2024-05-10T04:38:08.773314+0000","last_active":"2024-05-10T04:39:11.444313+0000","last_peered":"2024-05-10T04:39:11.444313+0000","last_clean":"2024-05-10T04:39:11.444313+0000","last_became_active":"2024-05-10T04:38:08.772973+0000","last_became_peered":"2024-05-10T04:38:08.772973+0000","last_unstale":"2024-05-10T04:39:11.444313+0000","last_undegraded":"2024-05-10T04:39:11.444313+0000","last_fullsized":"2024-05-10T04:39:11.444313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:53.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448852+0000","last_change":"2024-05-10T04:38:08.775632+0000","last_active":"2024-05-10T04:39:11.448852+0000","last_peered":"2024-05-10T04:39:11.448852+0000","last_clean":"2024-05-10T04:39:11.448852+0000","last_became_active":"2024-05-10T04:38:08.775308+0000","last_became_peered":"2024-05-10T04:38:08.775308+0000","last_unstale":"2024-05-10T04:39:11.448852+0000","last_undegraded":"2024-05-10T04:39:11.448852+0000","last_fullsized":"2024-05-10T04:39:11.448852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:58.758863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449579+0000","last_change":"2024-05-10T04:38:08.787354+0000","last_active":"2024-05-10T04:39:11.449579+0000","last_peered":"2024-05-10T04:39:11.449579+0000","last_clean":"2024-05-10T04:39:11.449579+0000","last_became_active":"2024-05-10T04:38:08.786645+0000","last_became_peered":"2024-05-10T04:38:08.786645+0000","last_unstale":"2024-05-10T04:39:11.449579+0000","last_undegraded":"2024-05-10T04:39:11.449579+0000","last_fullsized":"2024-05-10T04:39:11.449579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:35.060271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446479+0000","last_change":"2024-05-10T04:38:08.769233+0000","last_active":"2024-05-10T04:39:11.446479+0000","last_peered":"2024-05-10T04:39:11.446479+0000","last_clean":"2024-05-10T04:39:11.446479+0000","last_became_active":"2024-05-10T04:38:08.768936+0000","last_became_peered":"2024-05-10T04:38:08.768936+0000","last_unstale":"2024-05-10T04:39:11.446479+0000","last_undegraded":"2024-05-10T04:39:11.446479+0000","last_fullsized":"2024-05-10T04:39:11.446479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:08.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443044+0000","last_change":"2024-05-10T04:38:08.788189+0000","last_active":"2024-05-10T04:39:11.443044+0000","last_peered":"2024-05-10T04:39:11.443044+0000","last_clean":"2024-05-10T04:39:11.443044+0000","last_became_active":"2024-05-10T04:38:08.787806+0000","last_became_peered":"2024-05-10T04:38:08.787806+0000","last_unstale":"2024-05-10T04:39:11.443044+0000","last_undegraded":"2024-05-10T04:39:11.443044+0000","last_fullsized":"2024-05-10T04:39:11.443044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:00:51.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280050,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70249,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":639,"num_read_kb":411,"num_write":236,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":219689,"data_compressed":4791,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481821,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":974848,"data_stored":485047,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64600000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78800000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75700000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":0.874,"15min":0.874},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.372,"15min":0.372},"max":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84299999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92200000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.68500000000000005,"15min":0.69599999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89400000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.71099999999999997,"15min":0.72699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":1.02,"15min":1.02},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.391,"15min":1.391},"last":1.0680000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.875},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96799999999999997,"5min":1.829,"15min":1.829},"last":0.94599999999999995}]},{"osd":8,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.107}]},{"osd":9,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1579999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.012,"5min":1.117,"15min":1.117},"last":0.92400000000000004}]},{"osd":10,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.03,"5min":1.601,"15min":1.601},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.70999999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.1319999999999999,"5min":1.774,"15min":1.774},"last":1.1319999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743455,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":1134339,"data_compressed":54829,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.501}]},{"osd":1,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.497,"15min":0.496},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79000000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.34300000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63700000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81999999999999995,"5min":1.143,"15min":1.143},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.52600000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.497,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.625},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":0.872,"15min":0.872},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95599999999999996,"5min":1.252,"15min":1.252},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78000000000000003,"15min":0.77800000000000002},"min":{"1min":0.60499999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.52200000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80600000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":0.85199999999999998},"min":{"1min":0.56899999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":1.296,"15min":1.296},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.139,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90700000000000003,"5min":1.2,"15min":1.2},"last":0.79200000000000004}]},{"osd":11,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76700000000000002}]}]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":488616,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.44600000000000001,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55300000000000005,"5min":1.034,"15min":1.034},"last":0.442},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64900000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.498}]},{"osd":1,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90400000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.56599999999999995,"15min":0.58699999999999997},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.65900000000000003,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59699999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80500000000000005,"5min":1.101,"15min":1.101},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":1.411,"15min":1.411},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.29,"15min":1.29},"last":0.83899999999999997}]},{"osd":8,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93999999999999995,"5min":1.53,"15min":1.53},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91500000000000004,"5min":1.502,"15min":1.502},"last":0.748}]},{"osd":10,"last update":"Fri May 10 04:39:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98199999999999998,"5min":1.016,"15min":1.016},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97299999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78600000000000003}]},{"osd":11,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89800000000000002}]}]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":978944,"data_stored":484786,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.495}]},{"osd":1,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.497,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.443}]},{"osd":3,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.66900000000000004,"15min":0.64400000000000002},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.59399999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.70199999999999996},"min":{"1min":0.59099999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.67400000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82899999999999996}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.77400000000000002,"15min":0.76300000000000001},"min":{"1min":0.61799999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.626},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86299999999999999,"15min":0.85999999999999999},"min":{"1min":0.70899999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.089,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82399999999999995,"15min":0.81599999999999995},"min":{"1min":0.73099999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.83299999999999996,"15min":0.81899999999999995},"min":{"1min":0.754,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78600000000000003}]},{"osd":11,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68799999999999994}]}]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":1040384,"data_stored":1081174,"data_compressed":53232,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.45700000000000002},"min":{"1min":0.33700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56399999999999995,"5min":3.532,"15min":3.532},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.59499999999999997,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.51100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.77400000000000002,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63800000000000001,"15min":0.60299999999999998},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78100000000000003,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.76900000000000002,"5min":1.927,"15min":1.927},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.755,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86499999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.75},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.59299999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94799999999999995,"5min":3.4860000000000002,"15min":3.4860000000000002},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.81000000000000005,"5min":3.996,"15min":3.996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.56399999999999995}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.65300000000000002}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":2.028,"15min":2.028},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91400000000000003},"min":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95299999999999996,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.68400000000000005}]},{"osd":10,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.88500000000000001,"15min":0.90300000000000002},"min":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91100000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.89500000000000002,"15min":0.91100000000000003},"min":{"1min":0.64700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":3.653,"15min":3.653},"last":0.76700000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79800000000000004}]}]},{"osd":6,"up_from":45,"seq":193273528365,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":479976,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.46200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.51900000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.45300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.44400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.69599999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.309,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.376},{"interface":"front","average":{"1min":0.434,"5min":0.5,"15min":0.51900000000000002},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.53700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.439}]},{"osd":4,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.64200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.625,"15min":0.629},"min":{"1min":0.497,"5min":0.437,"15min":0.437},"max":{"1min":0.746,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.63800000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.74099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64100000000000001,"15min":0.65000000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri May 10 04:39:33 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.73399999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri May 10 04:38:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.501},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.623}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.46200000000000002}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.80000000000000004,"5min":1.083,"15min":1.083},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.80300000000000005,"5min":1.105,"15min":1.105},"last":0.60999999999999999}]},{"osd":11,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1036288,"data_stored":548946,"data_compressed":19210,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53500000000000003,"15min":0.5},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri May 10 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.57399999999999995,"15min":0.54200000000000004},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92500000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72399999999999998}]},{"osd":3,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.57299999999999995},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri May 10 04:39:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.497,"15min":0.45000000000000001},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.878,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.82899999999999996,"5min":1.083,"15min":1.083},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.58499999999999996,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":1.123,"15min":1.123},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.69199999999999995,"15min":0.64900000000000002},"min":{"1min":0.55100000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1100000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.502,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.623},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.55500000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.89500000000000002}]},{"osd":7,"last update":"Fri May 10 04:39:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.374,"15min":0.374},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.67600000000000005,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99199999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.65400000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.52800000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.626,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1040000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.81599999999999995}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.58399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78400000000000003,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0049999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77700000000000002,"15min":0.77000000000000002},"min":{"1min":0.56399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.72499999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.097,"5min":1.163,"15min":1.163},"last":0.58499999999999996}]},{"osd":11,"last update":"Fri May 10 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":481825,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.673,"5min":2.673,"15min":2.673},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.55800000000000005,"15min":0.48699999999999999},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.307}]},{"osd":2,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.41599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.41699999999999998}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.33000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.38600000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.47599999999999998}]},{"osd":8,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.751}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69599999999999995,"15min":0.67300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.7190000000000003,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.628}]},{"osd":10,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.70599999999999996,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.753,"15min":0.72599999999999998},"min":{"1min":0.60599999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":0.57199999999999995}]},{"osd":11,"last update":"Fri May 10 04:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.752,"15min":0.72399999999999998},"min":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.72599999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":547996,"data_compressed":19210,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.48199999999999998},"min":{"1min":0.45500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76100000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.503,"15min":0.46400000000000002},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri May 10 04:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.60999999999999999,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.94699999999999995}]},{"osd":3,"last update":"Fri May 10 04:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.84999999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.59999999999999998,"15min":0.55700000000000005},"min":{"1min":0.54400000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.45800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89900000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.022}]},{"osd":7,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.095}]},{"osd":8,"last update":"Fri May 10 04:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.91300000000000003}]},{"osd":9,"last update":"Fri May 10 04:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.60599999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0249999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.57999999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri May 10 04:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.70199999999999996,"5min":0.438,"15min":0.438},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.999}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":479732,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.372,"15min":0.34399999999999997},"min":{"1min":0.29999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.29199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54100000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.83599999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.52600000000000002},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri May 10 04:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.46000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82199999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri May 10 04:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.58099999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70099999999999996}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91600000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68400000000000005}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77500000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.72399999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":485879,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":1.143,"15min":1.143},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.36099999999999999},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.38200000000000001,"15min":0.36199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.627,"5min":0.748,"15min":0.748},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.623,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":1.03,"15min":1.03},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.85999999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88300000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90700000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.50700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.069,"15min":1.069},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.57199999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.871,"5min":1.079,"15min":1.079},"last":1.133}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63300000000000001,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.378,"15min":0.378},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0640000000000001}]},{"osd":8,"last update":"Fri May 10 04:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.52800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88500000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.99099999999999999}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.80300000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.032},{"interface":"front","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":1.1859999999999999}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.71799999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0069999999999999,"5min":1.012,"15min":1.012},"last":1.165},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.1539999999999999}]},{"osd":11,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.95299999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":1048576,"data_stored":1076538,"data_compressed":53232,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68999999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85499999999999998,"5min":1.008,"15min":1.008},"last":0.63}]},{"osd":4,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81599999999999995,"5min":1.089,"15min":1.089},"last":0.749},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84499999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.503},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.51500000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.002},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.56899999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90400000000000003,"5min":1.054,"15min":1.054},"last":0.877},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64300000000000002,"15min":0.63100000000000001},"min":{"1min":0.60399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.73499999999999999}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri May 10 04:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92000000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.84499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69644,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68694,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65000,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:39:43.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 -- 172.21.15.150:0/936504779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c102580 msgr2=0x7f886c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 --2- 172.21.15.150:0/936504779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c102580 0x7f886c108e30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f886400b0a0 tx=0x7f886400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 -- 172.21.15.150:0/936504779 shutdown_connections 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 --2- 172.21.15.150:0/936504779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c102580 0x7f886c108e30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 --2- 172.21.15.150:0/936504779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c100ac0 0x7f886c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 --2- 172.21.15.150:0/936504779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c101be0 0x7f886c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 -- 172.21.15.150:0/936504779 >> 172.21.15.150:0/936504779 conn(0x7f886c0fc0d0 msgr2=0x7f886c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 -- 172.21.15.150:0/936504779 shutdown_connections 2024-05-10T04:39:43.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 -- 172.21.15.150:0/936504779 wait complete. 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 Processor -- start 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.500+0000 7f8872e8e700 1 -- start start 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8872e8e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c100ac0 0x7f886c068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8872e8e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c101be0 0x7f886c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8872e8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c102580 0x7f886c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8872e8e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c19fb90 con 0x7f886c101be0 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8872e8e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c19fd00 con 0x7f886c100ac0 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8872e8e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c19fec0 con 0x7f886c102580 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c101be0 0x7f886c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c101be0 0x7f886c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47980/0 (socket says 172.21.15.150:47980) 2024-05-10T04:39:43.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 -- 172.21.15.150:0/855464792 learned_addr learned my addr 172.21.15.150:0/855464792 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8870c2a700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c100ac0 0x7f886c068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f887142b700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c102580 0x7f886c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 -- 172.21.15.150:0/855464792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c102580 msgr2=0x7f886c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c102580 0x7f886c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 -- 172.21.15.150:0/855464792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c100ac0 msgr2=0x7f886c068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c100ac0 0x7f886c068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 -- 172.21.15.150:0/855464792 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8864009d00 con 0x7f886c101be0 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f8870c2a700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c100ac0 0x7f886c068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.501+0000 7f886bfff700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c101be0 0x7f886c19ce00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f885800c920 tx=0x7f885800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.502+0000 7f8869ffb700 1 -- 172.21.15.150:0/855464792 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858007b00 con 0x7f886c101be0 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.502+0000 7f8869ffb700 1 -- 172.21.15.150:0/855464792 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f885800ce70 con 0x7f886c101be0 2024-05-10T04:39:43.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.502+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c1a0140 con 0x7f886c101be0 2024-05-10T04:39:43.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.502+0000 7f8869ffb700 1 -- 172.21.15.150:0/855464792 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88580054e0 con 0x7f886c101be0 2024-05-10T04:39:43.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.502+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886c1a0650 con 0x7f886c101be0 2024-05-10T04:39:43.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.503+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c107cd0 con 0x7f886c101be0 2024-05-10T04:39:43.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.504+0000 7f8869ffb700 1 -- 172.21.15.150:0/855464792 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8858004750 con 0x7f886c101be0 2024-05-10T04:39:43.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.504+0000 7f8869ffb700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f885c0a11b0 0x7f885c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.505+0000 7f8869ffb700 1 -- 172.21.15.150:0/855464792 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f88580cc930 con 0x7f886c101be0 2024-05-10T04:39:43.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.505+0000 7f8870c2a700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f885c0a11b0 0x7f885c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.505+0000 7f8870c2a700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f885c0a11b0 0x7f885c0a36c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f886c101ae0 tx=0x7f8860009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:43.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.509+0000 7f8869ffb700 1 -- 172.21.15.150:0/855464792 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8858095c50 con 0x7f886c101be0 2024-05-10T04:39:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:43 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2502446992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:43 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3521362722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:43 smithi150 ceph-mon[42404]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:43.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:43 smithi150 ceph-mon[42404]: pgmap v274: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-10T04:39:43.702 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.702+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f886c069470 con 0x7f885c0a11b0 2024-05-10T04:39:43.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.714+0000 7f8869ffb700 1 -- 172.21.15.150:0/855464792 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427129 (secure 0 0 0) 0x7f886c069470 con 0x7f885c0a11b0 2024-05-10T04:39:43.719 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f885c0a11b0 msgr2=0x7f885c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f885c0a11b0 0x7f885c0a36c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f886c101ae0 tx=0x7f8860009260 comp rx=0 tx=0).stop 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c101be0 msgr2=0x7f886c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c101be0 0x7f886c19ce00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f885800c920 tx=0x7f885800cc30 comp rx=0 tx=0).stop 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 shutdown_connections 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f885c0a11b0 0x7f885c0a36c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c102580 0x7f886c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c100ac0 0x7f886c068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 --2- 172.21.15.150:0/855464792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f886c101be0 0x7f886c19ce00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 >> 172.21.15.150:0/855464792 conn(0x7f886c0fc0d0 msgr2=0x7f886c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:43.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 shutdown_connections 2024-05-10T04:39:43.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.718+0000 7f8872e8e700 1 -- 172.21.15.150:0/855464792 wait complete. 2024-05-10T04:39:43.724 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:39:43.732 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-05-10T04:39:43.330411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665479,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":791,"num_read_kb":545,"num_write":423,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182363,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360692,"kb_used_data":11872,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581132,"statfs":{"total":1151940427776,"available":1151571079168,"internally_reserved":0,"allocated":12156928,"data_stored":7779796,"data_compressed":323004,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19127,"internal_metadata":357021001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4942,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":76,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003958"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445850+0000","last_change":"2024-05-10T04:38:08.784019+0000","last_active":"2024-05-10T04:39:11.445850+0000","last_peered":"2024-05-10T04:39:11.445850+0000","last_clean":"2024-05-10T04:39:11.445850+0000","last_became_active":"2024-05-10T04:38:08.783585+0000","last_became_peered":"2024-05-10T04:38:08.783585+0000","last_unstale":"2024-05-10T04:39:11.445850+0000","last_undegraded":"2024-05-10T04:39:11.445850+0000","last_fullsized":"2024-05-10T04:39:11.445850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:56.963068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439378+0000","last_change":"2024-05-10T04:38:08.777364+0000","last_active":"2024-05-10T04:39:11.439378+0000","last_peered":"2024-05-10T04:39:11.439378+0000","last_clean":"2024-05-10T04:39:11.439378+0000","last_became_active":"2024-05-10T04:38:08.776488+0000","last_became_peered":"2024-05-10T04:38:08.776488+0000","last_unstale":"2024-05-10T04:39:11.439378+0000","last_undegraded":"2024-05-10T04:39:11.439378+0000","last_fullsized":"2024-05-10T04:39:11.439378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:40.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439228+0000","last_change":"2024-05-10T04:38:08.777169+0000","last_active":"2024-05-10T04:39:11.439228+0000","last_peered":"2024-05-10T04:39:11.439228+0000","last_clean":"2024-05-10T04:39:11.439228+0000","last_became_active":"2024-05-10T04:38:08.776713+0000","last_became_peered":"2024-05-10T04:38:08.776713+0000","last_unstale":"2024-05-10T04:39:11.439228+0000","last_undegraded":"2024-05-10T04:39:11.439228+0000","last_fullsized":"2024-05-10T04:39:11.439228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:08:41.947577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443030+0000","last_change":"2024-05-10T04:38:08.788464+0000","last_active":"2024-05-10T04:39:11.443030+0000","last_peered":"2024-05-10T04:39:11.443030+0000","last_clean":"2024-05-10T04:39:11.443030+0000","last_became_active":"2024-05-10T04:38:08.787811+0000","last_became_peered":"2024-05-10T04:38:08.787811+0000","last_unstale":"2024-05-10T04:39:11.443030+0000","last_undegraded":"2024-05-10T04:39:11.443030+0000","last_fullsized":"2024-05-10T04:39:11.443030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:22.574882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437946+0000","last_change":"2024-05-10T04:38:08.772904+0000","last_active":"2024-05-10T04:39:11.437946+0000","last_peered":"2024-05-10T04:39:11.437946+0000","last_clean":"2024-05-10T04:39:11.437946+0000","last_became_active":"2024-05-10T04:38:08.772503+0000","last_became_peered":"2024-05-10T04:38:08.772503+0000","last_unstale":"2024-05-10T04:39:11.437946+0000","last_undegraded":"2024-05-10T04:39:11.437946+0000","last_fullsized":"2024-05-10T04:39:11.437946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:58.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442107+0000","last_change":"2024-05-10T04:38:08.779442+0000","last_active":"2024-05-10T04:39:11.442107+0000","last_peered":"2024-05-10T04:39:11.442107+0000","last_clean":"2024-05-10T04:39:11.442107+0000","last_became_active":"2024-05-10T04:38:08.777348+0000","last_became_peered":"2024-05-10T04:38:08.777348+0000","last_unstale":"2024-05-10T04:39:11.442107+0000","last_undegraded":"2024-05-10T04:39:11.442107+0000","last_fullsized":"2024-05-10T04:39:11.442107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:25.619031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446022+0000","last_change":"2024-05-10T04:38:08.775646+0000","last_active":"2024-05-10T04:39:11.446022+0000","last_peered":"2024-05-10T04:39:11.446022+0000","last_clean":"2024-05-10T04:39:11.446022+0000","last_became_active":"2024-05-10T04:38:08.768229+0000","last_became_peered":"2024-05-10T04:38:08.768229+0000","last_unstale":"2024-05-10T04:39:11.446022+0000","last_undegraded":"2024-05-10T04:39:11.446022+0000","last_fullsized":"2024-05-10T04:39:11.446022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:14.653065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450030+0000","last_change":"2024-05-10T04:38:08.771012+0000","last_active":"2024-05-10T04:39:11.450030+0000","last_peered":"2024-05-10T04:39:11.450030+0000","last_clean":"2024-05-10T04:39:11.450030+0000","last_became_active":"2024-05-10T04:38:08.770674+0000","last_became_peered":"2024-05-10T04:38:08.770674+0000","last_unstale":"2024-05-10T04:39:11.450030+0000","last_undegraded":"2024-05-10T04:39:11.450030+0000","last_fullsized":"2024-05-10T04:39:11.450030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:14.389263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450011+0000","last_change":"2024-05-10T04:38:08.782109+0000","last_active":"2024-05-10T04:39:11.450011+0000","last_peered":"2024-05-10T04:39:11.450011+0000","last_clean":"2024-05-10T04:39:11.450011+0000","last_became_active":"2024-05-10T04:38:08.781717+0000","last_became_peered":"2024-05-10T04:38:08.781717+0000","last_unstale":"2024-05-10T04:39:11.450011+0000","last_undegraded":"2024-05-10T04:39:11.450011+0000","last_fullsized":"2024-05-10T04:39:11.450011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:17.505448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442749+0000","last_change":"2024-05-10T04:38:08.778429+0000","last_active":"2024-05-10T04:39:11.442749+0000","last_peered":"2024-05-10T04:39:11.442749+0000","last_clean":"2024-05-10T04:39:11.442749+0000","last_became_active":"2024-05-10T04:38:08.777874+0000","last_became_peered":"2024-05-10T04:38:08.777874+0000","last_unstale":"2024-05-10T04:39:11.442749+0000","last_undegraded":"2024-05-10T04:39:11.442749+0000","last_fullsized":"2024-05-10T04:39:11.442749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:20:19.704678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446510+0000","last_change":"2024-05-10T04:38:08.780104+0000","last_active":"2024-05-10T04:39:11.446510+0000","last_peered":"2024-05-10T04:39:11.446510+0000","last_clean":"2024-05-10T04:39:11.446510+0000","last_became_active":"2024-05-10T04:38:08.779298+0000","last_became_peered":"2024-05-10T04:38:08.779298+0000","last_unstale":"2024-05-10T04:39:11.446510+0000","last_undegraded":"2024-05-10T04:39:11.446510+0000","last_fullsized":"2024-05-10T04:39:11.446510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:41.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443351+0000","last_change":"2024-05-10T04:38:08.788605+0000","last_active":"2024-05-10T04:39:11.443351+0000","last_peered":"2024-05-10T04:39:11.443351+0000","last_clean":"2024-05-10T04:39:11.443351+0000","last_became_active":"2024-05-10T04:38:08.788207+0000","last_became_peered":"2024-05-10T04:38:08.788207+0000","last_unstale":"2024-05-10T04:39:11.443351+0000","last_undegraded":"2024-05-10T04:39:11.443351+0000","last_fullsized":"2024-05-10T04:39:11.443351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:35.000980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446423+0000","last_change":"2024-05-10T04:38:08.761953+0000","last_active":"2024-05-10T04:39:11.446423+0000","last_peered":"2024-05-10T04:39:11.446423+0000","last_clean":"2024-05-10T04:39:11.446423+0000","last_became_active":"2024-05-10T04:38:08.761585+0000","last_became_peered":"2024-05-10T04:38:08.761585+0000","last_unstale":"2024-05-10T04:39:11.446423+0000","last_undegraded":"2024-05-10T04:39:11.446423+0000","last_fullsized":"2024-05-10T04:39:11.446423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:25.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439022+0000","last_change":"2024-05-10T04:38:08.777656+0000","last_active":"2024-05-10T04:39:11.439022+0000","last_peered":"2024-05-10T04:39:11.439022+0000","last_clean":"2024-05-10T04:39:11.439022+0000","last_became_active":"2024-05-10T04:38:08.777332+0000","last_became_peered":"2024-05-10T04:38:08.777332+0000","last_unstale":"2024-05-10T04:39:11.439022+0000","last_undegraded":"2024-05-10T04:39:11.439022+0000","last_fullsized":"2024-05-10T04:39:11.439022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:17:52.878943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438927+0000","last_change":"2024-05-10T04:38:08.784934+0000","last_active":"2024-05-10T04:39:11.438927+0000","last_peered":"2024-05-10T04:39:11.438927+0000","last_clean":"2024-05-10T04:39:11.438927+0000","last_became_active":"2024-05-10T04:38:08.784644+0000","last_became_peered":"2024-05-10T04:38:08.784644+0000","last_unstale":"2024-05-10T04:39:11.438927+0000","last_undegraded":"2024-05-10T04:39:11.438927+0000","last_fullsized":"2024-05-10T04:39:11.438927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:43.889305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448828+0000","last_change":"2024-05-10T04:38:08.784110+0000","last_active":"2024-05-10T04:39:11.448828+0000","last_peered":"2024-05-10T04:39:11.448828+0000","last_clean":"2024-05-10T04:39:11.448828+0000","last_became_active":"2024-05-10T04:38:08.783779+0000","last_became_peered":"2024-05-10T04:38:08.783779+0000","last_unstale":"2024-05-10T04:39:11.448828+0000","last_undegraded":"2024-05-10T04:39:11.448828+0000","last_fullsized":"2024-05-10T04:39:11.448828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:24.233024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445229+0000","last_change":"2024-05-10T04:38:08.788685+0000","last_active":"2024-05-10T04:39:11.445229+0000","last_peered":"2024-05-10T04:39:11.445229+0000","last_clean":"2024-05-10T04:39:11.445229+0000","last_became_active":"2024-05-10T04:38:08.788122+0000","last_became_peered":"2024-05-10T04:38:08.788122+0000","last_unstale":"2024-05-10T04:39:11.445229+0000","last_undegraded":"2024-05-10T04:39:11.445229+0000","last_fullsized":"2024-05-10T04:39:11.445229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:21:01.098784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444179+0000","last_change":"2024-05-10T04:38:08.776330+0000","last_active":"2024-05-10T04:39:11.444179+0000","last_peered":"2024-05-10T04:39:11.444179+0000","last_clean":"2024-05-10T04:39:11.444179+0000","last_became_active":"2024-05-10T04:38:08.776005+0000","last_became_peered":"2024-05-10T04:38:08.776005+0000","last_unstale":"2024-05-10T04:39:11.444179+0000","last_undegraded":"2024-05-10T04:39:11.444179+0000","last_fullsized":"2024-05-10T04:39:11.444179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:46:14.407766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449082+0000","last_change":"2024-05-10T04:38:08.772811+0000","last_active":"2024-05-10T04:39:11.449082+0000","last_peered":"2024-05-10T04:39:11.449082+0000","last_clean":"2024-05-10T04:39:11.449082+0000","last_became_active":"2024-05-10T04:38:08.772462+0000","last_became_peered":"2024-05-10T04:38:08.772462+0000","last_unstale":"2024-05-10T04:39:11.449082+0000","last_undegraded":"2024-05-10T04:39:11.449082+0000","last_fullsized":"2024-05-10T04:39:11.449082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:39.703137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447478+0000","last_change":"2024-05-10T04:38:12.799483+0000","last_active":"2024-05-10T04:39:11.447478+0000","last_peered":"2024-05-10T04:39:11.447478+0000","last_clean":"2024-05-10T04:39:11.447478+0000","last_became_active":"2024-05-10T04:38:12.799049+0000","last_became_peered":"2024-05-10T04:38:12.799049+0000","last_unstale":"2024-05-10T04:39:11.447478+0000","last_undegraded":"2024-05-10T04:39:11.447478+0000","last_fullsized":"2024-05-10T04:39:11.447478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:00.380645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:39.065705+0000","last_change":"2024-05-10T04:38:08.345935+0000","last_active":"2024-05-10T04:39:39.065705+0000","last_peered":"2024-05-10T04:39:39.065705+0000","last_clean":"2024-05-10T04:39:39.065705+0000","last_became_active":"2024-05-10T04:38:08.345580+0000","last_became_peered":"2024-05-10T04:38:08.345580+0000","last_unstale":"2024-05-10T04:39:39.065705+0000","last_undegraded":"2024-05-10T04:39:39.065705+0000","last_fullsized":"2024-05-10T04:39:39.065705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:11.527710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449675+0000","last_change":"2024-05-10T04:38:08.771072+0000","last_active":"2024-05-10T04:39:11.449675+0000","last_peered":"2024-05-10T04:39:11.449675+0000","last_clean":"2024-05-10T04:39:11.449675+0000","last_became_active":"2024-05-10T04:38:08.770711+0000","last_became_peered":"2024-05-10T04:38:08.770711+0000","last_unstale":"2024-05-10T04:39:11.449675+0000","last_undegraded":"2024-05-10T04:39:11.449675+0000","last_fullsized":"2024-05-10T04:39:11.449675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:44.803219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.452968+0000","last_change":"2024-05-10T04:39:12.452968+0000","last_active":"2024-05-10T04:39:12.452968+0000","last_peered":"2024-05-10T04:39:12.452968+0000","last_clean":"2024-05-10T04:39:12.452968+0000","last_became_active":"2024-05-10T04:39:12.452621+0000","last_became_peered":"2024-05-10T04:39:12.452621+0000","last_unstale":"2024-05-10T04:39:12.452968+0000","last_undegraded":"2024-05-10T04:39:12.452968+0000","last_fullsized":"2024-05-10T04:39:12.452968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:29.312810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449726+0000","last_change":"2024-05-10T04:38:08.348806+0000","last_active":"2024-05-10T04:39:11.449726+0000","last_peered":"2024-05-10T04:39:11.449726+0000","last_clean":"2024-05-10T04:39:11.449726+0000","last_became_active":"2024-05-10T04:38:08.348513+0000","last_became_peered":"2024-05-10T04:38:08.348513+0000","last_unstale":"2024-05-10T04:39:11.449726+0000","last_undegraded":"2024-05-10T04:39:11.449726+0000","last_fullsized":"2024-05-10T04:39:11.449726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:45.474403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439225+0000","last_change":"2024-05-10T04:38:08.782738+0000","last_active":"2024-05-10T04:39:11.439225+0000","last_peered":"2024-05-10T04:39:11.439225+0000","last_clean":"2024-05-10T04:39:11.439225+0000","last_became_active":"2024-05-10T04:38:08.782375+0000","last_became_peered":"2024-05-10T04:38:08.782375+0000","last_unstale":"2024-05-10T04:39:11.439225+0000","last_undegraded":"2024-05-10T04:39:11.439225+0000","last_fullsized":"2024-05-10T04:39:11.439225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:16.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442600+0000","last_change":"2024-05-10T04:38:12.801322+0000","last_active":"2024-05-10T04:39:11.442600+0000","last_peered":"2024-05-10T04:39:11.442600+0000","last_clean":"2024-05-10T04:39:11.442600+0000","last_became_active":"2024-05-10T04:38:12.800388+0000","last_became_peered":"2024-05-10T04:38:12.800388+0000","last_unstale":"2024-05-10T04:39:11.442600+0000","last_undegraded":"2024-05-10T04:39:11.442600+0000","last_fullsized":"2024-05-10T04:39:11.442600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:57.678691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438911+0000","last_change":"2024-05-10T04:38:08.347997+0000","last_active":"2024-05-10T04:39:11.438911+0000","last_peered":"2024-05-10T04:39:11.438911+0000","last_clean":"2024-05-10T04:39:11.438911+0000","last_became_active":"2024-05-10T04:38:08.347469+0000","last_became_peered":"2024-05-10T04:38:08.347469+0000","last_unstale":"2024-05-10T04:39:11.438911+0000","last_undegraded":"2024-05-10T04:39:11.438911+0000","last_fullsized":"2024-05-10T04:39:11.438911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:39.122148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449614+0000","last_change":"2024-05-10T04:38:08.781068+0000","last_active":"2024-05-10T04:39:11.449614+0000","last_peered":"2024-05-10T04:39:11.449614+0000","last_clean":"2024-05-10T04:39:11.449614+0000","last_became_active":"2024-05-10T04:38:08.780671+0000","last_became_peered":"2024-05-10T04:38:08.780671+0000","last_unstale":"2024-05-10T04:39:11.449614+0000","last_undegraded":"2024-05-10T04:39:11.449614+0000","last_fullsized":"2024-05-10T04:39:11.449614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442827+0000","last_change":"2024-05-10T04:38:12.801702+0000","last_active":"2024-05-10T04:39:11.442827+0000","last_peered":"2024-05-10T04:39:11.442827+0000","last_clean":"2024-05-10T04:39:11.442827+0000","last_became_active":"2024-05-10T04:38:12.801031+0000","last_became_peered":"2024-05-10T04:38:12.801031+0000","last_unstale":"2024-05-10T04:39:11.442827+0000","last_undegraded":"2024-05-10T04:39:11.442827+0000","last_fullsized":"2024-05-10T04:39:11.442827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:05:07.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437865+0000","last_change":"2024-05-10T04:38:08.347981+0000","last_active":"2024-05-10T04:39:11.437865+0000","last_peered":"2024-05-10T04:39:11.437865+0000","last_clean":"2024-05-10T04:39:11.437865+0000","last_became_active":"2024-05-10T04:38:08.346915+0000","last_became_peered":"2024-05-10T04:38:08.346915+0000","last_unstale":"2024-05-10T04:39:11.437865+0000","last_undegraded":"2024-05-10T04:39:11.437865+0000","last_fullsized":"2024-05-10T04:39:11.437865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:52.383734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444542+0000","last_change":"2024-05-10T04:38:08.779164+0000","last_active":"2024-05-10T04:39:11.444542+0000","last_peered":"2024-05-10T04:39:11.444542+0000","last_clean":"2024-05-10T04:39:11.444542+0000","last_became_active":"2024-05-10T04:38:08.778851+0000","last_became_peered":"2024-05-10T04:38:08.778851+0000","last_unstale":"2024-05-10T04:39:11.444542+0000","last_undegraded":"2024-05-10T04:39:11.444542+0000","last_fullsized":"2024-05-10T04:39:11.444542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:01.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.451673+0000","last_change":"2024-05-10T04:39:12.451673+0000","last_active":"2024-05-10T04:39:12.451673+0000","last_peered":"2024-05-10T04:39:12.451673+0000","last_clean":"2024-05-10T04:39:12.451673+0000","last_became_active":"2024-05-10T04:39:12.451181+0000","last_became_peered":"2024-05-10T04:39:12.451181+0000","last_unstale":"2024-05-10T04:39:12.451673+0000","last_undegraded":"2024-05-10T04:39:12.451673+0000","last_fullsized":"2024-05-10T04:39:12.451673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:46:30.325584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:39.041629+0000","last_change":"2024-05-10T04:38:08.341633+0000","last_active":"2024-05-10T04:39:39.041629+0000","last_peered":"2024-05-10T04:39:39.041629+0000","last_clean":"2024-05-10T04:39:39.041629+0000","last_became_active":"2024-05-10T04:38:08.341220+0000","last_became_peered":"2024-05-10T04:38:08.341220+0000","last_unstale":"2024-05-10T04:39:39.041629+0000","last_undegraded":"2024-05-10T04:39:39.041629+0000","last_fullsized":"2024-05-10T04:39:39.041629+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:36.861191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437912+0000","last_change":"2024-05-10T04:38:08.790969+0000","last_active":"2024-05-10T04:39:11.437912+0000","last_peered":"2024-05-10T04:39:11.437912+0000","last_clean":"2024-05-10T04:39:11.437912+0000","last_became_active":"2024-05-10T04:38:08.790539+0000","last_became_peered":"2024-05-10T04:38:08.790539+0000","last_unstale":"2024-05-10T04:39:11.437912+0000","last_undegraded":"2024-05-10T04:39:11.437912+0000","last_fullsized":"2024-05-10T04:39:11.437912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:50.174798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.453441+0000","last_change":"2024-05-10T04:39:12.453441+0000","last_active":"2024-05-10T04:39:12.453441+0000","last_peered":"2024-05-10T04:39:12.453441+0000","last_clean":"2024-05-10T04:39:12.453441+0000","last_became_active":"2024-05-10T04:39:12.452879+0000","last_became_peered":"2024-05-10T04:39:12.452879+0000","last_unstale":"2024-05-10T04:39:12.453441+0000","last_undegraded":"2024-05-10T04:39:12.453441+0000","last_fullsized":"2024-05-10T04:39:12.453441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:06.118264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442029+0000","last_change":"2024-05-10T04:38:08.342449+0000","last_active":"2024-05-10T04:39:11.442029+0000","last_peered":"2024-05-10T04:39:11.442029+0000","last_clean":"2024-05-10T04:39:11.442029+0000","last_became_active":"2024-05-10T04:38:08.342043+0000","last_became_peered":"2024-05-10T04:38:08.342043+0000","last_unstale":"2024-05-10T04:39:11.442029+0000","last_undegraded":"2024-05-10T04:39:11.442029+0000","last_fullsized":"2024-05-10T04:39:11.442029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:31.161203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444956+0000","last_change":"2024-05-10T04:38:08.783759+0000","last_active":"2024-05-10T04:39:11.444956+0000","last_peered":"2024-05-10T04:39:11.444956+0000","last_clean":"2024-05-10T04:39:11.444956+0000","last_became_active":"2024-05-10T04:38:08.783419+0000","last_became_peered":"2024-05-10T04:38:08.783419+0000","last_unstale":"2024-05-10T04:39:11.444956+0000","last_undegraded":"2024-05-10T04:39:11.444956+0000","last_fullsized":"2024-05-10T04:39:11.444956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:16.114017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441404+0000","last_change":"2024-05-10T04:38:12.803018+0000","last_active":"2024-05-10T04:39:11.441404+0000","last_peered":"2024-05-10T04:39:11.441404+0000","last_clean":"2024-05-10T04:39:11.441404+0000","last_became_active":"2024-05-10T04:38:12.802546+0000","last_became_peered":"2024-05-10T04:38:12.802546+0000","last_unstale":"2024-05-10T04:39:11.441404+0000","last_undegraded":"2024-05-10T04:39:11.441404+0000","last_fullsized":"2024-05-10T04:39:11.441404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:40.676335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446042+0000","last_change":"2024-05-10T04:38:08.341404+0000","last_active":"2024-05-10T04:39:11.446042+0000","last_peered":"2024-05-10T04:39:11.446042+0000","last_clean":"2024-05-10T04:39:11.446042+0000","last_became_active":"2024-05-10T04:38:08.341013+0000","last_became_peered":"2024-05-10T04:38:08.341013+0000","last_unstale":"2024-05-10T04:39:11.446042+0000","last_undegraded":"2024-05-10T04:39:11.446042+0000","last_fullsized":"2024-05-10T04:39:11.446042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:43.513989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450121+0000","last_change":"2024-05-10T04:38:08.775650+0000","last_active":"2024-05-10T04:39:11.450121+0000","last_peered":"2024-05-10T04:39:11.450121+0000","last_clean":"2024-05-10T04:39:11.450121+0000","last_became_active":"2024-05-10T04:38:08.775320+0000","last_became_peered":"2024-05-10T04:38:08.775320+0000","last_unstale":"2024-05-10T04:39:11.450121+0000","last_undegraded":"2024-05-10T04:39:11.450121+0000","last_fullsized":"2024-05-10T04:39:11.450121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:49:07.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439300+0000","last_change":"2024-05-10T04:38:12.797168+0000","last_active":"2024-05-10T04:39:11.439300+0000","last_peered":"2024-05-10T04:39:11.439300+0000","last_clean":"2024-05-10T04:39:11.439300+0000","last_became_active":"2024-05-10T04:38:12.796776+0000","last_became_peered":"2024-05-10T04:38:12.796776+0000","last_unstale":"2024-05-10T04:39:11.439300+0000","last_undegraded":"2024-05-10T04:39:11.439300+0000","last_fullsized":"2024-05-10T04:39:11.439300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:59:02.363398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444198+0000","last_change":"2024-05-10T04:38:08.344086+0000","last_active":"2024-05-10T04:39:11.444198+0000","last_peered":"2024-05-10T04:39:11.444198+0000","last_clean":"2024-05-10T04:39:11.444198+0000","last_became_active":"2024-05-10T04:38:08.343744+0000","last_became_peered":"2024-05-10T04:38:08.343744+0000","last_unstale":"2024-05-10T04:39:11.444198+0000","last_undegraded":"2024-05-10T04:39:11.444198+0000","last_fullsized":"2024-05-10T04:39:11.444198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:01.800590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443135+0000","last_change":"2024-05-10T04:38:08.773638+0000","last_active":"2024-05-10T04:39:11.443135+0000","last_peered":"2024-05-10T04:39:11.443135+0000","last_clean":"2024-05-10T04:39:11.443135+0000","last_became_active":"2024-05-10T04:38:08.773218+0000","last_became_peered":"2024-05-10T04:38:08.773218+0000","last_unstale":"2024-05-10T04:39:11.443135+0000","last_undegraded":"2024-05-10T04:39:11.443135+0000","last_fullsized":"2024-05-10T04:39:11.443135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:54:38.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448821+0000","last_change":"2024-05-10T04:38:12.796344+0000","last_active":"2024-05-10T04:39:11.448821+0000","last_peered":"2024-05-10T04:39:11.448821+0000","last_clean":"2024-05-10T04:39:11.448821+0000","last_became_active":"2024-05-10T04:38:12.795958+0000","last_became_peered":"2024-05-10T04:38:12.795958+0000","last_unstale":"2024-05-10T04:39:11.448821+0000","last_undegraded":"2024-05-10T04:39:11.448821+0000","last_fullsized":"2024-05-10T04:39:11.448821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:45.708078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":158,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:41.495403+0000","last_change":"2024-05-10T04:38:08.341327+0000","last_active":"2024-05-10T04:39:41.495403+0000","last_peered":"2024-05-10T04:39:41.495403+0000","last_clean":"2024-05-10T04:39:41.495403+0000","last_became_active":"2024-05-10T04:38:08.339606+0000","last_became_peered":"2024-05-10T04:38:08.339606+0000","last_unstale":"2024-05-10T04:39:41.495403+0000","last_undegraded":"2024-05-10T04:39:41.495403+0000","last_fullsized":"2024-05-10T04:39:41.495403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:22:40.212212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449986+0000","last_change":"2024-05-10T04:38:08.781195+0000","last_active":"2024-05-10T04:39:11.449986+0000","last_peered":"2024-05-10T04:39:11.449986+0000","last_clean":"2024-05-10T04:39:11.449986+0000","last_became_active":"2024-05-10T04:38:08.780779+0000","last_became_peered":"2024-05-10T04:38:08.780779+0000","last_unstale":"2024-05-10T04:39:11.449986+0000","last_undegraded":"2024-05-10T04:39:11.449986+0000","last_fullsized":"2024-05-10T04:39:11.449986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:24.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442936+0000","last_change":"2024-05-10T04:38:12.799494+0000","last_active":"2024-05-10T04:39:11.442936+0000","last_peered":"2024-05-10T04:39:11.442936+0000","last_clean":"2024-05-10T04:39:11.442936+0000","last_became_active":"2024-05-10T04:38:12.799071+0000","last_became_peered":"2024-05-10T04:38:12.799071+0000","last_unstale":"2024-05-10T04:39:11.442936+0000","last_undegraded":"2024-05-10T04:39:11.442936+0000","last_fullsized":"2024-05-10T04:39:11.442936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:00.814875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445725+0000","last_change":"2024-05-10T04:38:08.348569+0000","last_active":"2024-05-10T04:39:11.445725+0000","last_peered":"2024-05-10T04:39:11.445725+0000","last_clean":"2024-05-10T04:39:11.445725+0000","last_became_active":"2024-05-10T04:38:08.347942+0000","last_became_peered":"2024-05-10T04:38:08.347942+0000","last_unstale":"2024-05-10T04:39:11.445725+0000","last_undegraded":"2024-05-10T04:39:11.445725+0000","last_fullsized":"2024-05-10T04:39:11.445725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.865369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439473+0000","last_change":"2024-05-10T04:38:08.784447+0000","last_active":"2024-05-10T04:39:11.439473+0000","last_peered":"2024-05-10T04:39:11.439473+0000","last_clean":"2024-05-10T04:39:11.439473+0000","last_became_active":"2024-05-10T04:38:08.783989+0000","last_became_peered":"2024-05-10T04:38:08.783989+0000","last_unstale":"2024-05-10T04:39:11.439473+0000","last_undegraded":"2024-05-10T04:39:11.439473+0000","last_fullsized":"2024-05-10T04:39:11.439473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:37.511400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442966+0000","last_change":"2024-05-10T04:38:12.807478+0000","last_active":"2024-05-10T04:39:11.442966+0000","last_peered":"2024-05-10T04:39:11.442966+0000","last_clean":"2024-05-10T04:39:11.442966+0000","last_became_active":"2024-05-10T04:38:12.806801+0000","last_became_peered":"2024-05-10T04:38:12.806801+0000","last_unstale":"2024-05-10T04:39:11.442966+0000","last_undegraded":"2024-05-10T04:39:11.442966+0000","last_fullsized":"2024-05-10T04:39:11.442966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:32.168836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444446+0000","last_change":"2024-05-10T04:38:08.343610+0000","last_active":"2024-05-10T04:39:11.444446+0000","last_peered":"2024-05-10T04:39:11.444446+0000","last_clean":"2024-05-10T04:39:11.444446+0000","last_became_active":"2024-05-10T04:38:08.342880+0000","last_became_peered":"2024-05-10T04:38:08.342880+0000","last_unstale":"2024-05-10T04:39:11.444446+0000","last_undegraded":"2024-05-10T04:39:11.444446+0000","last_fullsized":"2024-05-10T04:39:11.444446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:45:11.502258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449795+0000","last_change":"2024-05-10T04:38:08.780354+0000","last_active":"2024-05-10T04:39:11.449795+0000","last_peered":"2024-05-10T04:39:11.449795+0000","last_clean":"2024-05-10T04:39:11.449795+0000","last_became_active":"2024-05-10T04:38:08.779777+0000","last_became_peered":"2024-05-10T04:38:08.779777+0000","last_unstale":"2024-05-10T04:39:11.449795+0000","last_undegraded":"2024-05-10T04:39:11.449795+0000","last_fullsized":"2024-05-10T04:39:11.449795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:28.243474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.450211+0000","last_change":"2024-05-10T04:39:12.450211+0000","last_active":"2024-05-10T04:39:12.450211+0000","last_peered":"2024-05-10T04:39:12.450211+0000","last_clean":"2024-05-10T04:39:12.450211+0000","last_became_active":"2024-05-10T04:39:12.449843+0000","last_became_peered":"2024-05-10T04:39:12.449843+0000","last_unstale":"2024-05-10T04:39:12.450211+0000","last_undegraded":"2024-05-10T04:39:12.450211+0000","last_fullsized":"2024-05-10T04:39:12.450211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:07.725273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'38","reported_seq":205,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:42.045516+0000","last_change":"2024-05-10T04:38:08.343873+0000","last_active":"2024-05-10T04:39:42.045516+0000","last_peered":"2024-05-10T04:39:42.045516+0000","last_clean":"2024-05-10T04:39:42.045516+0000","last_became_active":"2024-05-10T04:38:08.343360+0000","last_became_peered":"2024-05-10T04:38:08.343360+0000","last_unstale":"2024-05-10T04:39:42.045516+0000","last_undegraded":"2024-05-10T04:39:42.045516+0000","last_fullsized":"2024-05-10T04:39:42.045516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:06:10.816343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":141,"num_write":40,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437968+0000","last_change":"2024-05-10T04:38:08.783955+0000","last_active":"2024-05-10T04:39:11.437968+0000","last_peered":"2024-05-10T04:39:11.437968+0000","last_clean":"2024-05-10T04:39:11.437968+0000","last_became_active":"2024-05-10T04:38:08.779131+0000","last_became_peered":"2024-05-10T04:38:08.779131+0000","last_unstale":"2024-05-10T04:39:11.437968+0000","last_undegraded":"2024-05-10T04:39:11.437968+0000","last_fullsized":"2024-05-10T04:39:11.437968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:58.619571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445044+0000","last_change":"2024-05-10T04:38:12.795140+0000","last_active":"2024-05-10T04:39:11.445044+0000","last_peered":"2024-05-10T04:39:11.445044+0000","last_clean":"2024-05-10T04:39:11.445044+0000","last_became_active":"2024-05-10T04:38:12.794666+0000","last_became_peered":"2024-05-10T04:38:12.794666+0000","last_unstale":"2024-05-10T04:39:11.445044+0000","last_undegraded":"2024-05-10T04:39:11.445044+0000","last_fullsized":"2024-05-10T04:39:11.445044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:44.468321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447995+0000","last_change":"2024-05-10T04:38:08.338866+0000","last_active":"2024-05-10T04:39:11.447995+0000","last_peered":"2024-05-10T04:39:11.447995+0000","last_clean":"2024-05-10T04:39:11.447995+0000","last_became_active":"2024-05-10T04:38:08.338434+0000","last_became_peered":"2024-05-10T04:38:08.338434+0000","last_unstale":"2024-05-10T04:39:11.447995+0000","last_undegraded":"2024-05-10T04:39:11.447995+0000","last_fullsized":"2024-05-10T04:39:11.447995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:49.858034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446144+0000","last_change":"2024-05-10T04:38:08.768998+0000","last_active":"2024-05-10T04:39:11.446144+0000","last_peered":"2024-05-10T04:39:11.446144+0000","last_clean":"2024-05-10T04:39:11.446144+0000","last_became_active":"2024-05-10T04:38:08.768687+0000","last_became_peered":"2024-05-10T04:38:08.768687+0000","last_unstale":"2024-05-10T04:39:11.446144+0000","last_undegraded":"2024-05-10T04:39:11.446144+0000","last_fullsized":"2024-05-10T04:39:11.446144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:19.390746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442289+0000","last_change":"2024-05-10T04:38:12.801223+0000","last_active":"2024-05-10T04:39:11.442289+0000","last_peered":"2024-05-10T04:39:11.442289+0000","last_clean":"2024-05-10T04:39:11.442289+0000","last_became_active":"2024-05-10T04:38:12.800750+0000","last_became_peered":"2024-05-10T04:38:12.800750+0000","last_unstale":"2024-05-10T04:39:11.442289+0000","last_undegraded":"2024-05-10T04:39:11.442289+0000","last_fullsized":"2024-05-10T04:39:11.442289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:33.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445239+0000","last_change":"2024-05-10T04:38:08.344579+0000","last_active":"2024-05-10T04:39:11.445239+0000","last_peered":"2024-05-10T04:39:11.445239+0000","last_clean":"2024-05-10T04:39:11.445239+0000","last_became_active":"2024-05-10T04:38:08.344250+0000","last_became_peered":"2024-05-10T04:38:08.344250+0000","last_unstale":"2024-05-10T04:39:11.445239+0000","last_undegraded":"2024-05-10T04:39:11.445239+0000","last_fullsized":"2024-05-10T04:39:11.445239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:56.491732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445406+0000","last_change":"2024-05-10T04:38:08.784556+0000","last_active":"2024-05-10T04:39:11.445406+0000","last_peered":"2024-05-10T04:39:11.445406+0000","last_clean":"2024-05-10T04:39:11.445406+0000","last_became_active":"2024-05-10T04:38:08.784229+0000","last_became_peered":"2024-05-10T04:38:08.784229+0000","last_unstale":"2024-05-10T04:39:11.445406+0000","last_undegraded":"2024-05-10T04:39:11.445406+0000","last_fullsized":"2024-05-10T04:39:11.445406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:48:46.519837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443931+0000","last_change":"2024-05-10T04:38:12.805795+0000","last_active":"2024-05-10T04:39:11.443931+0000","last_peered":"2024-05-10T04:39:11.443931+0000","last_clean":"2024-05-10T04:39:11.443931+0000","last_became_active":"2024-05-10T04:38:12.805366+0000","last_became_peered":"2024-05-10T04:38:12.805366+0000","last_unstale":"2024-05-10T04:39:11.443931+0000","last_undegraded":"2024-05-10T04:39:11.443931+0000","last_fullsized":"2024-05-10T04:39:11.443931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:38:11.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:38.064666+0000","last_change":"2024-05-10T04:38:08.345396+0000","last_active":"2024-05-10T04:39:38.064666+0000","last_peered":"2024-05-10T04:39:38.064666+0000","last_clean":"2024-05-10T04:39:38.064666+0000","last_became_active":"2024-05-10T04:38:08.345076+0000","last_became_peered":"2024-05-10T04:38:08.345076+0000","last_unstale":"2024-05-10T04:39:38.064666+0000","last_undegraded":"2024-05-10T04:39:38.064666+0000","last_fullsized":"2024-05-10T04:39:38.064666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:38.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":854,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446201+0000","last_change":"2024-05-10T04:38:08.791089+0000","last_active":"2024-05-10T04:39:11.446201+0000","last_peered":"2024-05-10T04:39:11.446201+0000","last_clean":"2024-05-10T04:39:11.446201+0000","last_became_active":"2024-05-10T04:38:08.790813+0000","last_became_peered":"2024-05-10T04:38:08.790813+0000","last_unstale":"2024-05-10T04:39:11.446201+0000","last_undegraded":"2024-05-10T04:39:11.446201+0000","last_fullsized":"2024-05-10T04:39:11.446201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:52:45.753196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446807+0000","last_change":"2024-05-10T04:38:12.803896+0000","last_active":"2024-05-10T04:39:11.446807+0000","last_peered":"2024-05-10T04:39:11.446807+0000","last_clean":"2024-05-10T04:39:11.446807+0000","last_became_active":"2024-05-10T04:38:12.803512+0000","last_became_peered":"2024-05-10T04:38:12.803512+0000","last_unstale":"2024-05-10T04:39:11.446807+0000","last_undegraded":"2024-05-10T04:39:11.446807+0000","last_fullsized":"2024-05-10T04:39:11.446807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:01:23.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.467762+0000","last_change":"2024-05-10T04:39:12.467304+0000","last_active":"2024-05-10T04:39:12.467762+0000","last_peered":"2024-05-10T04:39:12.467762+0000","last_clean":"2024-05-10T04:39:12.467762+0000","last_became_active":"2024-05-10T04:39:12.448415+0000","last_became_peered":"2024-05-10T04:39:12.448415+0000","last_unstale":"2024-05-10T04:39:12.467762+0000","last_undegraded":"2024-05-10T04:39:12.467762+0000","last_fullsized":"2024-05-10T04:39:12.467762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:20.063125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.762052+0000","last_change":"2024-05-10T04:39:12.762052+0000","last_active":"2024-05-10T04:39:12.762052+0000","last_peered":"2024-05-10T04:39:12.762052+0000","last_clean":"2024-05-10T04:39:12.762052+0000","last_became_active":"2024-05-10T04:39:12.761658+0000","last_became_peered":"2024-05-10T04:39:12.761658+0000","last_unstale":"2024-05-10T04:39:12.762052+0000","last_undegraded":"2024-05-10T04:39:12.762052+0000","last_fullsized":"2024-05-10T04:39:12.762052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:05:07.565757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443495+0000","last_change":"2024-05-10T04:38:12.796046+0000","last_active":"2024-05-10T04:39:11.443495+0000","last_peered":"2024-05-10T04:39:11.443495+0000","last_clean":"2024-05-10T04:39:11.443495+0000","last_became_active":"2024-05-10T04:38:12.795635+0000","last_became_peered":"2024-05-10T04:38:12.795635+0000","last_unstale":"2024-05-10T04:39:11.443495+0000","last_undegraded":"2024-05-10T04:39:11.443495+0000","last_fullsized":"2024-05-10T04:39:11.443495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:50.822404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438595+0000","last_change":"2024-05-10T04:38:08.348701+0000","last_active":"2024-05-10T04:39:11.438595+0000","last_peered":"2024-05-10T04:39:11.438595+0000","last_clean":"2024-05-10T04:39:11.438595+0000","last_became_active":"2024-05-10T04:38:08.348345+0000","last_became_peered":"2024-05-10T04:38:08.348345+0000","last_unstale":"2024-05-10T04:39:11.438595+0000","last_undegraded":"2024-05-10T04:39:11.438595+0000","last_fullsized":"2024-05-10T04:39:11.438595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:31.505996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.450401+0000","last_change":"2024-05-10T04:39:12.450401+0000","last_active":"2024-05-10T04:39:12.450401+0000","last_peered":"2024-05-10T04:39:12.450401+0000","last_clean":"2024-05-10T04:39:12.450401+0000","last_became_active":"2024-05-10T04:39:12.449999+0000","last_became_peered":"2024-05-10T04:39:12.449999+0000","last_unstale":"2024-05-10T04:39:12.450401+0000","last_undegraded":"2024-05-10T04:39:12.450401+0000","last_fullsized":"2024-05-10T04:39:12.450401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:46.589660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437683+0000","last_change":"2024-05-10T04:38:12.809215+0000","last_active":"2024-05-10T04:39:11.437683+0000","last_peered":"2024-05-10T04:39:11.437683+0000","last_clean":"2024-05-10T04:39:11.437683+0000","last_became_active":"2024-05-10T04:38:12.808717+0000","last_became_peered":"2024-05-10T04:38:12.808717+0000","last_unstale":"2024-05-10T04:39:11.437683+0000","last_undegraded":"2024-05-10T04:39:11.437683+0000","last_fullsized":"2024-05-10T04:39:11.437683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:40.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443605+0000","last_change":"2024-05-10T04:38:08.347770+0000","last_active":"2024-05-10T04:39:11.443605+0000","last_peered":"2024-05-10T04:39:11.443605+0000","last_clean":"2024-05-10T04:39:11.443605+0000","last_became_active":"2024-05-10T04:38:08.347444+0000","last_became_peered":"2024-05-10T04:38:08.347444+0000","last_unstale":"2024-05-10T04:39:11.443605+0000","last_undegraded":"2024-05-10T04:39:11.443605+0000","last_fullsized":"2024-05-10T04:39:11.443605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:49.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442442+0000","last_change":"2024-05-10T04:38:08.765358+0000","last_active":"2024-05-10T04:39:11.442442+0000","last_peered":"2024-05-10T04:39:11.442442+0000","last_clean":"2024-05-10T04:39:11.442442+0000","last_became_active":"2024-05-10T04:38:08.765090+0000","last_became_peered":"2024-05-10T04:38:08.765090+0000","last_unstale":"2024-05-10T04:39:11.442442+0000","last_undegraded":"2024-05-10T04:39:11.442442+0000","last_fullsized":"2024-05-10T04:39:11.442442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:55:25.722331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446434+0000","last_change":"2024-05-10T04:38:12.794765+0000","last_active":"2024-05-10T04:39:11.446434+0000","last_peered":"2024-05-10T04:39:11.446434+0000","last_clean":"2024-05-10T04:39:11.446434+0000","last_became_active":"2024-05-10T04:38:12.794400+0000","last_became_peered":"2024-05-10T04:38:12.794400+0000","last_unstale":"2024-05-10T04:39:11.446434+0000","last_undegraded":"2024-05-10T04:39:11.446434+0000","last_fullsized":"2024-05-10T04:39:11.446434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:15:16.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:38.065103+0000","last_change":"2024-05-10T04:38:08.348812+0000","last_active":"2024-05-10T04:39:38.065103+0000","last_peered":"2024-05-10T04:39:38.065103+0000","last_clean":"2024-05-10T04:39:38.065103+0000","last_became_active":"2024-05-10T04:38:08.348207+0000","last_became_peered":"2024-05-10T04:38:08.348207+0000","last_unstale":"2024-05-10T04:39:38.065103+0000","last_undegraded":"2024-05-10T04:39:38.065103+0000","last_fullsized":"2024-05-10T04:39:38.065103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:17.350094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442205+0000","last_change":"2024-05-10T04:38:08.785502+0000","last_active":"2024-05-10T04:39:11.442205+0000","last_peered":"2024-05-10T04:39:11.442205+0000","last_clean":"2024-05-10T04:39:11.442205+0000","last_became_active":"2024-05-10T04:38:08.785073+0000","last_became_peered":"2024-05-10T04:38:08.785073+0000","last_unstale":"2024-05-10T04:39:11.442205+0000","last_undegraded":"2024-05-10T04:39:11.442205+0000","last_fullsized":"2024-05-10T04:39:11.442205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:08:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446700+0000","last_change":"2024-05-10T04:38:12.809599+0000","last_active":"2024-05-10T04:39:11.446700+0000","last_peered":"2024-05-10T04:39:11.446700+0000","last_clean":"2024-05-10T04:39:11.446700+0000","last_became_active":"2024-05-10T04:38:12.809200+0000","last_became_peered":"2024-05-10T04:38:12.809200+0000","last_unstale":"2024-05-10T04:39:11.446700+0000","last_undegraded":"2024-05-10T04:39:11.446700+0000","last_fullsized":"2024-05-10T04:39:11.446700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:54.940517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444413+0000","last_change":"2024-05-10T04:38:08.779837+0000","last_active":"2024-05-10T04:39:11.444413+0000","last_peered":"2024-05-10T04:39:11.444413+0000","last_clean":"2024-05-10T04:39:11.444413+0000","last_became_active":"2024-05-10T04:38:08.779501+0000","last_became_peered":"2024-05-10T04:38:08.779501+0000","last_unstale":"2024-05-10T04:39:11.444413+0000","last_undegraded":"2024-05-10T04:39:11.444413+0000","last_fullsized":"2024-05-10T04:39:11.444413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:00.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:40.067333+0000","last_change":"2024-05-10T04:38:08.343649+0000","last_active":"2024-05-10T04:39:40.067333+0000","last_peered":"2024-05-10T04:39:40.067333+0000","last_clean":"2024-05-10T04:39:40.067333+0000","last_became_active":"2024-05-10T04:38:08.343292+0000","last_became_peered":"2024-05-10T04:38:08.343292+0000","last_unstale":"2024-05-10T04:39:40.067333+0000","last_undegraded":"2024-05-10T04:39:40.067333+0000","last_fullsized":"2024-05-10T04:39:40.067333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:53.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446345+0000","last_change":"2024-05-10T04:38:12.796552+0000","last_active":"2024-05-10T04:39:11.446345+0000","last_peered":"2024-05-10T04:39:11.446345+0000","last_clean":"2024-05-10T04:39:11.446345+0000","last_became_active":"2024-05-10T04:38:12.796151+0000","last_became_peered":"2024-05-10T04:38:12.796151+0000","last_unstale":"2024-05-10T04:39:11.446345+0000","last_undegraded":"2024-05-10T04:39:11.446345+0000","last_fullsized":"2024-05-10T04:39:11.446345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:05.520859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448787+0000","last_change":"2024-05-10T04:38:08.770957+0000","last_active":"2024-05-10T04:39:11.448787+0000","last_peered":"2024-05-10T04:39:11.448787+0000","last_clean":"2024-05-10T04:39:11.448787+0000","last_became_active":"2024-05-10T04:38:08.770572+0000","last_became_peered":"2024-05-10T04:38:08.770572+0000","last_unstale":"2024-05-10T04:39:11.448787+0000","last_undegraded":"2024-05-10T04:39:11.448787+0000","last_fullsized":"2024-05-10T04:39:11.448787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:21.421033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442857+0000","last_change":"2024-05-10T04:38:08.340788+0000","last_active":"2024-05-10T04:39:11.442857+0000","last_peered":"2024-05-10T04:39:11.442857+0000","last_clean":"2024-05-10T04:39:11.442857+0000","last_became_active":"2024-05-10T04:38:08.340407+0000","last_became_peered":"2024-05-10T04:38:08.340407+0000","last_unstale":"2024-05-10T04:39:11.442857+0000","last_undegraded":"2024-05-10T04:39:11.442857+0000","last_fullsized":"2024-05-10T04:39:11.442857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:30:05.276951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":329,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446409+0000","last_change":"2024-05-10T04:37:20.961282+0000","last_active":"2024-05-10T04:39:11.446409+0000","last_peered":"2024-05-10T04:39:11.446409+0000","last_clean":"2024-05-10T04:39:11.446409+0000","last_became_active":"2024-05-10T04:37:20.960828+0000","last_became_peered":"2024-05-10T04:37:20.960828+0000","last_unstale":"2024-05-10T04:39:11.446409+0000","last_undegraded":"2024-05-10T04:39:11.446409+0000","last_fullsized":"2024-05-10T04:39:11.446409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_clean_scrub_stamp":"2024-05-10T04:35:00.248380+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:08:04.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446236+0000","last_change":"2024-05-10T04:38:12.809899+0000","last_active":"2024-05-10T04:39:11.446236+0000","last_peered":"2024-05-10T04:39:11.446236+0000","last_clean":"2024-05-10T04:39:11.446236+0000","last_became_active":"2024-05-10T04:38:12.809441+0000","last_became_peered":"2024-05-10T04:38:12.809441+0000","last_unstale":"2024-05-10T04:39:11.446236+0000","last_undegraded":"2024-05-10T04:39:11.446236+0000","last_fullsized":"2024-05-10T04:39:11.446236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:48.378890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445906+0000","last_change":"2024-05-10T04:38:08.784838+0000","last_active":"2024-05-10T04:39:11.445906+0000","last_peered":"2024-05-10T04:39:11.445906+0000","last_clean":"2024-05-10T04:39:11.445906+0000","last_became_active":"2024-05-10T04:38:08.784464+0000","last_became_peered":"2024-05-10T04:38:08.784464+0000","last_unstale":"2024-05-10T04:39:11.445906+0000","last_undegraded":"2024-05-10T04:39:11.445906+0000","last_fullsized":"2024-05-10T04:39:11.445906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:00.335350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.470361+0000","last_change":"2024-05-10T04:39:12.469901+0000","last_active":"2024-05-10T04:39:12.470361+0000","last_peered":"2024-05-10T04:39:12.470361+0000","last_clean":"2024-05-10T04:39:12.470361+0000","last_became_active":"2024-05-10T04:39:12.454202+0000","last_became_peered":"2024-05-10T04:39:12.454202+0000","last_unstale":"2024-05-10T04:39:12.470361+0000","last_undegraded":"2024-05-10T04:39:12.470361+0000","last_fullsized":"2024-05-10T04:39:12.470361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:18.646199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441308+0000","last_change":"2024-05-10T04:38:12.801079+0000","last_active":"2024-05-10T04:39:11.441308+0000","last_peered":"2024-05-10T04:39:11.441308+0000","last_clean":"2024-05-10T04:39:11.441308+0000","last_became_active":"2024-05-10T04:38:12.800630+0000","last_became_peered":"2024-05-10T04:38:12.800630+0000","last_unstale":"2024-05-10T04:39:11.441308+0000","last_undegraded":"2024-05-10T04:39:11.441308+0000","last_fullsized":"2024-05-10T04:39:11.441308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:05.213620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438080+0000","last_change":"2024-05-10T04:38:08.348764+0000","last_active":"2024-05-10T04:39:11.438080+0000","last_peered":"2024-05-10T04:39:11.438080+0000","last_clean":"2024-05-10T04:39:11.438080+0000","last_became_active":"2024-05-10T04:38:08.347939+0000","last_became_peered":"2024-05-10T04:38:08.347939+0000","last_unstale":"2024-05-10T04:39:11.438080+0000","last_undegraded":"2024-05-10T04:39:11.438080+0000","last_fullsized":"2024-05-10T04:39:11.438080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:01.308255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446594+0000","last_change":"2024-05-10T04:38:08.771382+0000","last_active":"2024-05-10T04:39:11.446594+0000","last_peered":"2024-05-10T04:39:11.446594+0000","last_clean":"2024-05-10T04:39:11.446594+0000","last_became_active":"2024-05-10T04:38:08.771108+0000","last_became_peered":"2024-05-10T04:38:08.771108+0000","last_unstale":"2024-05-10T04:39:11.446594+0000","last_undegraded":"2024-05-10T04:39:11.446594+0000","last_fullsized":"2024-05-10T04:39:11.446594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:32.664640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450061+0000","last_change":"2024-05-10T04:38:12.809582+0000","last_active":"2024-05-10T04:39:11.450061+0000","last_peered":"2024-05-10T04:39:11.450061+0000","last_clean":"2024-05-10T04:39:11.450061+0000","last_became_active":"2024-05-10T04:38:12.809134+0000","last_became_peered":"2024-05-10T04:38:12.809134+0000","last_unstale":"2024-05-10T04:39:11.450061+0000","last_undegraded":"2024-05-10T04:39:11.450061+0000","last_fullsized":"2024-05-10T04:39:11.450061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:31:07.694368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444107+0000","last_change":"2024-05-10T04:38:08.345903+0000","last_active":"2024-05-10T04:39:11.444107+0000","last_peered":"2024-05-10T04:39:11.444107+0000","last_clean":"2024-05-10T04:39:11.444107+0000","last_became_active":"2024-05-10T04:38:08.345371+0000","last_became_peered":"2024-05-10T04:38:08.345371+0000","last_unstale":"2024-05-10T04:39:11.444107+0000","last_undegraded":"2024-05-10T04:39:11.444107+0000","last_fullsized":"2024-05-10T04:39:11.444107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:03.581112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449208+0000","last_change":"2024-05-10T04:38:08.779853+0000","last_active":"2024-05-10T04:39:11.449208+0000","last_peered":"2024-05-10T04:39:11.449208+0000","last_clean":"2024-05-10T04:39:11.449208+0000","last_became_active":"2024-05-10T04:38:08.779531+0000","last_became_peered":"2024-05-10T04:38:08.779531+0000","last_unstale":"2024-05-10T04:39:11.449208+0000","last_undegraded":"2024-05-10T04:39:11.449208+0000","last_fullsized":"2024-05-10T04:39:11.449208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:05.079540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447323+0000","last_change":"2024-05-10T04:38:12.795970+0000","last_active":"2024-05-10T04:39:11.447323+0000","last_peered":"2024-05-10T04:39:11.447323+0000","last_clean":"2024-05-10T04:39:11.447323+0000","last_became_active":"2024-05-10T04:38:12.795497+0000","last_became_peered":"2024-05-10T04:38:12.795497+0000","last_unstale":"2024-05-10T04:39:11.447323+0000","last_undegraded":"2024-05-10T04:39:11.447323+0000","last_fullsized":"2024-05-10T04:39:11.447323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:11.174672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443466+0000","last_change":"2024-05-10T04:38:08.343268+0000","last_active":"2024-05-10T04:39:11.443466+0000","last_peered":"2024-05-10T04:39:11.443466+0000","last_clean":"2024-05-10T04:39:11.443466+0000","last_became_active":"2024-05-10T04:38:08.342952+0000","last_became_peered":"2024-05-10T04:38:08.342952+0000","last_unstale":"2024-05-10T04:39:11.443466+0000","last_undegraded":"2024-05-10T04:39:11.443466+0000","last_fullsized":"2024-05-10T04:39:11.443466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:49.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439365+0000","last_change":"2024-05-10T04:38:08.784276+0000","last_active":"2024-05-10T04:39:11.439365+0000","last_peered":"2024-05-10T04:39:11.439365+0000","last_clean":"2024-05-10T04:39:11.439365+0000","last_became_active":"2024-05-10T04:38:08.783951+0000","last_became_peered":"2024-05-10T04:38:08.783951+0000","last_unstale":"2024-05-10T04:39:11.439365+0000","last_undegraded":"2024-05-10T04:39:11.439365+0000","last_fullsized":"2024-05-10T04:39:11.439365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:45:43.992567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443129+0000","last_change":"2024-05-10T04:38:12.811689+0000","last_active":"2024-05-10T04:39:11.443129+0000","last_peered":"2024-05-10T04:39:11.443129+0000","last_clean":"2024-05-10T04:39:11.443129+0000","last_became_active":"2024-05-10T04:38:12.811138+0000","last_became_peered":"2024-05-10T04:38:12.811138+0000","last_unstale":"2024-05-10T04:39:11.443129+0000","last_undegraded":"2024-05-10T04:39:11.443129+0000","last_fullsized":"2024-05-10T04:39:11.443129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:09.291137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448922+0000","last_change":"2024-05-10T04:38:08.348738+0000","last_active":"2024-05-10T04:39:11.448922+0000","last_peered":"2024-05-10T04:39:11.448922+0000","last_clean":"2024-05-10T04:39:11.448922+0000","last_became_active":"2024-05-10T04:38:08.348382+0000","last_became_peered":"2024-05-10T04:38:08.348382+0000","last_unstale":"2024-05-10T04:39:11.448922+0000","last_undegraded":"2024-05-10T04:39:11.448922+0000","last_fullsized":"2024-05-10T04:39:11.448922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:02.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437662+0000","last_change":"2024-05-10T04:38:08.781483+0000","last_active":"2024-05-10T04:39:11.437662+0000","last_peered":"2024-05-10T04:39:11.437662+0000","last_clean":"2024-05-10T04:39:11.437662+0000","last_became_active":"2024-05-10T04:38:08.777713+0000","last_became_peered":"2024-05-10T04:38:08.777713+0000","last_unstale":"2024-05-10T04:39:11.437662+0000","last_undegraded":"2024-05-10T04:39:11.437662+0000","last_fullsized":"2024-05-10T04:39:11.437662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:41.050901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444353+0000","last_change":"2024-05-10T04:38:12.800656+0000","last_active":"2024-05-10T04:39:11.444353+0000","last_peered":"2024-05-10T04:39:11.444353+0000","last_clean":"2024-05-10T04:39:11.444353+0000","last_became_active":"2024-05-10T04:38:12.800191+0000","last_became_peered":"2024-05-10T04:38:12.800191+0000","last_unstale":"2024-05-10T04:39:11.444353+0000","last_undegraded":"2024-05-10T04:39:11.444353+0000","last_fullsized":"2024-05-10T04:39:11.444353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:47.017642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448670+0000","last_change":"2024-05-10T04:38:08.344017+0000","last_active":"2024-05-10T04:39:11.448670+0000","last_peered":"2024-05-10T04:39:11.448670+0000","last_clean":"2024-05-10T04:39:11.448670+0000","last_became_active":"2024-05-10T04:38:08.343646+0000","last_became_peered":"2024-05-10T04:38:08.343646+0000","last_unstale":"2024-05-10T04:39:11.448670+0000","last_undegraded":"2024-05-10T04:39:11.448670+0000","last_fullsized":"2024-05-10T04:39:11.448670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:02:30.455019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441892+0000","last_change":"2024-05-10T04:38:08.779065+0000","last_active":"2024-05-10T04:39:11.441892+0000","last_peered":"2024-05-10T04:39:11.441892+0000","last_clean":"2024-05-10T04:39:11.441892+0000","last_became_active":"2024-05-10T04:38:08.778685+0000","last_became_peered":"2024-05-10T04:38:08.778685+0000","last_unstale":"2024-05-10T04:39:11.441892+0000","last_undegraded":"2024-05-10T04:39:11.441892+0000","last_fullsized":"2024-05-10T04:39:11.441892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:25:58.780080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444508+0000","last_change":"2024-05-10T04:38:12.801524+0000","last_active":"2024-05-10T04:39:11.444508+0000","last_peered":"2024-05-10T04:39:11.444508+0000","last_clean":"2024-05-10T04:39:11.444508+0000","last_became_active":"2024-05-10T04:38:12.800777+0000","last_became_peered":"2024-05-10T04:38:12.800777+0000","last_unstale":"2024-05-10T04:39:11.444508+0000","last_undegraded":"2024-05-10T04:39:11.444508+0000","last_fullsized":"2024-05-10T04:39:11.444508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:14.139236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447769+0000","last_change":"2024-05-10T04:38:08.344785+0000","last_active":"2024-05-10T04:39:11.447769+0000","last_peered":"2024-05-10T04:39:11.447769+0000","last_clean":"2024-05-10T04:39:11.447769+0000","last_became_active":"2024-05-10T04:38:08.344337+0000","last_became_peered":"2024-05-10T04:38:08.344337+0000","last_unstale":"2024-05-10T04:39:11.447769+0000","last_undegraded":"2024-05-10T04:39:11.447769+0000","last_fullsized":"2024-05-10T04:39:11.447769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:47.682532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448971+0000","last_change":"2024-05-10T04:38:08.768686+0000","last_active":"2024-05-10T04:39:11.448971+0000","last_peered":"2024-05-10T04:39:11.448971+0000","last_clean":"2024-05-10T04:39:11.448971+0000","last_became_active":"2024-05-10T04:38:08.768317+0000","last_became_peered":"2024-05-10T04:38:08.768317+0000","last_unstale":"2024-05-10T04:39:11.448971+0000","last_undegraded":"2024-05-10T04:39:11.448971+0000","last_fullsized":"2024-05-10T04:39:11.448971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:42:33.670912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443063+0000","last_change":"2024-05-10T04:38:12.809825+0000","last_active":"2024-05-10T04:39:11.443063+0000","last_peered":"2024-05-10T04:39:11.443063+0000","last_clean":"2024-05-10T04:39:11.443063+0000","last_became_active":"2024-05-10T04:38:12.809378+0000","last_became_peered":"2024-05-10T04:38:12.809378+0000","last_unstale":"2024-05-10T04:39:11.443063+0000","last_undegraded":"2024-05-10T04:39:11.443063+0000","last_fullsized":"2024-05-10T04:39:11.443063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:28.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438131+0000","last_change":"2024-05-10T04:38:08.347871+0000","last_active":"2024-05-10T04:39:11.438131+0000","last_peered":"2024-05-10T04:39:11.438131+0000","last_clean":"2024-05-10T04:39:11.438131+0000","last_became_active":"2024-05-10T04:38:08.347618+0000","last_became_peered":"2024-05-10T04:38:08.347618+0000","last_unstale":"2024-05-10T04:39:11.438131+0000","last_undegraded":"2024-05-10T04:39:11.438131+0000","last_fullsized":"2024-05-10T04:39:11.438131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:09:42.060082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445616+0000","last_change":"2024-05-10T04:38:08.779191+0000","last_active":"2024-05-10T04:39:11.445616+0000","last_peered":"2024-05-10T04:39:11.445616+0000","last_clean":"2024-05-10T04:39:11.445616+0000","last_became_active":"2024-05-10T04:38:08.778829+0000","last_became_peered":"2024-05-10T04:38:08.778829+0000","last_unstale":"2024-05-10T04:39:11.445616+0000","last_undegraded":"2024-05-10T04:39:11.445616+0000","last_fullsized":"2024-05-10T04:39:11.445616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:07.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448922+0000","last_change":"2024-05-10T04:38:12.788693+0000","last_active":"2024-05-10T04:39:11.448922+0000","last_peered":"2024-05-10T04:39:11.448922+0000","last_clean":"2024-05-10T04:39:11.448922+0000","last_became_active":"2024-05-10T04:38:12.788156+0000","last_became_peered":"2024-05-10T04:38:12.788156+0000","last_unstale":"2024-05-10T04:39:11.448922+0000","last_undegraded":"2024-05-10T04:39:11.448922+0000","last_fullsized":"2024-05-10T04:39:11.448922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:30.119523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:36.487598+0000","last_change":"2024-05-10T04:39:12.455797+0000","last_active":"2024-05-10T04:39:36.487598+0000","last_peered":"2024-05-10T04:39:36.487598+0000","last_clean":"2024-05-10T04:39:36.487598+0000","last_became_active":"2024-05-10T04:39:12.449843+0000","last_became_peered":"2024-05-10T04:39:12.449843+0000","last_unstale":"2024-05-10T04:39:36.487598+0000","last_undegraded":"2024-05-10T04:39:36.487598+0000","last_fullsized":"2024-05-10T04:39:36.487598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:32:17.432455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1515,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446668+0000","last_change":"2024-05-10T04:38:08.780352+0000","last_active":"2024-05-10T04:39:11.446668+0000","last_peered":"2024-05-10T04:39:11.446668+0000","last_clean":"2024-05-10T04:39:11.446668+0000","last_became_active":"2024-05-10T04:38:08.779779+0000","last_became_peered":"2024-05-10T04:38:08.779779+0000","last_unstale":"2024-05-10T04:39:11.446668+0000","last_undegraded":"2024-05-10T04:39:11.446668+0000","last_fullsized":"2024-05-10T04:39:11.446668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:54:31.973896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439035+0000","last_change":"2024-05-10T04:38:12.805310+0000","last_active":"2024-05-10T04:39:11.439035+0000","last_peered":"2024-05-10T04:39:11.439035+0000","last_clean":"2024-05-10T04:39:11.439035+0000","last_became_active":"2024-05-10T04:38:12.804827+0000","last_became_peered":"2024-05-10T04:38:12.804827+0000","last_unstale":"2024-05-10T04:39:11.439035+0000","last_undegraded":"2024-05-10T04:39:11.439035+0000","last_fullsized":"2024-05-10T04:39:11.439035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:01.655788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445754+0000","last_change":"2024-05-10T04:38:08.343813+0000","last_active":"2024-05-10T04:39:11.445754+0000","last_peered":"2024-05-10T04:39:11.445754+0000","last_clean":"2024-05-10T04:39:11.445754+0000","last_became_active":"2024-05-10T04:38:08.343309+0000","last_became_peered":"2024-05-10T04:38:08.343309+0000","last_unstale":"2024-05-10T04:39:11.445754+0000","last_undegraded":"2024-05-10T04:39:11.445754+0000","last_fullsized":"2024-05-10T04:39:11.445754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:07.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445944+0000","last_change":"2024-05-10T04:38:08.780598+0000","last_active":"2024-05-10T04:39:11.445944+0000","last_peered":"2024-05-10T04:39:11.445944+0000","last_clean":"2024-05-10T04:39:11.445944+0000","last_became_active":"2024-05-10T04:38:08.775367+0000","last_became_peered":"2024-05-10T04:38:08.775367+0000","last_unstale":"2024-05-10T04:39:11.445944+0000","last_undegraded":"2024-05-10T04:39:11.445944+0000","last_fullsized":"2024-05-10T04:39:11.445944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:33.826116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444007+0000","last_change":"2024-05-10T04:38:12.795975+0000","last_active":"2024-05-10T04:39:11.444007+0000","last_peered":"2024-05-10T04:39:11.444007+0000","last_clean":"2024-05-10T04:39:11.444007+0000","last_became_active":"2024-05-10T04:38:12.795582+0000","last_became_peered":"2024-05-10T04:38:12.795582+0000","last_unstale":"2024-05-10T04:39:11.444007+0000","last_undegraded":"2024-05-10T04:39:11.444007+0000","last_fullsized":"2024-05-10T04:39:11.444007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:42:30.734833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.766085+0000","last_change":"2024-05-10T04:39:12.766085+0000","last_active":"2024-05-10T04:39:12.766085+0000","last_peered":"2024-05-10T04:39:12.766085+0000","last_clean":"2024-05-10T04:39:12.766085+0000","last_became_active":"2024-05-10T04:39:12.761991+0000","last_became_peered":"2024-05-10T04:39:12.761991+0000","last_unstale":"2024-05-10T04:39:12.766085+0000","last_undegraded":"2024-05-10T04:39:12.766085+0000","last_fullsized":"2024-05-10T04:39:12.766085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:00:52.188593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446261+0000","last_change":"2024-05-10T04:38:08.780925+0000","last_active":"2024-05-10T04:39:11.446261+0000","last_peered":"2024-05-10T04:39:11.446261+0000","last_clean":"2024-05-10T04:39:11.446261+0000","last_became_active":"2024-05-10T04:38:08.778777+0000","last_became_peered":"2024-05-10T04:38:08.778777+0000","last_unstale":"2024-05-10T04:39:11.446261+0000","last_undegraded":"2024-05-10T04:39:11.446261+0000","last_fullsized":"2024-05-10T04:39:11.446261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:57.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442028+0000","last_change":"2024-05-10T04:38:08.780281+0000","last_active":"2024-05-10T04:39:11.442028+0000","last_peered":"2024-05-10T04:39:11.442028+0000","last_clean":"2024-05-10T04:39:11.442028+0000","last_became_active":"2024-05-10T04:38:08.779927+0000","last_became_peered":"2024-05-10T04:38:08.779927+0000","last_unstale":"2024-05-10T04:39:11.442028+0000","last_undegraded":"2024-05-10T04:39:11.442028+0000","last_fullsized":"2024-05-10T04:39:11.442028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:33.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448040+0000","last_change":"2024-05-10T04:38:08.784916+0000","last_active":"2024-05-10T04:39:11.448040+0000","last_peered":"2024-05-10T04:39:11.448040+0000","last_clean":"2024-05-10T04:39:11.448040+0000","last_became_active":"2024-05-10T04:38:08.784544+0000","last_became_peered":"2024-05-10T04:38:08.784544+0000","last_unstale":"2024-05-10T04:39:11.448040+0000","last_undegraded":"2024-05-10T04:39:11.448040+0000","last_fullsized":"2024-05-10T04:39:11.448040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:03.669897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447644+0000","last_change":"2024-05-10T04:38:08.769149+0000","last_active":"2024-05-10T04:39:11.447644+0000","last_peered":"2024-05-10T04:39:11.447644+0000","last_clean":"2024-05-10T04:39:11.447644+0000","last_became_active":"2024-05-10T04:38:08.768808+0000","last_became_peered":"2024-05-10T04:38:08.768808+0000","last_unstale":"2024-05-10T04:39:11.447644+0000","last_undegraded":"2024-05-10T04:39:11.447644+0000","last_fullsized":"2024-05-10T04:39:11.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:42:46.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438965+0000","last_change":"2024-05-10T04:38:08.784411+0000","last_active":"2024-05-10T04:39:11.438965+0000","last_peered":"2024-05-10T04:39:11.438965+0000","last_clean":"2024-05-10T04:39:11.438965+0000","last_became_active":"2024-05-10T04:38:08.784064+0000","last_became_peered":"2024-05-10T04:38:08.784064+0000","last_unstale":"2024-05-10T04:39:11.438965+0000","last_undegraded":"2024-05-10T04:39:11.438965+0000","last_fullsized":"2024-05-10T04:39:11.438965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:32:46.493888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446647+0000","last_change":"2024-05-10T04:38:08.779643+0000","last_active":"2024-05-10T04:39:11.446647+0000","last_peered":"2024-05-10T04:39:11.446647+0000","last_clean":"2024-05-10T04:39:11.446647+0000","last_became_active":"2024-05-10T04:38:08.779291+0000","last_became_peered":"2024-05-10T04:38:08.779291+0000","last_unstale":"2024-05-10T04:39:11.446647+0000","last_undegraded":"2024-05-10T04:39:11.446647+0000","last_fullsized":"2024-05-10T04:39:11.446647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:24.535515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443918+0000","last_change":"2024-05-10T04:38:08.789382+0000","last_active":"2024-05-10T04:39:11.443918+0000","last_peered":"2024-05-10T04:39:11.443918+0000","last_clean":"2024-05-10T04:39:11.443918+0000","last_became_active":"2024-05-10T04:38:08.789003+0000","last_became_peered":"2024-05-10T04:38:08.789003+0000","last_unstale":"2024-05-10T04:39:11.443918+0000","last_undegraded":"2024-05-10T04:39:11.443918+0000","last_fullsized":"2024-05-10T04:39:11.443918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:03.031319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448507+0000","last_change":"2024-05-10T04:38:08.777290+0000","last_active":"2024-05-10T04:39:11.448507+0000","last_peered":"2024-05-10T04:39:11.448507+0000","last_clean":"2024-05-10T04:39:11.448507+0000","last_became_active":"2024-05-10T04:38:08.776331+0000","last_became_peered":"2024-05-10T04:38:08.776331+0000","last_unstale":"2024-05-10T04:39:11.448507+0000","last_undegraded":"2024-05-10T04:39:11.448507+0000","last_fullsized":"2024-05-10T04:39:11.448507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:51.074991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449405+0000","last_change":"2024-05-10T04:38:08.780937+0000","last_active":"2024-05-10T04:39:11.449405+0000","last_peered":"2024-05-10T04:39:11.449405+0000","last_clean":"2024-05-10T04:39:11.449405+0000","last_became_active":"2024-05-10T04:38:08.780620+0000","last_became_peered":"2024-05-10T04:38:08.780620+0000","last_unstale":"2024-05-10T04:39:11.449405+0000","last_undegraded":"2024-05-10T04:39:11.449405+0000","last_fullsized":"2024-05-10T04:39:11.449405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:27.697571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444313+0000","last_change":"2024-05-10T04:38:08.773314+0000","last_active":"2024-05-10T04:39:11.444313+0000","last_peered":"2024-05-10T04:39:11.444313+0000","last_clean":"2024-05-10T04:39:11.444313+0000","last_became_active":"2024-05-10T04:38:08.772973+0000","last_became_peered":"2024-05-10T04:38:08.772973+0000","last_unstale":"2024-05-10T04:39:11.444313+0000","last_undegraded":"2024-05-10T04:39:11.444313+0000","last_fullsized":"2024-05-10T04:39:11.444313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:53.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448852+0000","last_change":"2024-05-10T04:38:08.775632+0000","last_active":"2024-05-10T04:39:11.448852+0000","last_peered":"2024-05-10T04:39:11.448852+0000","last_clean":"2024-05-10T04:39:11.448852+0000","last_became_active":"2024-05-10T04:38:08.775308+0000","last_became_peered":"2024-05-10T04:38:08.775308+0000","last_unstale":"2024-05-10T04:39:11.448852+0000","last_undegraded":"2024-05-10T04:39:11.448852+0000","last_fullsized":"2024-05-10T04:39:11.448852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:58.758863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449579+0000","last_change":"2024-05-10T04:38:08.787354+0000","last_active":"2024-05-10T04:39:11.449579+0000","last_peered":"2024-05-10T04:39:11.449579+0000","last_clean":"2024-05-10T04:39:11.449579+0000","last_became_active":"2024-05-10T04:38:08.786645+0000","last_became_peered":"2024-05-10T04:38:08.786645+0000","last_unstale":"2024-05-10T04:39:11.449579+0000","last_undegraded":"2024-05-10T04:39:11.449579+0000","last_fullsized":"2024-05-10T04:39:11.449579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:35.060271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446479+0000","last_change":"2024-05-10T04:38:08.769233+0000","last_active":"2024-05-10T04:39:11.446479+0000","last_peered":"2024-05-10T04:39:11.446479+0000","last_clean":"2024-05-10T04:39:11.446479+0000","last_became_active":"2024-05-10T04:38:08.768936+0000","last_became_peered":"2024-05-10T04:38:08.768936+0000","last_unstale":"2024-05-10T04:39:11.446479+0000","last_undegraded":"2024-05-10T04:39:11.446479+0000","last_fullsized":"2024-05-10T04:39:11.446479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:08.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443044+0000","last_change":"2024-05-10T04:38:08.788189+0000","last_active":"2024-05-10T04:39:11.443044+0000","last_peered":"2024-05-10T04:39:11.443044+0000","last_clean":"2024-05-10T04:39:11.443044+0000","last_became_active":"2024-05-10T04:38:08.787806+0000","last_became_peered":"2024-05-10T04:38:08.787806+0000","last_unstale":"2024-05-10T04:39:11.443044+0000","last_undegraded":"2024-05-10T04:39:11.443044+0000","last_fullsized":"2024-05-10T04:39:11.443044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:00:51.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280050,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74877,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":665,"num_read_kb":436,"num_write":240,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":224631,"data_compressed":4791,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481821,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":974848,"data_stored":485047,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64600000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78800000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75700000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":0.874,"15min":0.874},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.372,"15min":0.372},"max":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84299999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92200000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.68500000000000005,"15min":0.69599999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89400000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.71099999999999997,"15min":0.72699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":1.02,"15min":1.02},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.391,"15min":1.391},"last":1.0680000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.875},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96799999999999997,"5min":1.829,"15min":1.829},"last":0.94599999999999995}]},{"osd":8,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.107}]},{"osd":9,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1579999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.012,"5min":1.117,"15min":1.117},"last":0.92400000000000004}]},{"osd":10,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.03,"5min":1.601,"15min":1.601},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.70999999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.1319999999999999,"5min":1.774,"15min":1.774},"last":1.1319999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":1139281,"data_compressed":54829,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.442}]},{"osd":1,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.497,"15min":0.496},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79000000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.34300000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63700000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81999999999999995,"5min":1.143,"15min":1.143},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.52600000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.497,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.625}]},{"osd":4,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":0.872,"15min":0.872},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95599999999999996,"5min":1.252,"15min":1.252},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78000000000000003,"15min":0.77800000000000002},"min":{"1min":0.60499999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.52200000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80600000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":0.85199999999999998},"min":{"1min":0.56899999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":1.296,"15min":1.296},"last":0.82099999999999995}]},{"osd":9,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.139,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90700000000000003,"5min":1.2,"15min":1.2},"last":0.66800000000000004}]},{"osd":11,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83699999999999997,"15min":0.83099999999999996},"min":{"1min":0.68799999999999994,"5min":0.621,"15min":0.621},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84599999999999997,"15min":0.83799999999999997},"min":{"1min":0.749,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.86799999999999999}]}]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":488616,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.44600000000000001,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55300000000000005,"5min":1.034,"15min":1.034},"last":0.442},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64900000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.498}]},{"osd":1,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90400000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.56599999999999995,"15min":0.58699999999999997},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.65900000000000003,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59699999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80500000000000005,"5min":1.101,"15min":1.101},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":1.411,"15min":1.411},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.29,"15min":1.29},"last":0.83899999999999997}]},{"osd":8,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93999999999999995,"5min":1.53,"15min":1.53},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91500000000000004,"5min":1.502,"15min":1.502},"last":0.748}]},{"osd":10,"last update":"Fri May 10 04:39:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98199999999999998,"5min":1.016,"15min":1.016},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97299999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78600000000000003}]},{"osd":11,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89800000000000002}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":978944,"data_stored":484786,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.439}]},{"osd":2,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.497,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65700000000000003}]},{"osd":3,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.66900000000000004,"15min":0.64400000000000002},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.59399999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.625}]},{"osd":5,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.748},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.70199999999999996},"min":{"1min":0.59099999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.67400000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.874},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77800000000000002}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.77400000000000002,"15min":0.76300000000000001},"min":{"1min":0.61799999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86299999999999999,"15min":0.85999999999999999},"min":{"1min":0.70899999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.089,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":0.88900000000000001}]},{"osd":10,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82399999999999995,"15min":0.81599999999999995},"min":{"1min":0.73099999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.83299999999999996,"15min":0.81899999999999995},"min":{"1min":0.754,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.85799999999999998}]},{"osd":11,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.81999999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":1040384,"data_stored":1081174,"data_compressed":53232,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.45700000000000002},"min":{"1min":0.33700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56399999999999995,"5min":3.532,"15min":3.532},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.59499999999999997,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.51100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.77400000000000002,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63800000000000001,"15min":0.60299999999999998},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78100000000000003,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.76900000000000002,"5min":1.927,"15min":1.927},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.755,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86499999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.75},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.59299999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94799999999999995,"5min":3.4860000000000002,"15min":3.4860000000000002},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.81000000000000005,"5min":3.996,"15min":3.996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.56399999999999995}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.65300000000000002}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":2.028,"15min":2.028},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91400000000000003},"min":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95299999999999996,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.68400000000000005}]},{"osd":10,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.88500000000000001,"15min":0.90300000000000002},"min":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91100000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.89500000000000002,"15min":0.91100000000000003},"min":{"1min":0.64700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":3.653,"15min":3.653},"last":0.76700000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79800000000000004}]}]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":479976,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.46200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.51900000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.45300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.44400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.69599999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.309,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.5,"15min":0.51900000000000002},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.53700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.64200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.625,"15min":0.629},"min":{"1min":0.497,"5min":0.437,"15min":0.437},"max":{"1min":0.746,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.63800000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.74099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64100000000000001,"15min":0.65000000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri May 10 04:39:33 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.73399999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.65200000000000002}]},{"osd":8,"last update":"Fri May 10 04:38:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.496}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.80000000000000004,"5min":1.083,"15min":1.083},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.80300000000000005,"5min":1.105,"15min":1.105},"last":0.64500000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1036288,"data_stored":548946,"data_compressed":19210,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53500000000000003,"15min":0.5},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri May 10 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.57399999999999995,"15min":0.54200000000000004},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92500000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72399999999999998}]},{"osd":3,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.57299999999999995},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri May 10 04:39:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.497,"15min":0.45000000000000001},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.878,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.82899999999999996,"5min":1.083,"15min":1.083},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.58499999999999996,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":1.123,"15min":1.123},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.69199999999999995,"15min":0.64900000000000002},"min":{"1min":0.55100000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1100000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.502,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.623},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.55500000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.89500000000000002}]},{"osd":7,"last update":"Fri May 10 04:39:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.374,"15min":0.374},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.67600000000000005,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99199999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.65400000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.52800000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.626,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1040000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.81599999999999995}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.58399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78400000000000003,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0049999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77700000000000002,"15min":0.77000000000000002},"min":{"1min":0.56399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.72499999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.097,"5min":1.163,"15min":1.163},"last":0.58499999999999996}]},{"osd":11,"last update":"Fri May 10 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":481825,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.673,"5min":2.673,"15min":2.673},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.55800000000000005,"15min":0.48699999999999999},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.307}]},{"osd":2,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.41599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.41699999999999998}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.33000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.38600000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.47599999999999998}]},{"osd":8,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.751}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69599999999999995,"15min":0.67300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.7190000000000003,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.628}]},{"osd":10,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.70599999999999996,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.753,"15min":0.72599999999999998},"min":{"1min":0.60599999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":0.57199999999999995}]},{"osd":11,"last update":"Fri May 10 04:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.752,"15min":0.72399999999999998},"min":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.72599999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":547996,"data_compressed":19210,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.48199999999999998},"min":{"1min":0.45500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76100000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.503,"15min":0.46400000000000002},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri May 10 04:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.60999999999999999,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.94699999999999995}]},{"osd":3,"last update":"Fri May 10 04:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.84999999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.59999999999999998,"15min":0.55700000000000005},"min":{"1min":0.54400000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.45800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89900000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.022}]},{"osd":7,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.095}]},{"osd":8,"last update":"Fri May 10 04:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.91300000000000003}]},{"osd":9,"last update":"Fri May 10 04:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.60599999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0249999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.57999999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri May 10 04:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.70199999999999996,"5min":0.438,"15min":0.438},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.999}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":479732,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.372,"15min":0.34399999999999997},"min":{"1min":0.29999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.29199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54100000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.83599999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.52600000000000002},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri May 10 04:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.46000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82199999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri May 10 04:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.58099999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70099999999999996}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91600000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68400000000000005}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77500000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.72399999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":485879,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":1.143,"15min":1.143},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.36099999999999999},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.38200000000000001,"15min":0.36199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.627,"5min":0.748,"15min":0.748},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.623,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":1.03,"15min":1.03},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.85999999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88300000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90700000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.50700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.069,"15min":1.069},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.57199999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.871,"5min":1.079,"15min":1.079},"last":1.133}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63300000000000001,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.378,"15min":0.378},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0640000000000001}]},{"osd":8,"last update":"Fri May 10 04:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.52800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88500000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.99099999999999999}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.80300000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.032},{"interface":"front","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":1.1859999999999999}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.71799999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0069999999999999,"5min":1.012,"15min":1.012},"last":1.165},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.1539999999999999}]},{"osd":11,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.95299999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":1048576,"data_stored":1076538,"data_compressed":53232,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68999999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85499999999999998,"5min":1.008,"15min":1.008},"last":0.63}]},{"osd":4,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81599999999999995,"5min":1.089,"15min":1.089},"last":0.749},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84499999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.503},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.51500000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.002},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.56899999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90400000000000003,"5min":1.054,"15min":1.054},"last":0.877},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64300000000000002,"15min":0.63100000000000001},"min":{"1min":0.60399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.73499999999999999}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri May 10 04:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92000000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.84499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69644,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68694,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69942,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:39:43.781 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-10T04:39:43.781 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-10T04:39:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:43 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2502446992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:43 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3521362722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:43 smithi161 ceph-mon[46912]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:43.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:43 smithi161 ceph-mon[46912]: pgmap v274: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-10T04:39:43.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:43 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2502446992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:43.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:43 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3521362722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:43.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:43 smithi181 ceph-mon[47314]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:43.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:43 smithi181 ceph-mon[47314]: pgmap v274: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-10T04:39:43.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2810465808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98102950 msgr2=0x7f9b98109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2810465808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98102950 0x7f9b98109250 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000b0a0 tx=0x7f9b9000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2810465808 shutdown_connections 2024-05-10T04:39:43.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2810465808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98102950 0x7f9b98109250 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2810465808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98101110 0x7f9b98101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2810465808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b98102000 0x7f9b98102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2810465808 >> 172.21.15.150:0/2810465808 conn(0x7f9b980fb8f0 msgr2=0x7f9b980fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2810465808 shutdown_connections 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2810465808 wait complete. 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 Processor -- start 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.914+0000 7f9b9f06d700 1 -- start start 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9f06d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98101110 0x7f9b98104ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9f06d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98102000 0x7f9b98103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9f06d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b98102950 0x7f9b98103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9f06d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98103f60 con 0x7f9b98101110 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9f06d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b981040d0 con 0x7f9b98102000 2024-05-10T04:39:43.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9f06d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b981a23a0 con 0x7f9b98102950 2024-05-10T04:39:43.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98101110 0x7f9b98104ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98101110 0x7f9b98104ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:47994/0 (socket says 172.21.15.150:47994) 2024-05-10T04:39:43.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 -- 172.21.15.150:0/2228840126 learned_addr learned my addr 172.21.15.150:0/2228840126 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:43.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9d60a700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b98102950 0x7f9b98103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b97fff700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98102000 0x7f9b98103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 -- 172.21.15.150:0/2228840126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b98102950 msgr2=0x7f9b98103880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b98102950 0x7f9b98103880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 -- 172.21.15.150:0/2228840126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98102000 msgr2=0x7f9b98103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98102000 0x7f9b98103340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 -- 172.21.15.150:0/2228840126 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90009d00 con 0x7f9b98101110 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b97fff700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98102000 0x7f9b98103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9d60a700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b98102950 0x7f9b98103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9ce09700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98101110 0x7f9b98104ca0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d980 tx=0x7f9b8c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b95ffb700 1 -- 172.21.15.150:0/2228840126 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c009b40 con 0x7f9b98101110 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b95ffb700 1 -- 172.21.15.150:0/2228840126 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8c00de60 con 0x7f9b98101110 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b981a2600 con 0x7f9b98101110 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.915+0000 7f9b95ffb700 1 -- 172.21.15.150:0/2228840126 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00f620 con 0x7f9b98101110 2024-05-10T04:39:43.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.916+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b981a2b30 con 0x7f9b98101110 2024-05-10T04:39:43.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.916+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b980678b0 con 0x7f9b98101110 2024-05-10T04:39:43.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.918+0000 7f9b95ffb700 1 -- 172.21.15.150:0/2228840126 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9b8c010460 con 0x7f9b98101110 2024-05-10T04:39:43.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.918+0000 7f9b95ffb700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b880a1230 0x7f9b880a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:43.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.918+0000 7f9b97fff700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b880a1230 0x7f9b880a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:43.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.918+0000 7f9b95ffb700 1 -- 172.21.15.150:0/2228840126 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9b8c0cc8c0 con 0x7f9b98101110 2024-05-10T04:39:43.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.919+0000 7f9b97fff700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b880a1230 0x7f9b880a3740 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9b98104640 tx=0x7f9b84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:43.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:43.923+0000 7f9b95ffb700 1 -- 172.21.15.150:0/2228840126 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b8c095be0 con 0x7f9b98101110 2024-05-10T04:39:44.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.116+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 --> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b980fd8f0 con 0x7f9b880a1230 2024-05-10T04:39:44.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.128+0000 7f9b95ffb700 1 -- 172.21.15.150:0/2228840126 <== mgr.14152 v2:172.21.15.150:6800/3064659046 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427129 (secure 0 0 0) 0x7f9b980fd8f0 con 0x7f9b880a1230 2024-05-10T04:39:44.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:44.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b880a1230 msgr2=0x7f9b880a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b880a1230 0x7f9b880a3740 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9b98104640 tx=0x7f9b84009260 comp rx=0 tx=0).stop 2024-05-10T04:39:44.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98101110 msgr2=0x7f9b98104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98101110 0x7f9b98104ca0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d980 tx=0x7f9b8c00dd40 comp rx=0 tx=0).stop 2024-05-10T04:39:44.137 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 shutdown_connections 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b880a1230 0x7f9b880a3740 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b98102950 0x7f9b98103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b98101110 0x7f9b98104ca0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 --2- 172.21.15.150:0/2228840126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98102000 0x7f9b98103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 >> 172.21.15.150:0/2228840126 conn(0x7f9b980fb8f0 msgr2=0x7f9b980fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 shutdown_connections 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.132+0000 7f9b9f06d700 1 -- 172.21.15.150:0/2228840126 wait complete. 2024-05-10T04:39:44.138 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-05-10T04:39:44.146 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-05-10T04:39:43.330411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665479,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":791,"num_read_kb":545,"num_write":423,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182363,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360692,"kb_used_data":11872,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581132,"statfs":{"total":1151940427776,"available":1151571079168,"internally_reserved":0,"allocated":12156928,"data_stored":7779796,"data_compressed":323004,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19127,"internal_metadata":357021001},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4942,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":76,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003958"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445850+0000","last_change":"2024-05-10T04:38:08.784019+0000","last_active":"2024-05-10T04:39:11.445850+0000","last_peered":"2024-05-10T04:39:11.445850+0000","last_clean":"2024-05-10T04:39:11.445850+0000","last_became_active":"2024-05-10T04:38:08.783585+0000","last_became_peered":"2024-05-10T04:38:08.783585+0000","last_unstale":"2024-05-10T04:39:11.445850+0000","last_undegraded":"2024-05-10T04:39:11.445850+0000","last_fullsized":"2024-05-10T04:39:11.445850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:56.963068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439378+0000","last_change":"2024-05-10T04:38:08.777364+0000","last_active":"2024-05-10T04:39:11.439378+0000","last_peered":"2024-05-10T04:39:11.439378+0000","last_clean":"2024-05-10T04:39:11.439378+0000","last_became_active":"2024-05-10T04:38:08.776488+0000","last_became_peered":"2024-05-10T04:38:08.776488+0000","last_unstale":"2024-05-10T04:39:11.439378+0000","last_undegraded":"2024-05-10T04:39:11.439378+0000","last_fullsized":"2024-05-10T04:39:11.439378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:21:40.088812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439228+0000","last_change":"2024-05-10T04:38:08.777169+0000","last_active":"2024-05-10T04:39:11.439228+0000","last_peered":"2024-05-10T04:39:11.439228+0000","last_clean":"2024-05-10T04:39:11.439228+0000","last_became_active":"2024-05-10T04:38:08.776713+0000","last_became_peered":"2024-05-10T04:38:08.776713+0000","last_unstale":"2024-05-10T04:39:11.439228+0000","last_undegraded":"2024-05-10T04:39:11.439228+0000","last_fullsized":"2024-05-10T04:39:11.439228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:08:41.947577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443030+0000","last_change":"2024-05-10T04:38:08.788464+0000","last_active":"2024-05-10T04:39:11.443030+0000","last_peered":"2024-05-10T04:39:11.443030+0000","last_clean":"2024-05-10T04:39:11.443030+0000","last_became_active":"2024-05-10T04:38:08.787811+0000","last_became_peered":"2024-05-10T04:38:08.787811+0000","last_unstale":"2024-05-10T04:39:11.443030+0000","last_undegraded":"2024-05-10T04:39:11.443030+0000","last_fullsized":"2024-05-10T04:39:11.443030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:22.574882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437946+0000","last_change":"2024-05-10T04:38:08.772904+0000","last_active":"2024-05-10T04:39:11.437946+0000","last_peered":"2024-05-10T04:39:11.437946+0000","last_clean":"2024-05-10T04:39:11.437946+0000","last_became_active":"2024-05-10T04:38:08.772503+0000","last_became_peered":"2024-05-10T04:38:08.772503+0000","last_unstale":"2024-05-10T04:39:11.437946+0000","last_undegraded":"2024-05-10T04:39:11.437946+0000","last_fullsized":"2024-05-10T04:39:11.437946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:27:58.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442107+0000","last_change":"2024-05-10T04:38:08.779442+0000","last_active":"2024-05-10T04:39:11.442107+0000","last_peered":"2024-05-10T04:39:11.442107+0000","last_clean":"2024-05-10T04:39:11.442107+0000","last_became_active":"2024-05-10T04:38:08.777348+0000","last_became_peered":"2024-05-10T04:38:08.777348+0000","last_unstale":"2024-05-10T04:39:11.442107+0000","last_undegraded":"2024-05-10T04:39:11.442107+0000","last_fullsized":"2024-05-10T04:39:11.442107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:36:25.619031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446022+0000","last_change":"2024-05-10T04:38:08.775646+0000","last_active":"2024-05-10T04:39:11.446022+0000","last_peered":"2024-05-10T04:39:11.446022+0000","last_clean":"2024-05-10T04:39:11.446022+0000","last_became_active":"2024-05-10T04:38:08.768229+0000","last_became_peered":"2024-05-10T04:38:08.768229+0000","last_unstale":"2024-05-10T04:39:11.446022+0000","last_undegraded":"2024-05-10T04:39:11.446022+0000","last_fullsized":"2024-05-10T04:39:11.446022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:50:14.653065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450030+0000","last_change":"2024-05-10T04:38:08.771012+0000","last_active":"2024-05-10T04:39:11.450030+0000","last_peered":"2024-05-10T04:39:11.450030+0000","last_clean":"2024-05-10T04:39:11.450030+0000","last_became_active":"2024-05-10T04:38:08.770674+0000","last_became_peered":"2024-05-10T04:38:08.770674+0000","last_unstale":"2024-05-10T04:39:11.450030+0000","last_undegraded":"2024-05-10T04:39:11.450030+0000","last_fullsized":"2024-05-10T04:39:11.450030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:14.389263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450011+0000","last_change":"2024-05-10T04:38:08.782109+0000","last_active":"2024-05-10T04:39:11.450011+0000","last_peered":"2024-05-10T04:39:11.450011+0000","last_clean":"2024-05-10T04:39:11.450011+0000","last_became_active":"2024-05-10T04:38:08.781717+0000","last_became_peered":"2024-05-10T04:38:08.781717+0000","last_unstale":"2024-05-10T04:39:11.450011+0000","last_undegraded":"2024-05-10T04:39:11.450011+0000","last_fullsized":"2024-05-10T04:39:11.450011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:25:17.505448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442749+0000","last_change":"2024-05-10T04:38:08.778429+0000","last_active":"2024-05-10T04:39:11.442749+0000","last_peered":"2024-05-10T04:39:11.442749+0000","last_clean":"2024-05-10T04:39:11.442749+0000","last_became_active":"2024-05-10T04:38:08.777874+0000","last_became_peered":"2024-05-10T04:38:08.777874+0000","last_unstale":"2024-05-10T04:39:11.442749+0000","last_undegraded":"2024-05-10T04:39:11.442749+0000","last_fullsized":"2024-05-10T04:39:11.442749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:20:19.704678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446510+0000","last_change":"2024-05-10T04:38:08.780104+0000","last_active":"2024-05-10T04:39:11.446510+0000","last_peered":"2024-05-10T04:39:11.446510+0000","last_clean":"2024-05-10T04:39:11.446510+0000","last_became_active":"2024-05-10T04:38:08.779298+0000","last_became_peered":"2024-05-10T04:38:08.779298+0000","last_unstale":"2024-05-10T04:39:11.446510+0000","last_undegraded":"2024-05-10T04:39:11.446510+0000","last_fullsized":"2024-05-10T04:39:11.446510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:37:41.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443351+0000","last_change":"2024-05-10T04:38:08.788605+0000","last_active":"2024-05-10T04:39:11.443351+0000","last_peered":"2024-05-10T04:39:11.443351+0000","last_clean":"2024-05-10T04:39:11.443351+0000","last_became_active":"2024-05-10T04:38:08.788207+0000","last_became_peered":"2024-05-10T04:38:08.788207+0000","last_unstale":"2024-05-10T04:39:11.443351+0000","last_undegraded":"2024-05-10T04:39:11.443351+0000","last_fullsized":"2024-05-10T04:39:11.443351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:35.000980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446423+0000","last_change":"2024-05-10T04:38:08.761953+0000","last_active":"2024-05-10T04:39:11.446423+0000","last_peered":"2024-05-10T04:39:11.446423+0000","last_clean":"2024-05-10T04:39:11.446423+0000","last_became_active":"2024-05-10T04:38:08.761585+0000","last_became_peered":"2024-05-10T04:38:08.761585+0000","last_unstale":"2024-05-10T04:39:11.446423+0000","last_undegraded":"2024-05-10T04:39:11.446423+0000","last_fullsized":"2024-05-10T04:39:11.446423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:53:25.246643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439022+0000","last_change":"2024-05-10T04:38:08.777656+0000","last_active":"2024-05-10T04:39:11.439022+0000","last_peered":"2024-05-10T04:39:11.439022+0000","last_clean":"2024-05-10T04:39:11.439022+0000","last_became_active":"2024-05-10T04:38:08.777332+0000","last_became_peered":"2024-05-10T04:38:08.777332+0000","last_unstale":"2024-05-10T04:39:11.439022+0000","last_undegraded":"2024-05-10T04:39:11.439022+0000","last_fullsized":"2024-05-10T04:39:11.439022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:17:52.878943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438927+0000","last_change":"2024-05-10T04:38:08.784934+0000","last_active":"2024-05-10T04:39:11.438927+0000","last_peered":"2024-05-10T04:39:11.438927+0000","last_clean":"2024-05-10T04:39:11.438927+0000","last_became_active":"2024-05-10T04:38:08.784644+0000","last_became_peered":"2024-05-10T04:38:08.784644+0000","last_unstale":"2024-05-10T04:39:11.438927+0000","last_undegraded":"2024-05-10T04:39:11.438927+0000","last_fullsized":"2024-05-10T04:39:11.438927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:43.889305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448828+0000","last_change":"2024-05-10T04:38:08.784110+0000","last_active":"2024-05-10T04:39:11.448828+0000","last_peered":"2024-05-10T04:39:11.448828+0000","last_clean":"2024-05-10T04:39:11.448828+0000","last_became_active":"2024-05-10T04:38:08.783779+0000","last_became_peered":"2024-05-10T04:38:08.783779+0000","last_unstale":"2024-05-10T04:39:11.448828+0000","last_undegraded":"2024-05-10T04:39:11.448828+0000","last_fullsized":"2024-05-10T04:39:11.448828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:29:24.233024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445229+0000","last_change":"2024-05-10T04:38:08.788685+0000","last_active":"2024-05-10T04:39:11.445229+0000","last_peered":"2024-05-10T04:39:11.445229+0000","last_clean":"2024-05-10T04:39:11.445229+0000","last_became_active":"2024-05-10T04:38:08.788122+0000","last_became_peered":"2024-05-10T04:38:08.788122+0000","last_unstale":"2024-05-10T04:39:11.445229+0000","last_undegraded":"2024-05-10T04:39:11.445229+0000","last_fullsized":"2024-05-10T04:39:11.445229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:21:01.098784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444179+0000","last_change":"2024-05-10T04:38:08.776330+0000","last_active":"2024-05-10T04:39:11.444179+0000","last_peered":"2024-05-10T04:39:11.444179+0000","last_clean":"2024-05-10T04:39:11.444179+0000","last_became_active":"2024-05-10T04:38:08.776005+0000","last_became_peered":"2024-05-10T04:38:08.776005+0000","last_unstale":"2024-05-10T04:39:11.444179+0000","last_undegraded":"2024-05-10T04:39:11.444179+0000","last_fullsized":"2024-05-10T04:39:11.444179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:46:14.407766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449082+0000","last_change":"2024-05-10T04:38:08.772811+0000","last_active":"2024-05-10T04:39:11.449082+0000","last_peered":"2024-05-10T04:39:11.449082+0000","last_clean":"2024-05-10T04:39:11.449082+0000","last_became_active":"2024-05-10T04:38:08.772462+0000","last_became_peered":"2024-05-10T04:38:08.772462+0000","last_unstale":"2024-05-10T04:39:11.449082+0000","last_undegraded":"2024-05-10T04:39:11.449082+0000","last_fullsized":"2024-05-10T04:39:11.449082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:59:39.703137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447478+0000","last_change":"2024-05-10T04:38:12.799483+0000","last_active":"2024-05-10T04:39:11.447478+0000","last_peered":"2024-05-10T04:39:11.447478+0000","last_clean":"2024-05-10T04:39:11.447478+0000","last_became_active":"2024-05-10T04:38:12.799049+0000","last_became_peered":"2024-05-10T04:38:12.799049+0000","last_unstale":"2024-05-10T04:39:11.447478+0000","last_undegraded":"2024-05-10T04:39:11.447478+0000","last_fullsized":"2024-05-10T04:39:11.447478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:28:00.380645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:39.065705+0000","last_change":"2024-05-10T04:38:08.345935+0000","last_active":"2024-05-10T04:39:39.065705+0000","last_peered":"2024-05-10T04:39:39.065705+0000","last_clean":"2024-05-10T04:39:39.065705+0000","last_became_active":"2024-05-10T04:38:08.345580+0000","last_became_peered":"2024-05-10T04:38:08.345580+0000","last_unstale":"2024-05-10T04:39:39.065705+0000","last_undegraded":"2024-05-10T04:39:39.065705+0000","last_fullsized":"2024-05-10T04:39:39.065705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:11.527710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449675+0000","last_change":"2024-05-10T04:38:08.771072+0000","last_active":"2024-05-10T04:39:11.449675+0000","last_peered":"2024-05-10T04:39:11.449675+0000","last_clean":"2024-05-10T04:39:11.449675+0000","last_became_active":"2024-05-10T04:38:08.770711+0000","last_became_peered":"2024-05-10T04:38:08.770711+0000","last_unstale":"2024-05-10T04:39:11.449675+0000","last_undegraded":"2024-05-10T04:39:11.449675+0000","last_fullsized":"2024-05-10T04:39:11.449675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:53:44.803219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.452968+0000","last_change":"2024-05-10T04:39:12.452968+0000","last_active":"2024-05-10T04:39:12.452968+0000","last_peered":"2024-05-10T04:39:12.452968+0000","last_clean":"2024-05-10T04:39:12.452968+0000","last_became_active":"2024-05-10T04:39:12.452621+0000","last_became_peered":"2024-05-10T04:39:12.452621+0000","last_unstale":"2024-05-10T04:39:12.452968+0000","last_undegraded":"2024-05-10T04:39:12.452968+0000","last_fullsized":"2024-05-10T04:39:12.452968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:04:29.312810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449726+0000","last_change":"2024-05-10T04:38:08.348806+0000","last_active":"2024-05-10T04:39:11.449726+0000","last_peered":"2024-05-10T04:39:11.449726+0000","last_clean":"2024-05-10T04:39:11.449726+0000","last_became_active":"2024-05-10T04:38:08.348513+0000","last_became_peered":"2024-05-10T04:38:08.348513+0000","last_unstale":"2024-05-10T04:39:11.449726+0000","last_undegraded":"2024-05-10T04:39:11.449726+0000","last_fullsized":"2024-05-10T04:39:11.449726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:36:45.474403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439225+0000","last_change":"2024-05-10T04:38:08.782738+0000","last_active":"2024-05-10T04:39:11.439225+0000","last_peered":"2024-05-10T04:39:11.439225+0000","last_clean":"2024-05-10T04:39:11.439225+0000","last_became_active":"2024-05-10T04:38:08.782375+0000","last_became_peered":"2024-05-10T04:38:08.782375+0000","last_unstale":"2024-05-10T04:39:11.439225+0000","last_undegraded":"2024-05-10T04:39:11.439225+0000","last_fullsized":"2024-05-10T04:39:11.439225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:00:16.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442600+0000","last_change":"2024-05-10T04:38:12.801322+0000","last_active":"2024-05-10T04:39:11.442600+0000","last_peered":"2024-05-10T04:39:11.442600+0000","last_clean":"2024-05-10T04:39:11.442600+0000","last_became_active":"2024-05-10T04:38:12.800388+0000","last_became_peered":"2024-05-10T04:38:12.800388+0000","last_unstale":"2024-05-10T04:39:11.442600+0000","last_undegraded":"2024-05-10T04:39:11.442600+0000","last_fullsized":"2024-05-10T04:39:11.442600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:58:57.678691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438911+0000","last_change":"2024-05-10T04:38:08.347997+0000","last_active":"2024-05-10T04:39:11.438911+0000","last_peered":"2024-05-10T04:39:11.438911+0000","last_clean":"2024-05-10T04:39:11.438911+0000","last_became_active":"2024-05-10T04:38:08.347469+0000","last_became_peered":"2024-05-10T04:38:08.347469+0000","last_unstale":"2024-05-10T04:39:11.438911+0000","last_undegraded":"2024-05-10T04:39:11.438911+0000","last_fullsized":"2024-05-10T04:39:11.438911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:59:39.122148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449614+0000","last_change":"2024-05-10T04:38:08.781068+0000","last_active":"2024-05-10T04:39:11.449614+0000","last_peered":"2024-05-10T04:39:11.449614+0000","last_clean":"2024-05-10T04:39:11.449614+0000","last_became_active":"2024-05-10T04:38:08.780671+0000","last_became_peered":"2024-05-10T04:38:08.780671+0000","last_unstale":"2024-05-10T04:39:11.449614+0000","last_undegraded":"2024-05-10T04:39:11.449614+0000","last_fullsized":"2024-05-10T04:39:11.449614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442827+0000","last_change":"2024-05-10T04:38:12.801702+0000","last_active":"2024-05-10T04:39:11.442827+0000","last_peered":"2024-05-10T04:39:11.442827+0000","last_clean":"2024-05-10T04:39:11.442827+0000","last_became_active":"2024-05-10T04:38:12.801031+0000","last_became_peered":"2024-05-10T04:38:12.801031+0000","last_unstale":"2024-05-10T04:39:11.442827+0000","last_undegraded":"2024-05-10T04:39:11.442827+0000","last_fullsized":"2024-05-10T04:39:11.442827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:05:07.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437865+0000","last_change":"2024-05-10T04:38:08.347981+0000","last_active":"2024-05-10T04:39:11.437865+0000","last_peered":"2024-05-10T04:39:11.437865+0000","last_clean":"2024-05-10T04:39:11.437865+0000","last_became_active":"2024-05-10T04:38:08.346915+0000","last_became_peered":"2024-05-10T04:38:08.346915+0000","last_unstale":"2024-05-10T04:39:11.437865+0000","last_undegraded":"2024-05-10T04:39:11.437865+0000","last_fullsized":"2024-05-10T04:39:11.437865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:00:52.383734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444542+0000","last_change":"2024-05-10T04:38:08.779164+0000","last_active":"2024-05-10T04:39:11.444542+0000","last_peered":"2024-05-10T04:39:11.444542+0000","last_clean":"2024-05-10T04:39:11.444542+0000","last_became_active":"2024-05-10T04:38:08.778851+0000","last_became_peered":"2024-05-10T04:38:08.778851+0000","last_unstale":"2024-05-10T04:39:11.444542+0000","last_undegraded":"2024-05-10T04:39:11.444542+0000","last_fullsized":"2024-05-10T04:39:11.444542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:01.683566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.451673+0000","last_change":"2024-05-10T04:39:12.451673+0000","last_active":"2024-05-10T04:39:12.451673+0000","last_peered":"2024-05-10T04:39:12.451673+0000","last_clean":"2024-05-10T04:39:12.451673+0000","last_became_active":"2024-05-10T04:39:12.451181+0000","last_became_peered":"2024-05-10T04:39:12.451181+0000","last_unstale":"2024-05-10T04:39:12.451673+0000","last_undegraded":"2024-05-10T04:39:12.451673+0000","last_fullsized":"2024-05-10T04:39:12.451673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:46:30.325584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:39.041629+0000","last_change":"2024-05-10T04:38:08.341633+0000","last_active":"2024-05-10T04:39:39.041629+0000","last_peered":"2024-05-10T04:39:39.041629+0000","last_clean":"2024-05-10T04:39:39.041629+0000","last_became_active":"2024-05-10T04:38:08.341220+0000","last_became_peered":"2024-05-10T04:38:08.341220+0000","last_unstale":"2024-05-10T04:39:39.041629+0000","last_undegraded":"2024-05-10T04:39:39.041629+0000","last_fullsized":"2024-05-10T04:39:39.041629+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:36.861191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437912+0000","last_change":"2024-05-10T04:38:08.790969+0000","last_active":"2024-05-10T04:39:11.437912+0000","last_peered":"2024-05-10T04:39:11.437912+0000","last_clean":"2024-05-10T04:39:11.437912+0000","last_became_active":"2024-05-10T04:38:08.790539+0000","last_became_peered":"2024-05-10T04:38:08.790539+0000","last_unstale":"2024-05-10T04:39:11.437912+0000","last_undegraded":"2024-05-10T04:39:11.437912+0000","last_fullsized":"2024-05-10T04:39:11.437912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:13:50.174798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.453441+0000","last_change":"2024-05-10T04:39:12.453441+0000","last_active":"2024-05-10T04:39:12.453441+0000","last_peered":"2024-05-10T04:39:12.453441+0000","last_clean":"2024-05-10T04:39:12.453441+0000","last_became_active":"2024-05-10T04:39:12.452879+0000","last_became_peered":"2024-05-10T04:39:12.452879+0000","last_unstale":"2024-05-10T04:39:12.453441+0000","last_undegraded":"2024-05-10T04:39:12.453441+0000","last_fullsized":"2024-05-10T04:39:12.453441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:17:06.118264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442029+0000","last_change":"2024-05-10T04:38:08.342449+0000","last_active":"2024-05-10T04:39:11.442029+0000","last_peered":"2024-05-10T04:39:11.442029+0000","last_clean":"2024-05-10T04:39:11.442029+0000","last_became_active":"2024-05-10T04:38:08.342043+0000","last_became_peered":"2024-05-10T04:38:08.342043+0000","last_unstale":"2024-05-10T04:39:11.442029+0000","last_undegraded":"2024-05-10T04:39:11.442029+0000","last_fullsized":"2024-05-10T04:39:11.442029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:22:31.161203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444956+0000","last_change":"2024-05-10T04:38:08.783759+0000","last_active":"2024-05-10T04:39:11.444956+0000","last_peered":"2024-05-10T04:39:11.444956+0000","last_clean":"2024-05-10T04:39:11.444956+0000","last_became_active":"2024-05-10T04:38:08.783419+0000","last_became_peered":"2024-05-10T04:38:08.783419+0000","last_unstale":"2024-05-10T04:39:11.444956+0000","last_undegraded":"2024-05-10T04:39:11.444956+0000","last_fullsized":"2024-05-10T04:39:11.444956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:50:16.114017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441404+0000","last_change":"2024-05-10T04:38:12.803018+0000","last_active":"2024-05-10T04:39:11.441404+0000","last_peered":"2024-05-10T04:39:11.441404+0000","last_clean":"2024-05-10T04:39:11.441404+0000","last_became_active":"2024-05-10T04:38:12.802546+0000","last_became_peered":"2024-05-10T04:38:12.802546+0000","last_unstale":"2024-05-10T04:39:11.441404+0000","last_undegraded":"2024-05-10T04:39:11.441404+0000","last_fullsized":"2024-05-10T04:39:11.441404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:50:40.676335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446042+0000","last_change":"2024-05-10T04:38:08.341404+0000","last_active":"2024-05-10T04:39:11.446042+0000","last_peered":"2024-05-10T04:39:11.446042+0000","last_clean":"2024-05-10T04:39:11.446042+0000","last_became_active":"2024-05-10T04:38:08.341013+0000","last_became_peered":"2024-05-10T04:38:08.341013+0000","last_unstale":"2024-05-10T04:39:11.446042+0000","last_undegraded":"2024-05-10T04:39:11.446042+0000","last_fullsized":"2024-05-10T04:39:11.446042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:45:43.513989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450121+0000","last_change":"2024-05-10T04:38:08.775650+0000","last_active":"2024-05-10T04:39:11.450121+0000","last_peered":"2024-05-10T04:39:11.450121+0000","last_clean":"2024-05-10T04:39:11.450121+0000","last_became_active":"2024-05-10T04:38:08.775320+0000","last_became_peered":"2024-05-10T04:38:08.775320+0000","last_unstale":"2024-05-10T04:39:11.450121+0000","last_undegraded":"2024-05-10T04:39:11.450121+0000","last_fullsized":"2024-05-10T04:39:11.450121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:49:07.715486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439300+0000","last_change":"2024-05-10T04:38:12.797168+0000","last_active":"2024-05-10T04:39:11.439300+0000","last_peered":"2024-05-10T04:39:11.439300+0000","last_clean":"2024-05-10T04:39:11.439300+0000","last_became_active":"2024-05-10T04:38:12.796776+0000","last_became_peered":"2024-05-10T04:38:12.796776+0000","last_unstale":"2024-05-10T04:39:11.439300+0000","last_undegraded":"2024-05-10T04:39:11.439300+0000","last_fullsized":"2024-05-10T04:39:11.439300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:59:02.363398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444198+0000","last_change":"2024-05-10T04:38:08.344086+0000","last_active":"2024-05-10T04:39:11.444198+0000","last_peered":"2024-05-10T04:39:11.444198+0000","last_clean":"2024-05-10T04:39:11.444198+0000","last_became_active":"2024-05-10T04:38:08.343744+0000","last_became_peered":"2024-05-10T04:38:08.343744+0000","last_unstale":"2024-05-10T04:39:11.444198+0000","last_undegraded":"2024-05-10T04:39:11.444198+0000","last_fullsized":"2024-05-10T04:39:11.444198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:16:01.800590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443135+0000","last_change":"2024-05-10T04:38:08.773638+0000","last_active":"2024-05-10T04:39:11.443135+0000","last_peered":"2024-05-10T04:39:11.443135+0000","last_clean":"2024-05-10T04:39:11.443135+0000","last_became_active":"2024-05-10T04:38:08.773218+0000","last_became_peered":"2024-05-10T04:38:08.773218+0000","last_unstale":"2024-05-10T04:39:11.443135+0000","last_undegraded":"2024-05-10T04:39:11.443135+0000","last_fullsized":"2024-05-10T04:39:11.443135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:54:38.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448821+0000","last_change":"2024-05-10T04:38:12.796344+0000","last_active":"2024-05-10T04:39:11.448821+0000","last_peered":"2024-05-10T04:39:11.448821+0000","last_clean":"2024-05-10T04:39:11.448821+0000","last_became_active":"2024-05-10T04:38:12.795958+0000","last_became_peered":"2024-05-10T04:38:12.795958+0000","last_unstale":"2024-05-10T04:39:11.448821+0000","last_undegraded":"2024-05-10T04:39:11.448821+0000","last_fullsized":"2024-05-10T04:39:11.448821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:45.708078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":158,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:41.495403+0000","last_change":"2024-05-10T04:38:08.341327+0000","last_active":"2024-05-10T04:39:41.495403+0000","last_peered":"2024-05-10T04:39:41.495403+0000","last_clean":"2024-05-10T04:39:41.495403+0000","last_became_active":"2024-05-10T04:38:08.339606+0000","last_became_peered":"2024-05-10T04:38:08.339606+0000","last_unstale":"2024-05-10T04:39:41.495403+0000","last_undegraded":"2024-05-10T04:39:41.495403+0000","last_fullsized":"2024-05-10T04:39:41.495403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:22:40.212212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449986+0000","last_change":"2024-05-10T04:38:08.781195+0000","last_active":"2024-05-10T04:39:11.449986+0000","last_peered":"2024-05-10T04:39:11.449986+0000","last_clean":"2024-05-10T04:39:11.449986+0000","last_became_active":"2024-05-10T04:38:08.780779+0000","last_became_peered":"2024-05-10T04:38:08.780779+0000","last_unstale":"2024-05-10T04:39:11.449986+0000","last_undegraded":"2024-05-10T04:39:11.449986+0000","last_fullsized":"2024-05-10T04:39:11.449986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:24.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442936+0000","last_change":"2024-05-10T04:38:12.799494+0000","last_active":"2024-05-10T04:39:11.442936+0000","last_peered":"2024-05-10T04:39:11.442936+0000","last_clean":"2024-05-10T04:39:11.442936+0000","last_became_active":"2024-05-10T04:38:12.799071+0000","last_became_peered":"2024-05-10T04:38:12.799071+0000","last_unstale":"2024-05-10T04:39:11.442936+0000","last_undegraded":"2024-05-10T04:39:11.442936+0000","last_fullsized":"2024-05-10T04:39:11.442936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:24:00.814875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445725+0000","last_change":"2024-05-10T04:38:08.348569+0000","last_active":"2024-05-10T04:39:11.445725+0000","last_peered":"2024-05-10T04:39:11.445725+0000","last_clean":"2024-05-10T04:39:11.445725+0000","last_became_active":"2024-05-10T04:38:08.347942+0000","last_became_peered":"2024-05-10T04:38:08.347942+0000","last_unstale":"2024-05-10T04:39:11.445725+0000","last_undegraded":"2024-05-10T04:39:11.445725+0000","last_fullsized":"2024-05-10T04:39:11.445725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:32:17.865369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439473+0000","last_change":"2024-05-10T04:38:08.784447+0000","last_active":"2024-05-10T04:39:11.439473+0000","last_peered":"2024-05-10T04:39:11.439473+0000","last_clean":"2024-05-10T04:39:11.439473+0000","last_became_active":"2024-05-10T04:38:08.783989+0000","last_became_peered":"2024-05-10T04:38:08.783989+0000","last_unstale":"2024-05-10T04:39:11.439473+0000","last_undegraded":"2024-05-10T04:39:11.439473+0000","last_fullsized":"2024-05-10T04:39:11.439473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:41:37.511400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442966+0000","last_change":"2024-05-10T04:38:12.807478+0000","last_active":"2024-05-10T04:39:11.442966+0000","last_peered":"2024-05-10T04:39:11.442966+0000","last_clean":"2024-05-10T04:39:11.442966+0000","last_became_active":"2024-05-10T04:38:12.806801+0000","last_became_peered":"2024-05-10T04:38:12.806801+0000","last_unstale":"2024-05-10T04:39:11.442966+0000","last_undegraded":"2024-05-10T04:39:11.442966+0000","last_fullsized":"2024-05-10T04:39:11.442966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:33:32.168836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444446+0000","last_change":"2024-05-10T04:38:08.343610+0000","last_active":"2024-05-10T04:39:11.444446+0000","last_peered":"2024-05-10T04:39:11.444446+0000","last_clean":"2024-05-10T04:39:11.444446+0000","last_became_active":"2024-05-10T04:38:08.342880+0000","last_became_peered":"2024-05-10T04:38:08.342880+0000","last_unstale":"2024-05-10T04:39:11.444446+0000","last_undegraded":"2024-05-10T04:39:11.444446+0000","last_fullsized":"2024-05-10T04:39:11.444446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:45:11.502258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449795+0000","last_change":"2024-05-10T04:38:08.780354+0000","last_active":"2024-05-10T04:39:11.449795+0000","last_peered":"2024-05-10T04:39:11.449795+0000","last_clean":"2024-05-10T04:39:11.449795+0000","last_became_active":"2024-05-10T04:38:08.779777+0000","last_became_peered":"2024-05-10T04:38:08.779777+0000","last_unstale":"2024-05-10T04:39:11.449795+0000","last_undegraded":"2024-05-10T04:39:11.449795+0000","last_fullsized":"2024-05-10T04:39:11.449795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:07:28.243474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.450211+0000","last_change":"2024-05-10T04:39:12.450211+0000","last_active":"2024-05-10T04:39:12.450211+0000","last_peered":"2024-05-10T04:39:12.450211+0000","last_clean":"2024-05-10T04:39:12.450211+0000","last_became_active":"2024-05-10T04:39:12.449843+0000","last_became_peered":"2024-05-10T04:39:12.449843+0000","last_unstale":"2024-05-10T04:39:12.450211+0000","last_undegraded":"2024-05-10T04:39:12.450211+0000","last_fullsized":"2024-05-10T04:39:12.450211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:07.725273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'38","reported_seq":205,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:42.045516+0000","last_change":"2024-05-10T04:38:08.343873+0000","last_active":"2024-05-10T04:39:42.045516+0000","last_peered":"2024-05-10T04:39:42.045516+0000","last_clean":"2024-05-10T04:39:42.045516+0000","last_became_active":"2024-05-10T04:38:08.343360+0000","last_became_peered":"2024-05-10T04:38:08.343360+0000","last_unstale":"2024-05-10T04:39:42.045516+0000","last_undegraded":"2024-05-10T04:39:42.045516+0000","last_fullsized":"2024-05-10T04:39:42.045516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:06:10.816343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":141,"num_write":40,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437968+0000","last_change":"2024-05-10T04:38:08.783955+0000","last_active":"2024-05-10T04:39:11.437968+0000","last_peered":"2024-05-10T04:39:11.437968+0000","last_clean":"2024-05-10T04:39:11.437968+0000","last_became_active":"2024-05-10T04:38:08.779131+0000","last_became_peered":"2024-05-10T04:38:08.779131+0000","last_unstale":"2024-05-10T04:39:11.437968+0000","last_undegraded":"2024-05-10T04:39:11.437968+0000","last_fullsized":"2024-05-10T04:39:11.437968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:11:58.619571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445044+0000","last_change":"2024-05-10T04:38:12.795140+0000","last_active":"2024-05-10T04:39:11.445044+0000","last_peered":"2024-05-10T04:39:11.445044+0000","last_clean":"2024-05-10T04:39:11.445044+0000","last_became_active":"2024-05-10T04:38:12.794666+0000","last_became_peered":"2024-05-10T04:38:12.794666+0000","last_unstale":"2024-05-10T04:39:11.445044+0000","last_undegraded":"2024-05-10T04:39:11.445044+0000","last_fullsized":"2024-05-10T04:39:11.445044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:51:44.468321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447995+0000","last_change":"2024-05-10T04:38:08.338866+0000","last_active":"2024-05-10T04:39:11.447995+0000","last_peered":"2024-05-10T04:39:11.447995+0000","last_clean":"2024-05-10T04:39:11.447995+0000","last_became_active":"2024-05-10T04:38:08.338434+0000","last_became_peered":"2024-05-10T04:38:08.338434+0000","last_unstale":"2024-05-10T04:39:11.447995+0000","last_undegraded":"2024-05-10T04:39:11.447995+0000","last_fullsized":"2024-05-10T04:39:11.447995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:31:49.858034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446144+0000","last_change":"2024-05-10T04:38:08.768998+0000","last_active":"2024-05-10T04:39:11.446144+0000","last_peered":"2024-05-10T04:39:11.446144+0000","last_clean":"2024-05-10T04:39:11.446144+0000","last_became_active":"2024-05-10T04:38:08.768687+0000","last_became_peered":"2024-05-10T04:38:08.768687+0000","last_unstale":"2024-05-10T04:39:11.446144+0000","last_undegraded":"2024-05-10T04:39:11.446144+0000","last_fullsized":"2024-05-10T04:39:11.446144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:19.390746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442289+0000","last_change":"2024-05-10T04:38:12.801223+0000","last_active":"2024-05-10T04:39:11.442289+0000","last_peered":"2024-05-10T04:39:11.442289+0000","last_clean":"2024-05-10T04:39:11.442289+0000","last_became_active":"2024-05-10T04:38:12.800750+0000","last_became_peered":"2024-05-10T04:38:12.800750+0000","last_unstale":"2024-05-10T04:39:11.442289+0000","last_undegraded":"2024-05-10T04:39:11.442289+0000","last_fullsized":"2024-05-10T04:39:11.442289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:12:33.133528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445239+0000","last_change":"2024-05-10T04:38:08.344579+0000","last_active":"2024-05-10T04:39:11.445239+0000","last_peered":"2024-05-10T04:39:11.445239+0000","last_clean":"2024-05-10T04:39:11.445239+0000","last_became_active":"2024-05-10T04:38:08.344250+0000","last_became_peered":"2024-05-10T04:38:08.344250+0000","last_unstale":"2024-05-10T04:39:11.445239+0000","last_undegraded":"2024-05-10T04:39:11.445239+0000","last_fullsized":"2024-05-10T04:39:11.445239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:33:56.491732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445406+0000","last_change":"2024-05-10T04:38:08.784556+0000","last_active":"2024-05-10T04:39:11.445406+0000","last_peered":"2024-05-10T04:39:11.445406+0000","last_clean":"2024-05-10T04:39:11.445406+0000","last_became_active":"2024-05-10T04:38:08.784229+0000","last_became_peered":"2024-05-10T04:38:08.784229+0000","last_unstale":"2024-05-10T04:39:11.445406+0000","last_undegraded":"2024-05-10T04:39:11.445406+0000","last_fullsized":"2024-05-10T04:39:11.445406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:48:46.519837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443931+0000","last_change":"2024-05-10T04:38:12.805795+0000","last_active":"2024-05-10T04:39:11.443931+0000","last_peered":"2024-05-10T04:39:11.443931+0000","last_clean":"2024-05-10T04:39:11.443931+0000","last_became_active":"2024-05-10T04:38:12.805366+0000","last_became_peered":"2024-05-10T04:38:12.805366+0000","last_unstale":"2024-05-10T04:39:11.443931+0000","last_undegraded":"2024-05-10T04:39:11.443931+0000","last_fullsized":"2024-05-10T04:39:11.443931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:38:11.641598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:38.064666+0000","last_change":"2024-05-10T04:38:08.345396+0000","last_active":"2024-05-10T04:39:38.064666+0000","last_peered":"2024-05-10T04:39:38.064666+0000","last_clean":"2024-05-10T04:39:38.064666+0000","last_became_active":"2024-05-10T04:38:08.345076+0000","last_became_peered":"2024-05-10T04:38:08.345076+0000","last_unstale":"2024-05-10T04:39:38.064666+0000","last_undegraded":"2024-05-10T04:39:38.064666+0000","last_fullsized":"2024-05-10T04:39:38.064666+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:12:38.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":854,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446201+0000","last_change":"2024-05-10T04:38:08.791089+0000","last_active":"2024-05-10T04:39:11.446201+0000","last_peered":"2024-05-10T04:39:11.446201+0000","last_clean":"2024-05-10T04:39:11.446201+0000","last_became_active":"2024-05-10T04:38:08.790813+0000","last_became_peered":"2024-05-10T04:38:08.790813+0000","last_unstale":"2024-05-10T04:39:11.446201+0000","last_undegraded":"2024-05-10T04:39:11.446201+0000","last_fullsized":"2024-05-10T04:39:11.446201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:52:45.753196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446807+0000","last_change":"2024-05-10T04:38:12.803896+0000","last_active":"2024-05-10T04:39:11.446807+0000","last_peered":"2024-05-10T04:39:11.446807+0000","last_clean":"2024-05-10T04:39:11.446807+0000","last_became_active":"2024-05-10T04:38:12.803512+0000","last_became_peered":"2024-05-10T04:38:12.803512+0000","last_unstale":"2024-05-10T04:39:11.446807+0000","last_undegraded":"2024-05-10T04:39:11.446807+0000","last_fullsized":"2024-05-10T04:39:11.446807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:01:23.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.467762+0000","last_change":"2024-05-10T04:39:12.467304+0000","last_active":"2024-05-10T04:39:12.467762+0000","last_peered":"2024-05-10T04:39:12.467762+0000","last_clean":"2024-05-10T04:39:12.467762+0000","last_became_active":"2024-05-10T04:39:12.448415+0000","last_became_peered":"2024-05-10T04:39:12.448415+0000","last_unstale":"2024-05-10T04:39:12.467762+0000","last_undegraded":"2024-05-10T04:39:12.467762+0000","last_fullsized":"2024-05-10T04:39:12.467762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:03:20.063125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.762052+0000","last_change":"2024-05-10T04:39:12.762052+0000","last_active":"2024-05-10T04:39:12.762052+0000","last_peered":"2024-05-10T04:39:12.762052+0000","last_clean":"2024-05-10T04:39:12.762052+0000","last_became_active":"2024-05-10T04:39:12.761658+0000","last_became_peered":"2024-05-10T04:39:12.761658+0000","last_unstale":"2024-05-10T04:39:12.762052+0000","last_undegraded":"2024-05-10T04:39:12.762052+0000","last_fullsized":"2024-05-10T04:39:12.762052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:05:07.565757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443495+0000","last_change":"2024-05-10T04:38:12.796046+0000","last_active":"2024-05-10T04:39:11.443495+0000","last_peered":"2024-05-10T04:39:11.443495+0000","last_clean":"2024-05-10T04:39:11.443495+0000","last_became_active":"2024-05-10T04:38:12.795635+0000","last_became_peered":"2024-05-10T04:38:12.795635+0000","last_unstale":"2024-05-10T04:39:11.443495+0000","last_undegraded":"2024-05-10T04:39:11.443495+0000","last_fullsized":"2024-05-10T04:39:11.443495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:17:50.822404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438595+0000","last_change":"2024-05-10T04:38:08.348701+0000","last_active":"2024-05-10T04:39:11.438595+0000","last_peered":"2024-05-10T04:39:11.438595+0000","last_clean":"2024-05-10T04:39:11.438595+0000","last_became_active":"2024-05-10T04:38:08.348345+0000","last_became_peered":"2024-05-10T04:38:08.348345+0000","last_unstale":"2024-05-10T04:39:11.438595+0000","last_undegraded":"2024-05-10T04:39:11.438595+0000","last_fullsized":"2024-05-10T04:39:11.438595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:31.505996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.450401+0000","last_change":"2024-05-10T04:39:12.450401+0000","last_active":"2024-05-10T04:39:12.450401+0000","last_peered":"2024-05-10T04:39:12.450401+0000","last_clean":"2024-05-10T04:39:12.450401+0000","last_became_active":"2024-05-10T04:39:12.449999+0000","last_became_peered":"2024-05-10T04:39:12.449999+0000","last_unstale":"2024-05-10T04:39:12.450401+0000","last_undegraded":"2024-05-10T04:39:12.450401+0000","last_fullsized":"2024-05-10T04:39:12.450401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:42:46.589660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437683+0000","last_change":"2024-05-10T04:38:12.809215+0000","last_active":"2024-05-10T04:39:11.437683+0000","last_peered":"2024-05-10T04:39:11.437683+0000","last_clean":"2024-05-10T04:39:11.437683+0000","last_became_active":"2024-05-10T04:38:12.808717+0000","last_became_peered":"2024-05-10T04:38:12.808717+0000","last_unstale":"2024-05-10T04:39:11.437683+0000","last_undegraded":"2024-05-10T04:39:11.437683+0000","last_fullsized":"2024-05-10T04:39:11.437683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:29:40.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443605+0000","last_change":"2024-05-10T04:38:08.347770+0000","last_active":"2024-05-10T04:39:11.443605+0000","last_peered":"2024-05-10T04:39:11.443605+0000","last_clean":"2024-05-10T04:39:11.443605+0000","last_became_active":"2024-05-10T04:38:08.347444+0000","last_became_peered":"2024-05-10T04:38:08.347444+0000","last_unstale":"2024-05-10T04:39:11.443605+0000","last_undegraded":"2024-05-10T04:39:11.443605+0000","last_fullsized":"2024-05-10T04:39:11.443605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:29:49.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442442+0000","last_change":"2024-05-10T04:38:08.765358+0000","last_active":"2024-05-10T04:39:11.442442+0000","last_peered":"2024-05-10T04:39:11.442442+0000","last_clean":"2024-05-10T04:39:11.442442+0000","last_became_active":"2024-05-10T04:38:08.765090+0000","last_became_peered":"2024-05-10T04:38:08.765090+0000","last_unstale":"2024-05-10T04:39:11.442442+0000","last_undegraded":"2024-05-10T04:39:11.442442+0000","last_fullsized":"2024-05-10T04:39:11.442442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:55:25.722331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446434+0000","last_change":"2024-05-10T04:38:12.794765+0000","last_active":"2024-05-10T04:39:11.446434+0000","last_peered":"2024-05-10T04:39:11.446434+0000","last_clean":"2024-05-10T04:39:11.446434+0000","last_became_active":"2024-05-10T04:38:12.794400+0000","last_became_peered":"2024-05-10T04:38:12.794400+0000","last_unstale":"2024-05-10T04:39:11.446434+0000","last_undegraded":"2024-05-10T04:39:11.446434+0000","last_fullsized":"2024-05-10T04:39:11.446434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:15:16.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:38.065103+0000","last_change":"2024-05-10T04:38:08.348812+0000","last_active":"2024-05-10T04:39:38.065103+0000","last_peered":"2024-05-10T04:39:38.065103+0000","last_clean":"2024-05-10T04:39:38.065103+0000","last_became_active":"2024-05-10T04:38:08.348207+0000","last_became_peered":"2024-05-10T04:38:08.348207+0000","last_unstale":"2024-05-10T04:39:38.065103+0000","last_undegraded":"2024-05-10T04:39:38.065103+0000","last_fullsized":"2024-05-10T04:39:38.065103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:20:17.350094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442205+0000","last_change":"2024-05-10T04:38:08.785502+0000","last_active":"2024-05-10T04:39:11.442205+0000","last_peered":"2024-05-10T04:39:11.442205+0000","last_clean":"2024-05-10T04:39:11.442205+0000","last_became_active":"2024-05-10T04:38:08.785073+0000","last_became_peered":"2024-05-10T04:38:08.785073+0000","last_unstale":"2024-05-10T04:39:11.442205+0000","last_undegraded":"2024-05-10T04:39:11.442205+0000","last_fullsized":"2024-05-10T04:39:11.442205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:08:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446700+0000","last_change":"2024-05-10T04:38:12.809599+0000","last_active":"2024-05-10T04:39:11.446700+0000","last_peered":"2024-05-10T04:39:11.446700+0000","last_clean":"2024-05-10T04:39:11.446700+0000","last_became_active":"2024-05-10T04:38:12.809200+0000","last_became_peered":"2024-05-10T04:38:12.809200+0000","last_unstale":"2024-05-10T04:39:11.446700+0000","last_undegraded":"2024-05-10T04:39:11.446700+0000","last_fullsized":"2024-05-10T04:39:11.446700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:19:54.940517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444413+0000","last_change":"2024-05-10T04:38:08.779837+0000","last_active":"2024-05-10T04:39:11.444413+0000","last_peered":"2024-05-10T04:39:11.444413+0000","last_clean":"2024-05-10T04:39:11.444413+0000","last_became_active":"2024-05-10T04:38:08.779501+0000","last_became_peered":"2024-05-10T04:38:08.779501+0000","last_unstale":"2024-05-10T04:39:11.444413+0000","last_undegraded":"2024-05-10T04:39:11.444413+0000","last_fullsized":"2024-05-10T04:39:11.444413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:08:00.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:40.067333+0000","last_change":"2024-05-10T04:38:08.343649+0000","last_active":"2024-05-10T04:39:40.067333+0000","last_peered":"2024-05-10T04:39:40.067333+0000","last_clean":"2024-05-10T04:39:40.067333+0000","last_became_active":"2024-05-10T04:38:08.343292+0000","last_became_peered":"2024-05-10T04:38:08.343292+0000","last_unstale":"2024-05-10T04:39:40.067333+0000","last_undegraded":"2024-05-10T04:39:40.067333+0000","last_fullsized":"2024-05-10T04:39:40.067333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:17:53.716455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446345+0000","last_change":"2024-05-10T04:38:12.796552+0000","last_active":"2024-05-10T04:39:11.446345+0000","last_peered":"2024-05-10T04:39:11.446345+0000","last_clean":"2024-05-10T04:39:11.446345+0000","last_became_active":"2024-05-10T04:38:12.796151+0000","last_became_peered":"2024-05-10T04:38:12.796151+0000","last_unstale":"2024-05-10T04:39:11.446345+0000","last_undegraded":"2024-05-10T04:39:11.446345+0000","last_fullsized":"2024-05-10T04:39:11.446345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:35:05.520859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448787+0000","last_change":"2024-05-10T04:38:08.770957+0000","last_active":"2024-05-10T04:39:11.448787+0000","last_peered":"2024-05-10T04:39:11.448787+0000","last_clean":"2024-05-10T04:39:11.448787+0000","last_became_active":"2024-05-10T04:38:08.770572+0000","last_became_peered":"2024-05-10T04:38:08.770572+0000","last_unstale":"2024-05-10T04:39:11.448787+0000","last_undegraded":"2024-05-10T04:39:11.448787+0000","last_fullsized":"2024-05-10T04:39:11.448787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:24:21.421033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442857+0000","last_change":"2024-05-10T04:38:08.340788+0000","last_active":"2024-05-10T04:39:11.442857+0000","last_peered":"2024-05-10T04:39:11.442857+0000","last_clean":"2024-05-10T04:39:11.442857+0000","last_became_active":"2024-05-10T04:38:08.340407+0000","last_became_peered":"2024-05-10T04:38:08.340407+0000","last_unstale":"2024-05-10T04:39:11.442857+0000","last_undegraded":"2024-05-10T04:39:11.442857+0000","last_fullsized":"2024-05-10T04:39:11.442857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:30:05.276951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":329,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446409+0000","last_change":"2024-05-10T04:37:20.961282+0000","last_active":"2024-05-10T04:39:11.446409+0000","last_peered":"2024-05-10T04:39:11.446409+0000","last_clean":"2024-05-10T04:39:11.446409+0000","last_became_active":"2024-05-10T04:37:20.960828+0000","last_became_peered":"2024-05-10T04:37:20.960828+0000","last_unstale":"2024-05-10T04:39:11.446409+0000","last_undegraded":"2024-05-10T04:39:11.446409+0000","last_fullsized":"2024-05-10T04:39:11.446409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:35:00.248380+0000","last_clean_scrub_stamp":"2024-05-10T04:35:00.248380+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:08:04.013907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446236+0000","last_change":"2024-05-10T04:38:12.809899+0000","last_active":"2024-05-10T04:39:11.446236+0000","last_peered":"2024-05-10T04:39:11.446236+0000","last_clean":"2024-05-10T04:39:11.446236+0000","last_became_active":"2024-05-10T04:38:12.809441+0000","last_became_peered":"2024-05-10T04:38:12.809441+0000","last_unstale":"2024-05-10T04:39:11.446236+0000","last_undegraded":"2024-05-10T04:39:11.446236+0000","last_fullsized":"2024-05-10T04:39:11.446236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:57:48.378890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445906+0000","last_change":"2024-05-10T04:38:08.784838+0000","last_active":"2024-05-10T04:39:11.445906+0000","last_peered":"2024-05-10T04:39:11.445906+0000","last_clean":"2024-05-10T04:39:11.445906+0000","last_became_active":"2024-05-10T04:38:08.784464+0000","last_became_peered":"2024-05-10T04:38:08.784464+0000","last_unstale":"2024-05-10T04:39:11.445906+0000","last_undegraded":"2024-05-10T04:39:11.445906+0000","last_fullsized":"2024-05-10T04:39:11.445906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:03:00.335350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.470361+0000","last_change":"2024-05-10T04:39:12.469901+0000","last_active":"2024-05-10T04:39:12.470361+0000","last_peered":"2024-05-10T04:39:12.470361+0000","last_clean":"2024-05-10T04:39:12.470361+0000","last_became_active":"2024-05-10T04:39:12.454202+0000","last_became_peered":"2024-05-10T04:39:12.454202+0000","last_unstale":"2024-05-10T04:39:12.470361+0000","last_undegraded":"2024-05-10T04:39:12.470361+0000","last_fullsized":"2024-05-10T04:39:12.470361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:14:18.646199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441308+0000","last_change":"2024-05-10T04:38:12.801079+0000","last_active":"2024-05-10T04:39:11.441308+0000","last_peered":"2024-05-10T04:39:11.441308+0000","last_clean":"2024-05-10T04:39:11.441308+0000","last_became_active":"2024-05-10T04:38:12.800630+0000","last_became_peered":"2024-05-10T04:38:12.800630+0000","last_unstale":"2024-05-10T04:39:11.441308+0000","last_undegraded":"2024-05-10T04:39:11.441308+0000","last_fullsized":"2024-05-10T04:39:11.441308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:12:05.213620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438080+0000","last_change":"2024-05-10T04:38:08.348764+0000","last_active":"2024-05-10T04:39:11.438080+0000","last_peered":"2024-05-10T04:39:11.438080+0000","last_clean":"2024-05-10T04:39:11.438080+0000","last_became_active":"2024-05-10T04:38:08.347939+0000","last_became_peered":"2024-05-10T04:38:08.347939+0000","last_unstale":"2024-05-10T04:39:11.438080+0000","last_undegraded":"2024-05-10T04:39:11.438080+0000","last_fullsized":"2024-05-10T04:39:11.438080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:23:01.308255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446594+0000","last_change":"2024-05-10T04:38:08.771382+0000","last_active":"2024-05-10T04:39:11.446594+0000","last_peered":"2024-05-10T04:39:11.446594+0000","last_clean":"2024-05-10T04:39:11.446594+0000","last_became_active":"2024-05-10T04:38:08.771108+0000","last_became_peered":"2024-05-10T04:38:08.771108+0000","last_unstale":"2024-05-10T04:39:11.446594+0000","last_undegraded":"2024-05-10T04:39:11.446594+0000","last_fullsized":"2024-05-10T04:39:11.446594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:38:32.664640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.450061+0000","last_change":"2024-05-10T04:38:12.809582+0000","last_active":"2024-05-10T04:39:11.450061+0000","last_peered":"2024-05-10T04:39:11.450061+0000","last_clean":"2024-05-10T04:39:11.450061+0000","last_became_active":"2024-05-10T04:38:12.809134+0000","last_became_peered":"2024-05-10T04:38:12.809134+0000","last_unstale":"2024-05-10T04:39:11.450061+0000","last_undegraded":"2024-05-10T04:39:11.450061+0000","last_fullsized":"2024-05-10T04:39:11.450061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:31:07.694368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444107+0000","last_change":"2024-05-10T04:38:08.345903+0000","last_active":"2024-05-10T04:39:11.444107+0000","last_peered":"2024-05-10T04:39:11.444107+0000","last_clean":"2024-05-10T04:39:11.444107+0000","last_became_active":"2024-05-10T04:38:08.345371+0000","last_became_peered":"2024-05-10T04:38:08.345371+0000","last_unstale":"2024-05-10T04:39:11.444107+0000","last_undegraded":"2024-05-10T04:39:11.444107+0000","last_fullsized":"2024-05-10T04:39:11.444107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:25:03.581112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449208+0000","last_change":"2024-05-10T04:38:08.779853+0000","last_active":"2024-05-10T04:39:11.449208+0000","last_peered":"2024-05-10T04:39:11.449208+0000","last_clean":"2024-05-10T04:39:11.449208+0000","last_became_active":"2024-05-10T04:38:08.779531+0000","last_became_peered":"2024-05-10T04:38:08.779531+0000","last_unstale":"2024-05-10T04:39:11.449208+0000","last_undegraded":"2024-05-10T04:39:11.449208+0000","last_fullsized":"2024-05-10T04:39:11.449208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:22:05.079540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447323+0000","last_change":"2024-05-10T04:38:12.795970+0000","last_active":"2024-05-10T04:39:11.447323+0000","last_peered":"2024-05-10T04:39:11.447323+0000","last_clean":"2024-05-10T04:39:11.447323+0000","last_became_active":"2024-05-10T04:38:12.795497+0000","last_became_peered":"2024-05-10T04:38:12.795497+0000","last_unstale":"2024-05-10T04:39:11.447323+0000","last_undegraded":"2024-05-10T04:39:11.447323+0000","last_fullsized":"2024-05-10T04:39:11.447323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:57:11.174672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443466+0000","last_change":"2024-05-10T04:38:08.343268+0000","last_active":"2024-05-10T04:39:11.443466+0000","last_peered":"2024-05-10T04:39:11.443466+0000","last_clean":"2024-05-10T04:39:11.443466+0000","last_became_active":"2024-05-10T04:38:08.342952+0000","last_became_peered":"2024-05-10T04:38:08.342952+0000","last_unstale":"2024-05-10T04:39:11.443466+0000","last_undegraded":"2024-05-10T04:39:11.443466+0000","last_fullsized":"2024-05-10T04:39:11.443466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:55:49.326810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439365+0000","last_change":"2024-05-10T04:38:08.784276+0000","last_active":"2024-05-10T04:39:11.439365+0000","last_peered":"2024-05-10T04:39:11.439365+0000","last_clean":"2024-05-10T04:39:11.439365+0000","last_became_active":"2024-05-10T04:38:08.783951+0000","last_became_peered":"2024-05-10T04:38:08.783951+0000","last_unstale":"2024-05-10T04:39:11.439365+0000","last_undegraded":"2024-05-10T04:39:11.439365+0000","last_fullsized":"2024-05-10T04:39:11.439365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:45:43.992567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443129+0000","last_change":"2024-05-10T04:38:12.811689+0000","last_active":"2024-05-10T04:39:11.443129+0000","last_peered":"2024-05-10T04:39:11.443129+0000","last_clean":"2024-05-10T04:39:11.443129+0000","last_became_active":"2024-05-10T04:38:12.811138+0000","last_became_peered":"2024-05-10T04:38:12.811138+0000","last_unstale":"2024-05-10T04:39:11.443129+0000","last_undegraded":"2024-05-10T04:39:11.443129+0000","last_fullsized":"2024-05-10T04:39:11.443129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:13:09.291137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448922+0000","last_change":"2024-05-10T04:38:08.348738+0000","last_active":"2024-05-10T04:39:11.448922+0000","last_peered":"2024-05-10T04:39:11.448922+0000","last_clean":"2024-05-10T04:39:11.448922+0000","last_became_active":"2024-05-10T04:38:08.348382+0000","last_became_peered":"2024-05-10T04:38:08.348382+0000","last_unstale":"2024-05-10T04:39:11.448922+0000","last_undegraded":"2024-05-10T04:39:11.448922+0000","last_fullsized":"2024-05-10T04:39:11.448922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:51:02.230504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.437662+0000","last_change":"2024-05-10T04:38:08.781483+0000","last_active":"2024-05-10T04:39:11.437662+0000","last_peered":"2024-05-10T04:39:11.437662+0000","last_clean":"2024-05-10T04:39:11.437662+0000","last_became_active":"2024-05-10T04:38:08.777713+0000","last_became_peered":"2024-05-10T04:38:08.777713+0000","last_unstale":"2024-05-10T04:39:11.437662+0000","last_undegraded":"2024-05-10T04:39:11.437662+0000","last_fullsized":"2024-05-10T04:39:11.437662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:18:41.050901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444353+0000","last_change":"2024-05-10T04:38:12.800656+0000","last_active":"2024-05-10T04:39:11.444353+0000","last_peered":"2024-05-10T04:39:11.444353+0000","last_clean":"2024-05-10T04:39:11.444353+0000","last_became_active":"2024-05-10T04:38:12.800191+0000","last_became_peered":"2024-05-10T04:38:12.800191+0000","last_unstale":"2024-05-10T04:39:11.444353+0000","last_undegraded":"2024-05-10T04:39:11.444353+0000","last_fullsized":"2024-05-10T04:39:11.444353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T09:08:47.017642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448670+0000","last_change":"2024-05-10T04:38:08.344017+0000","last_active":"2024-05-10T04:39:11.448670+0000","last_peered":"2024-05-10T04:39:11.448670+0000","last_clean":"2024-05-10T04:39:11.448670+0000","last_became_active":"2024-05-10T04:38:08.343646+0000","last_became_peered":"2024-05-10T04:38:08.343646+0000","last_unstale":"2024-05-10T04:39:11.448670+0000","last_undegraded":"2024-05-10T04:39:11.448670+0000","last_fullsized":"2024-05-10T04:39:11.448670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:02:30.455019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.441892+0000","last_change":"2024-05-10T04:38:08.779065+0000","last_active":"2024-05-10T04:39:11.441892+0000","last_peered":"2024-05-10T04:39:11.441892+0000","last_clean":"2024-05-10T04:39:11.441892+0000","last_became_active":"2024-05-10T04:38:08.778685+0000","last_became_peered":"2024-05-10T04:38:08.778685+0000","last_unstale":"2024-05-10T04:39:11.441892+0000","last_undegraded":"2024-05-10T04:39:11.441892+0000","last_fullsized":"2024-05-10T04:39:11.441892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:25:58.780080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444508+0000","last_change":"2024-05-10T04:38:12.801524+0000","last_active":"2024-05-10T04:39:11.444508+0000","last_peered":"2024-05-10T04:39:11.444508+0000","last_clean":"2024-05-10T04:39:11.444508+0000","last_became_active":"2024-05-10T04:38:12.800777+0000","last_became_peered":"2024-05-10T04:38:12.800777+0000","last_unstale":"2024-05-10T04:39:11.444508+0000","last_undegraded":"2024-05-10T04:39:11.444508+0000","last_fullsized":"2024-05-10T04:39:11.444508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:07:14.139236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447769+0000","last_change":"2024-05-10T04:38:08.344785+0000","last_active":"2024-05-10T04:39:11.447769+0000","last_peered":"2024-05-10T04:39:11.447769+0000","last_clean":"2024-05-10T04:39:11.447769+0000","last_became_active":"2024-05-10T04:38:08.344337+0000","last_became_peered":"2024-05-10T04:38:08.344337+0000","last_unstale":"2024-05-10T04:39:11.447769+0000","last_undegraded":"2024-05-10T04:39:11.447769+0000","last_fullsized":"2024-05-10T04:39:11.447769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:20:47.682532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448971+0000","last_change":"2024-05-10T04:38:08.768686+0000","last_active":"2024-05-10T04:39:11.448971+0000","last_peered":"2024-05-10T04:39:11.448971+0000","last_clean":"2024-05-10T04:39:11.448971+0000","last_became_active":"2024-05-10T04:38:08.768317+0000","last_became_peered":"2024-05-10T04:38:08.768317+0000","last_unstale":"2024-05-10T04:39:11.448971+0000","last_undegraded":"2024-05-10T04:39:11.448971+0000","last_fullsized":"2024-05-10T04:39:11.448971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:42:33.670912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443063+0000","last_change":"2024-05-10T04:38:12.809825+0000","last_active":"2024-05-10T04:39:11.443063+0000","last_peered":"2024-05-10T04:39:11.443063+0000","last_clean":"2024-05-10T04:39:11.443063+0000","last_became_active":"2024-05-10T04:38:12.809378+0000","last_became_peered":"2024-05-10T04:38:12.809378+0000","last_unstale":"2024-05-10T04:39:11.443063+0000","last_undegraded":"2024-05-10T04:39:11.443063+0000","last_fullsized":"2024-05-10T04:39:11.443063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:53:28.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438131+0000","last_change":"2024-05-10T04:38:08.347871+0000","last_active":"2024-05-10T04:39:11.438131+0000","last_peered":"2024-05-10T04:39:11.438131+0000","last_clean":"2024-05-10T04:39:11.438131+0000","last_became_active":"2024-05-10T04:38:08.347618+0000","last_became_peered":"2024-05-10T04:38:08.347618+0000","last_unstale":"2024-05-10T04:39:11.438131+0000","last_undegraded":"2024-05-10T04:39:11.438131+0000","last_fullsized":"2024-05-10T04:39:11.438131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:09:42.060082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445616+0000","last_change":"2024-05-10T04:38:08.779191+0000","last_active":"2024-05-10T04:39:11.445616+0000","last_peered":"2024-05-10T04:39:11.445616+0000","last_clean":"2024-05-10T04:39:11.445616+0000","last_became_active":"2024-05-10T04:38:08.778829+0000","last_became_peered":"2024-05-10T04:38:08.778829+0000","last_unstale":"2024-05-10T04:39:11.445616+0000","last_undegraded":"2024-05-10T04:39:11.445616+0000","last_fullsized":"2024-05-10T04:39:11.445616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:48:07.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448922+0000","last_change":"2024-05-10T04:38:12.788693+0000","last_active":"2024-05-10T04:39:11.448922+0000","last_peered":"2024-05-10T04:39:11.448922+0000","last_clean":"2024-05-10T04:39:11.448922+0000","last_became_active":"2024-05-10T04:38:12.788156+0000","last_became_peered":"2024-05-10T04:38:12.788156+0000","last_unstale":"2024-05-10T04:39:11.448922+0000","last_undegraded":"2024-05-10T04:39:11.448922+0000","last_fullsized":"2024-05-10T04:39:11.448922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:39:30.119523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:36.487598+0000","last_change":"2024-05-10T04:39:12.455797+0000","last_active":"2024-05-10T04:39:36.487598+0000","last_peered":"2024-05-10T04:39:36.487598+0000","last_clean":"2024-05-10T04:39:36.487598+0000","last_became_active":"2024-05-10T04:39:12.449843+0000","last_became_peered":"2024-05-10T04:39:12.449843+0000","last_unstale":"2024-05-10T04:39:36.487598+0000","last_undegraded":"2024-05-10T04:39:36.487598+0000","last_fullsized":"2024-05-10T04:39:36.487598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:32:17.432455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1515,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446668+0000","last_change":"2024-05-10T04:38:08.780352+0000","last_active":"2024-05-10T04:39:11.446668+0000","last_peered":"2024-05-10T04:39:11.446668+0000","last_clean":"2024-05-10T04:39:11.446668+0000","last_became_active":"2024-05-10T04:38:08.779779+0000","last_became_peered":"2024-05-10T04:38:08.779779+0000","last_unstale":"2024-05-10T04:39:11.446668+0000","last_undegraded":"2024-05-10T04:39:11.446668+0000","last_fullsized":"2024-05-10T04:39:11.446668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:54:31.973896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.439035+0000","last_change":"2024-05-10T04:38:12.805310+0000","last_active":"2024-05-10T04:39:11.439035+0000","last_peered":"2024-05-10T04:39:11.439035+0000","last_clean":"2024-05-10T04:39:11.439035+0000","last_became_active":"2024-05-10T04:38:12.804827+0000","last_became_peered":"2024-05-10T04:38:12.804827+0000","last_unstale":"2024-05-10T04:39:11.439035+0000","last_undegraded":"2024-05-10T04:39:11.439035+0000","last_fullsized":"2024-05-10T04:39:11.439035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:44:01.655788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445754+0000","last_change":"2024-05-10T04:38:08.343813+0000","last_active":"2024-05-10T04:39:11.445754+0000","last_peered":"2024-05-10T04:39:11.445754+0000","last_clean":"2024-05-10T04:39:11.445754+0000","last_became_active":"2024-05-10T04:38:08.343309+0000","last_became_peered":"2024-05-10T04:38:08.343309+0000","last_unstale":"2024-05-10T04:39:11.445754+0000","last_undegraded":"2024-05-10T04:39:11.445754+0000","last_fullsized":"2024-05-10T04:39:11.445754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T15:34:07.233228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.445944+0000","last_change":"2024-05-10T04:38:08.780598+0000","last_active":"2024-05-10T04:39:11.445944+0000","last_peered":"2024-05-10T04:39:11.445944+0000","last_clean":"2024-05-10T04:39:11.445944+0000","last_became_active":"2024-05-10T04:38:08.775367+0000","last_became_peered":"2024-05-10T04:38:08.775367+0000","last_unstale":"2024-05-10T04:39:11.445944+0000","last_undegraded":"2024-05-10T04:39:11.445944+0000","last_fullsized":"2024-05-10T04:39:11.445944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:44:33.826116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444007+0000","last_change":"2024-05-10T04:38:12.795975+0000","last_active":"2024-05-10T04:39:11.444007+0000","last_peered":"2024-05-10T04:39:11.444007+0000","last_clean":"2024-05-10T04:39:11.444007+0000","last_became_active":"2024-05-10T04:38:12.795582+0000","last_became_peered":"2024-05-10T04:38:12.795582+0000","last_unstale":"2024-05-10T04:39:11.444007+0000","last_undegraded":"2024-05-10T04:39:11.444007+0000","last_fullsized":"2024-05-10T04:39:11.444007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:11.767474+0000","last_clean_scrub_stamp":"2024-05-10T04:38:11.767474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:42:30.734833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-10T04:39:12.766085+0000","last_change":"2024-05-10T04:39:12.766085+0000","last_active":"2024-05-10T04:39:12.766085+0000","last_peered":"2024-05-10T04:39:12.766085+0000","last_clean":"2024-05-10T04:39:12.766085+0000","last_became_active":"2024-05-10T04:39:12.761991+0000","last_became_peered":"2024-05-10T04:39:12.761991+0000","last_unstale":"2024-05-10T04:39:12.766085+0000","last_undegraded":"2024-05-10T04:39:12.766085+0000","last_fullsized":"2024-05-10T04:39:12.766085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:07.316344+0000","last_clean_scrub_stamp":"2024-05-10T04:38:07.316344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T11:00:52.188593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446261+0000","last_change":"2024-05-10T04:38:08.780925+0000","last_active":"2024-05-10T04:39:11.446261+0000","last_peered":"2024-05-10T04:39:11.446261+0000","last_clean":"2024-05-10T04:39:11.446261+0000","last_became_active":"2024-05-10T04:38:08.778777+0000","last_became_peered":"2024-05-10T04:38:08.778777+0000","last_unstale":"2024-05-10T04:39:11.446261+0000","last_undegraded":"2024-05-10T04:39:11.446261+0000","last_fullsized":"2024-05-10T04:39:11.446261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:06:57.194258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.442028+0000","last_change":"2024-05-10T04:38:08.780281+0000","last_active":"2024-05-10T04:39:11.442028+0000","last_peered":"2024-05-10T04:39:11.442028+0000","last_clean":"2024-05-10T04:39:11.442028+0000","last_became_active":"2024-05-10T04:38:08.779927+0000","last_became_peered":"2024-05-10T04:38:08.779927+0000","last_unstale":"2024-05-10T04:39:11.442028+0000","last_undegraded":"2024-05-10T04:39:11.442028+0000","last_fullsized":"2024-05-10T04:39:11.442028+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T16:19:33.872931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448040+0000","last_change":"2024-05-10T04:38:08.784916+0000","last_active":"2024-05-10T04:39:11.448040+0000","last_peered":"2024-05-10T04:39:11.448040+0000","last_clean":"2024-05-10T04:39:11.448040+0000","last_became_active":"2024-05-10T04:38:08.784544+0000","last_became_peered":"2024-05-10T04:38:08.784544+0000","last_unstale":"2024-05-10T04:39:11.448040+0000","last_undegraded":"2024-05-10T04:39:11.448040+0000","last_fullsized":"2024-05-10T04:39:11.448040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:49:03.669897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.447644+0000","last_change":"2024-05-10T04:38:08.769149+0000","last_active":"2024-05-10T04:39:11.447644+0000","last_peered":"2024-05-10T04:39:11.447644+0000","last_clean":"2024-05-10T04:39:11.447644+0000","last_became_active":"2024-05-10T04:38:08.768808+0000","last_became_peered":"2024-05-10T04:38:08.768808+0000","last_unstale":"2024-05-10T04:39:11.447644+0000","last_undegraded":"2024-05-10T04:39:11.447644+0000","last_fullsized":"2024-05-10T04:39:11.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:42:46.001895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.438965+0000","last_change":"2024-05-10T04:38:08.784411+0000","last_active":"2024-05-10T04:39:11.438965+0000","last_peered":"2024-05-10T04:39:11.438965+0000","last_clean":"2024-05-10T04:39:11.438965+0000","last_became_active":"2024-05-10T04:38:08.784064+0000","last_became_peered":"2024-05-10T04:38:08.784064+0000","last_unstale":"2024-05-10T04:39:11.438965+0000","last_undegraded":"2024-05-10T04:39:11.438965+0000","last_fullsized":"2024-05-10T04:39:11.438965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T10:32:46.493888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446647+0000","last_change":"2024-05-10T04:38:08.779643+0000","last_active":"2024-05-10T04:39:11.446647+0000","last_peered":"2024-05-10T04:39:11.446647+0000","last_clean":"2024-05-10T04:39:11.446647+0000","last_became_active":"2024-05-10T04:38:08.779291+0000","last_became_peered":"2024-05-10T04:38:08.779291+0000","last_unstale":"2024-05-10T04:39:11.446647+0000","last_undegraded":"2024-05-10T04:39:11.446647+0000","last_fullsized":"2024-05-10T04:39:11.446647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:50:24.535515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443918+0000","last_change":"2024-05-10T04:38:08.789382+0000","last_active":"2024-05-10T04:39:11.443918+0000","last_peered":"2024-05-10T04:39:11.443918+0000","last_clean":"2024-05-10T04:39:11.443918+0000","last_became_active":"2024-05-10T04:38:08.789003+0000","last_became_peered":"2024-05-10T04:38:08.789003+0000","last_unstale":"2024-05-10T04:39:11.443918+0000","last_undegraded":"2024-05-10T04:39:11.443918+0000","last_fullsized":"2024-05-10T04:39:11.443918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:28:03.031319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448507+0000","last_change":"2024-05-10T04:38:08.777290+0000","last_active":"2024-05-10T04:39:11.448507+0000","last_peered":"2024-05-10T04:39:11.448507+0000","last_clean":"2024-05-10T04:39:11.448507+0000","last_became_active":"2024-05-10T04:38:08.776331+0000","last_became_peered":"2024-05-10T04:38:08.776331+0000","last_unstale":"2024-05-10T04:39:11.448507+0000","last_undegraded":"2024-05-10T04:39:11.448507+0000","last_fullsized":"2024-05-10T04:39:11.448507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:54:51.074991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449405+0000","last_change":"2024-05-10T04:38:08.780937+0000","last_active":"2024-05-10T04:39:11.449405+0000","last_peered":"2024-05-10T04:39:11.449405+0000","last_clean":"2024-05-10T04:39:11.449405+0000","last_became_active":"2024-05-10T04:38:08.780620+0000","last_became_peered":"2024-05-10T04:38:08.780620+0000","last_unstale":"2024-05-10T04:39:11.449405+0000","last_undegraded":"2024-05-10T04:39:11.449405+0000","last_fullsized":"2024-05-10T04:39:11.449405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T08:50:27.697571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.444313+0000","last_change":"2024-05-10T04:38:08.773314+0000","last_active":"2024-05-10T04:39:11.444313+0000","last_peered":"2024-05-10T04:39:11.444313+0000","last_clean":"2024-05-10T04:39:11.444313+0000","last_became_active":"2024-05-10T04:38:08.772973+0000","last_became_peered":"2024-05-10T04:38:08.772973+0000","last_unstale":"2024-05-10T04:39:11.444313+0000","last_undegraded":"2024-05-10T04:39:11.444313+0000","last_fullsized":"2024-05-10T04:39:11.444313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:35:53.661899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.448852+0000","last_change":"2024-05-10T04:38:08.775632+0000","last_active":"2024-05-10T04:39:11.448852+0000","last_peered":"2024-05-10T04:39:11.448852+0000","last_clean":"2024-05-10T04:39:11.448852+0000","last_became_active":"2024-05-10T04:38:08.775308+0000","last_became_peered":"2024-05-10T04:38:08.775308+0000","last_unstale":"2024-05-10T04:39:11.448852+0000","last_undegraded":"2024-05-10T04:39:11.448852+0000","last_fullsized":"2024-05-10T04:39:11.448852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T14:12:58.758863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.449579+0000","last_change":"2024-05-10T04:38:08.787354+0000","last_active":"2024-05-10T04:39:11.449579+0000","last_peered":"2024-05-10T04:39:11.449579+0000","last_clean":"2024-05-10T04:39:11.449579+0000","last_became_active":"2024-05-10T04:38:08.786645+0000","last_became_peered":"2024-05-10T04:38:08.786645+0000","last_unstale":"2024-05-10T04:39:11.449579+0000","last_undegraded":"2024-05-10T04:39:11.449579+0000","last_fullsized":"2024-05-10T04:39:11.449579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T13:46:35.060271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.446479+0000","last_change":"2024-05-10T04:38:08.769233+0000","last_active":"2024-05-10T04:39:11.446479+0000","last_peered":"2024-05-10T04:39:11.446479+0000","last_clean":"2024-05-10T04:39:11.446479+0000","last_became_active":"2024-05-10T04:38:08.768936+0000","last_became_peered":"2024-05-10T04:38:08.768936+0000","last_unstale":"2024-05-10T04:39:11.446479+0000","last_undegraded":"2024-05-10T04:39:11.446479+0000","last_fullsized":"2024-05-10T04:39:11.446479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T12:09:08.000090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-10T04:39:11.443044+0000","last_change":"2024-05-10T04:38:08.788189+0000","last_active":"2024-05-10T04:39:11.443044+0000","last_peered":"2024-05-10T04:39:11.443044+0000","last_clean":"2024-05-10T04:39:11.443044+0000","last_became_active":"2024-05-10T04:38:08.787806+0000","last_became_peered":"2024-05-10T04:38:08.787806+0000","last_unstale":"2024-05-10T04:39:11.443044+0000","last_undegraded":"2024-05-10T04:39:11.443044+0000","last_fullsized":"2024-05-10T04:39:11.443044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-10T04:38:08.321986+0000","last_clean_scrub_stamp":"2024-05-10T04:38:08.321986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T07:00:51.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280050,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74877,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":665,"num_read_kb":436,"num_write":240,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":224631,"data_compressed":4791,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481821,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":974848,"data_stored":485047,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.64600000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78800000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75700000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":0.874,"15min":0.874},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.372,"15min":0.372},"max":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84299999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.72399999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.92200000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.68500000000000005,"15min":0.69599999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89400000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.71099999999999997,"15min":0.72699999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":1.02,"15min":1.02},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78600000000000003,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0680000000000001,"5min":1.391,"15min":1.391},"last":1.0680000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78600000000000003,"15min":0.79000000000000004},"min":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.98399999999999999,"5min":1.046,"15min":1.046},"last":0.875},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83799999999999997,"15min":0.84999999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96799999999999997,"5min":1.829,"15min":1.829},"last":0.94599999999999995}]},{"osd":8,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.107}]},{"osd":9,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.86599999999999999,"15min":0.876},"min":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1579999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.82199999999999995,"15min":0.82699999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.012,"5min":1.117,"15min":1.117},"last":0.92400000000000004}]},{"osd":10,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.03,"5min":1.601,"15min":1.601},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91300000000000003,"15min":0.92400000000000004},"min":{"1min":0.70999999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.1319999999999999,"5min":1.774,"15min":1.774},"last":1.1319999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":1139281,"data_compressed":54829,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.442}]},{"osd":1,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.497,"15min":0.496},"min":{"1min":0.317,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79000000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.34300000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63700000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81999999999999995,"5min":1.143,"15min":1.143},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.52900000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.52600000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.497,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.625}]},{"osd":4,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":0.872,"15min":0.872},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95599999999999996,"5min":1.252,"15min":1.252},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78000000000000003,"15min":0.77800000000000002},"min":{"1min":0.60499999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.52200000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":8,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80600000000000005,"15min":0.80400000000000005},"min":{"1min":0.61699999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":0.85199999999999998},"min":{"1min":0.56899999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":1.296,"15min":1.296},"last":0.82099999999999995}]},{"osd":9,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.624,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.139,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90700000000000003,"5min":1.2,"15min":1.2},"last":0.66800000000000004}]},{"osd":11,"last update":"Fri May 10 04:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83699999999999997,"15min":0.83099999999999996},"min":{"1min":0.68799999999999994,"5min":0.621,"15min":0.621},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.84599999999999997,"15min":0.83799999999999997},"min":{"1min":0.749,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.86799999999999999}]}]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":488616,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.44600000000000001,"15min":0.45700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.55300000000000005,"5min":1.034,"15min":1.034},"last":0.442},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46700000000000003,"15min":0.46999999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.64900000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.498}]},{"osd":1,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.625,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90400000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.56599999999999995,"15min":0.58699999999999997},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.65900000000000003,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.59699999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.67300000000000004,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80500000000000005,"5min":1.101,"15min":1.101},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":1.411,"15min":1.411},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri May 10 04:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.79100000000000004,"15min":0.80900000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.51200000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.29,"15min":1.29},"last":0.83899999999999997}]},{"osd":8,"last update":"Fri May 10 04:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.77600000000000002,"15min":0.79700000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93999999999999995,"5min":1.53,"15min":1.53},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.78900000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91500000000000004,"5min":1.502,"15min":1.502},"last":0.748}]},{"osd":10,"last update":"Fri May 10 04:39:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72899999999999998,"15min":0.73799999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98199999999999998,"5min":1.016,"15min":1.016},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.97299999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78600000000000003}]},{"osd":11,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89800000000000002}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":978944,"data_stored":484786,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.46100000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.439}]},{"osd":2,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.497,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65700000000000003}]},{"osd":3,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.66900000000000004,"15min":0.64400000000000002},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.41699999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88600000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.59399999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.625}]},{"osd":5,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.748},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71599999999999997,"15min":0.70199999999999996},"min":{"1min":0.59099999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.67400000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri May 10 04:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.76300000000000001,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.874},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.63900000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77800000000000002}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.77400000000000002,"15min":0.76300000000000001},"min":{"1min":0.61799999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86299999999999999,"15min":0.85999999999999999},"min":{"1min":0.70899999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.089,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":0.88900000000000001}]},{"osd":10,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.82399999999999995,"15min":0.81599999999999995},"min":{"1min":0.73099999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.83299999999999996,"15min":0.81899999999999995},"min":{"1min":0.754,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.85799999999999998}]},{"osd":11,"last update":"Fri May 10 04:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.81999999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":1040384,"data_stored":1081174,"data_compressed":53232,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.47499999999999998,"15min":0.45700000000000002},"min":{"1min":0.33700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56399999999999995,"5min":3.532,"15min":3.532},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.48999999999999999},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.59499999999999997,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80400000000000005,"5min":3.3759999999999999,"15min":3.3759999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.51100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.77400000000000002,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.58599999999999997},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83399999999999996,"5min":3.4209999999999998,"15min":3.4209999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63800000000000001,"15min":0.60299999999999998},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78100000000000003,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.76900000000000002,"5min":1.927,"15min":1.927},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.60499999999999998},"min":{"1min":0.48299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.755,"5min":3.6179999999999999,"15min":3.6179999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86499999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.75},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.59299999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94799999999999995,"5min":3.4860000000000002,"15min":3.4860000000000002},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri May 10 04:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70599999999999996,"15min":0.68799999999999994},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.81000000000000005,"5min":3.996,"15min":3.996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.56399999999999995}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69199999999999995,"15min":0.68400000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.65300000000000002}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":2.028,"15min":2.028},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89400000000000002,"15min":0.91400000000000003},"min":{"1min":0.53700000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95299999999999996,"5min":3.5630000000000002,"15min":3.5630000000000002},"last":0.68400000000000005}]},{"osd":10,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.88500000000000001,"15min":0.90300000000000002},"min":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91100000000000003,"5min":4.2670000000000003,"15min":4.2670000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.89500000000000002,"15min":0.91100000000000003},"min":{"1min":0.64700000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.95299999999999996,"5min":3.653,"15min":3.653},"last":0.76700000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79800000000000004}]}]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":479976,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.46200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.51900000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57599999999999996,"15min":0.56200000000000006},"min":{"1min":0.45300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67700000000000005,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.44400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.69599999999999995,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71799999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.47499999999999998,"15min":0.48199999999999998},"min":{"1min":0.309,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.5,"15min":0.51900000000000002},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.53700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.44900000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.64200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.625,"15min":0.629},"min":{"1min":0.497,"5min":0.437,"15min":0.437},"max":{"1min":0.746,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.63800000000000001,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.74099999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64100000000000001,"15min":0.65000000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri May 10 04:39:33 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.73399999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.68000000000000005,"15min":0.69499999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.76000000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.65200000000000002}]},{"osd":8,"last update":"Fri May 10 04:38:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.70099999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72499999999999998,"15min":0.73399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.45500000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.496}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72499999999999998,"15min":0.73499999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.80000000000000004,"5min":1.083,"15min":1.083},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.80300000000000005,"5min":1.105,"15min":1.105},"last":0.64500000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":1036288,"data_stored":548946,"data_compressed":19210,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53500000000000003,"15min":0.5},"min":{"1min":0.40300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri May 10 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.57399999999999995,"15min":0.54200000000000004},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92500000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.46700000000000003,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72399999999999998}]},{"osd":3,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.57599999999999996},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.57299999999999995},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri May 10 04:39:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.497,"15min":0.45000000000000001},"min":{"1min":0.45200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.878,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.52000000000000002,"15min":0.46000000000000002},"min":{"1min":0.39100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.82899999999999996,"5min":1.083,"15min":1.083},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri May 10 04:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.58499999999999996,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":1.123,"15min":1.123},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.69199999999999995,"15min":0.64900000000000002},"min":{"1min":0.55100000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1100000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.502,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.623},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.55500000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.89500000000000002}]},{"osd":7,"last update":"Fri May 10 04:39:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.374,"15min":0.374},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.67600000000000005,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99199999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.65400000000000003}]},{"osd":8,"last update":"Fri May 10 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.52800000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.95999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.626,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1040000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.81599999999999995}]},{"osd":9,"last update":"Fri May 10 04:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.58399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78400000000000003,"15min":0.77600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0049999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri May 10 04:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77700000000000002,"15min":0.77000000000000002},"min":{"1min":0.56399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.86399999999999999,"15min":0.85199999999999998},"min":{"1min":0.72499999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.097,"5min":1.163,"15min":1.163},"last":0.58499999999999996}]},{"osd":11,"last update":"Fri May 10 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":481825,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.48499999999999999},"min":{"1min":0.371,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.673,"5min":2.673,"15min":2.673},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.55800000000000005,"15min":0.48699999999999999},"min":{"1min":0.39700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.307}]},{"osd":2,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.55600000000000005},"min":{"1min":0.41599999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.41699999999999998}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61499999999999999,"15min":0.59299999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.33000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.38600000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.47599999999999998}]},{"osd":8,"last update":"Fri May 10 04:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.751}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69599999999999995,"15min":0.67300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":4.7190000000000003,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.628}]},{"osd":10,"last update":"Fri May 10 04:39:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.70599999999999996,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.753,"15min":0.72599999999999998},"min":{"1min":0.60599999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":0.57199999999999995}]},{"osd":11,"last update":"Fri May 10 04:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.752,"15min":0.72399999999999998},"min":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.72599999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":547996,"data_compressed":19210,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52300000000000002,"15min":0.48199999999999998},"min":{"1min":0.45500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76100000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.503,"15min":0.46400000000000002},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.73199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri May 10 04:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.53100000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.60999999999999999,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.94699999999999995}]},{"osd":3,"last update":"Fri May 10 04:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.84999999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.59999999999999998,"15min":0.55700000000000005},"min":{"1min":0.54400000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.48699999999999999,"15min":0.46800000000000003},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.44800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.45800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89900000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.022}]},{"osd":7,"last update":"Fri May 10 04:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":1.095}]},{"osd":8,"last update":"Fri May 10 04:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.58399999999999996,"15min":0.55800000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.91300000000000003}]},{"osd":9,"last update":"Fri May 10 04:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.60599999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0249999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.72199999999999998,"15min":0.70399999999999996},"min":{"1min":0.57999999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri May 10 04:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.70199999999999996,"5min":0.438,"15min":0.438},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.66900000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Fri May 10 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.999}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":479732,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.372,"15min":0.34399999999999997},"min":{"1min":0.29999999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.36399999999999999,"15min":0.35199999999999998},"min":{"1min":0.29199999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54100000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.83599999999999997,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.52400000000000002},"min":{"1min":0.373,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54700000000000004,"15min":0.52600000000000002},"min":{"1min":0.46899999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86499999999999999,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri May 10 04:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81599999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.46000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82199999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri May 10 04:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.624,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.52600000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84399999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.58099999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70099999999999996}]},{"osd":8,"last update":"Fri May 10 04:37:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.76500000000000001}]},{"osd":9,"last update":"Fri May 10 04:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67600000000000005,"15min":0.68799999999999994},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91600000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68400000000000005}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77500000000000002}]},{"osd":11,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.72399999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":485879,"data_compressed":17613,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":1.143,"15min":1.143},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38100000000000001,"15min":0.36099999999999999},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.38200000000000001,"15min":0.36199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.627,"5min":0.748,"15min":0.748},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.623,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91100000000000003,"5min":1.03,"15min":1.03},"last":0.84399999999999997}]},{"osd":3,"last update":"Fri May 10 04:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.85999999999999999}]},{"osd":5,"last update":"Fri May 10 04:38:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88300000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90700000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.50700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.069,"15min":1.069},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.57199999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.871,"5min":1.079,"15min":1.079},"last":1.133}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.63300000000000001,"15min":0.61199999999999999},"min":{"1min":0.497,"5min":0.378,"15min":0.378},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.61199999999999999,"15min":0.59799999999999998},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0640000000000001}]},{"osd":8,"last update":"Fri May 10 04:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69699999999999995,"15min":0.69399999999999995},"min":{"1min":0.52800000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88500000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.73099999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.99099999999999999}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.80300000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.032},{"interface":"front","average":{"1min":0.753,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":1.1859999999999999}]},{"osd":10,"last update":"Fri May 10 04:39:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.71799999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0069999999999999,"5min":1.012,"15min":1.012},"last":1.165},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98099999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.1539999999999999}]},{"osd":11,"last update":"Fri May 10 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.95299999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":1048576,"data_stored":1076538,"data_compressed":53232,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54300000000000004,"15min":0.53100000000000003},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41799999999999998,"15min":0.40699999999999997},"min":{"1min":0.311,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68999999999999995,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85499999999999998,"5min":1.008,"15min":1.008},"last":0.63}]},{"osd":4,"last update":"Fri May 10 04:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81599999999999995,"5min":1.089,"15min":1.089},"last":0.749},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84499999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri May 10 04:39:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.503},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.51500000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":1.002,"15min":1.002},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri May 10 04:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.56899999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90400000000000003,"5min":1.054,"15min":1.054},"last":0.877},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64300000000000002,"15min":0.63100000000000001},"min":{"1min":0.60399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.73499999999999999}]},{"osd":8,"last update":"Fri May 10 04:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri May 10 04:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri May 10 04:39:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92000000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.84499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33985,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35619,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69644,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68694,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69942,"data_compressed":1597,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-10T04:39:44.153 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:44.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 -- 172.21.15.150:0/2973410213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 msgr2=0x7f53f0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 --2- 172.21.15.150:0/2973410213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0102000 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009990 tx=0x7f53e0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 -- 172.21.15.150:0/2973410213 shutdown_connections 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 --2- 172.21.15.150:0/2973410213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f0102590 0x7f53f0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 --2- 172.21.15.150:0/2973410213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f0100ad0 0x7f53f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 --2- 172.21.15.150:0/2973410213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0102000 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 -- 172.21.15.150:0/2973410213 >> 172.21.15.150:0/2973410213 conn(0x7f53f00fc0e0 msgr2=0x7f53f00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 -- 172.21.15.150:0/2973410213 shutdown_connections 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.323+0000 7f53f548a700 1 -- 172.21.15.150:0/2973410213 wait complete. 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 Processor -- start 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 -- start start 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f0100ad0 0x7f53f0197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f0102590 0x7f53f019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f019eb00 con 0x7f53f0101bf0 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f019ec70 con 0x7f53f0100ad0 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.324+0000 7f53f548a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f019ee30 con 0x7f53f0102590 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48026/0 (socket says 172.21.15.150:48026) 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 -- 172.21.15.150:0/1599985732 learned_addr learned my addr 172.21.15.150:0/1599985732 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53eeffd700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f0100ad0 0x7f53f0197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ef7fe700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f0102590 0x7f53f019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 -- 172.21.15.150:0/1599985732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f0102590 msgr2=0x7f53f019e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f0102590 0x7f53f019e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 -- 172.21.15.150:0/1599985732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f0100ad0 msgr2=0x7f53f0197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f0100ad0 0x7f53f0197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 -- 172.21.15.150:0/1599985732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00095f0 con 0x7f53f0101bf0 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53ee7fc700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0197560 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005910 tx=0x7f53e0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:44.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53d7fff700 1 -- 172.21.15.150:0/1599985732 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e001f070 con 0x7f53f0101bf0 2024-05-10T04:39:44.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f0073440 con 0x7f53f0101bf0 2024-05-10T04:39:44.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53d7fff700 1 -- 172.21.15.150:0/1599985732 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e001acb0 con 0x7f53f0101bf0 2024-05-10T04:39:44.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53d7fff700 1 -- 172.21.15.150:0/1599985732 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000f460 con 0x7f53f0101bf0 2024-05-10T04:39:44.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.325+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f0073940 con 0x7f53f0101bf0 2024-05-10T04:39:44.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.326+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f0068730 con 0x7f53f0101bf0 2024-05-10T04:39:44.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.328+0000 7f53d7fff700 1 -- 172.21.15.150:0/1599985732 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f53e0024470 con 0x7f53f0101bf0 2024-05-10T04:39:44.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.329+0000 7f53d7fff700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f53d80a11b0 0x7f53d80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.329+0000 7f53eeffd700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f53d80a11b0 0x7f53d80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.329+0000 7f53d7fff700 1 -- 172.21.15.150:0/1599985732 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53e00d0660 con 0x7f53f0101bf0 2024-05-10T04:39:44.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.329+0000 7f53eeffd700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f53d80a11b0 0x7f53d80a36c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53f0101af0 tx=0x7f53e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:44.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.332+0000 7f53d7fff700 1 -- 172.21.15.150:0/1599985732 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f53e005fd60 con 0x7f53f0101bf0 2024-05-10T04:39:44.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.536+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53f01076c0 con 0x7f53f0101bf0 2024-05-10T04:39:44.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.538+0000 7f53d7fff700 1 -- 172.21.15.150:0/1599985732 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f53e00999b0 con 0x7f53f0101bf0 2024-05-10T04:39:44.539 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:44.539 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:44.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f53d80a11b0 msgr2=0x7f53d80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f53d80a11b0 0x7f53d80a36c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53f0101af0 tx=0x7f53e4009260 comp rx=0 tx=0).stop 2024-05-10T04:39:44.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 msgr2=0x7f53f0197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0197560 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f53e0005910 tx=0x7f53e0009580 comp rx=0 tx=0).stop 2024-05-10T04:39:44.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 shutdown_connections 2024-05-10T04:39:44.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f53d80a11b0 0x7f53d80a36c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53f0102590 0x7f53f019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f0100ad0 0x7f53f0197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 --2- 172.21.15.150:0/1599985732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53f0101bf0 0x7f53f0197560 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 >> 172.21.15.150:0/1599985732 conn(0x7f53f00fc0e0 msgr2=0x7f53f0104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:44.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 shutdown_connections 2024-05-10T04:39:44.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.541+0000 7f53f548a700 1 -- 172.21.15.150:0/1599985732 wait complete. 2024-05-10T04:39:44.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:39:44.555 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:39:44.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 -- 172.21.15.150:0/2273378881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8102630 msgr2=0x7fbdf8108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/2273378881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8102630 0x7fbdf8108ee0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000b0a0 tx=0x7fbdf000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 -- 172.21.15.150:0/2273378881 shutdown_connections 2024-05-10T04:39:44.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/2273378881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8102630 0x7fbdf8108ee0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/2273378881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf8100b70 0x7fbdf8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.694 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/2273378881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8101c90 0x7fbdf81020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 -- 172.21.15.150:0/2273378881 >> 172.21.15.150:0/2273378881 conn(0x7fbdf80fc180 msgr2=0x7fbdf80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 -- 172.21.15.150:0/2273378881 shutdown_connections 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 -- 172.21.15.150:0/2273378881 wait complete. 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 Processor -- start 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.692+0000 7fbdffbc9700 1 -- start start 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdffbc9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8100b70 0x7fbdf806a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdffbc9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf8101c90 0x7fbdf819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdffbc9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8102630 0x7fbdf819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdffbc9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf819fa50 con 0x7fbdf8100b70 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdffbc9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf819fbc0 con 0x7fbdf8102630 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdffbc9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf819fd80 con 0x7fbdf8101c90 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8100b70 0x7fbdf806a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8100b70 0x7fbdf806a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48050/0 (socket says 172.21.15.150:48050) 2024-05-10T04:39:44.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 -- 172.21.15.150:0/3302251234 learned_addr learned my addr 172.21.15.150:0/3302251234 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfe166700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8102630 0x7fbdf819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd164700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf8101c90 0x7fbdf819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 -- 172.21.15.150:0/3302251234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf8101c90 msgr2=0x7fbdf819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf8101c90 0x7fbdf819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 -- 172.21.15.150:0/3302251234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8102630 msgr2=0x7fbdf819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8102630 0x7fbdf819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 -- 172.21.15.150:0/3302251234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf0009d00 con 0x7fbdf8100b70 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdfd965700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8100b70 0x7fbdf806a230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400d950 tx=0x7fbdf400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdeeffd700 1 -- 172.21.15.150:0/3302251234 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4004220 con 0x7fbdf8100b70 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdeeffd700 1 -- 172.21.15.150:0/3302251234 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdf4004380 con 0x7fbdf8100b70 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf81a0060 con 0x7fbdf8100b70 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.693+0000 7fbdeeffd700 1 -- 172.21.15.150:0/3302251234 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4003be0 con 0x7fbdf8100b70 2024-05-10T04:39:44.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.694+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf81a0590 con 0x7fbdf8100b70 2024-05-10T04:39:44.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.694+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf8107d80 con 0x7fbdf8100b70 2024-05-10T04:39:44.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.696+0000 7fbdeeffd700 1 -- 172.21.15.150:0/3302251234 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fbdf4003d40 con 0x7fbdf8100b70 2024-05-10T04:39:44.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.697+0000 7fbdeeffd700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbde40a12e0 0x7fbde40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:44.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.697+0000 7fbdfd164700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbde40a12e0 0x7fbde40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:44.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.697+0000 7fbdeeffd700 1 -- 172.21.15.150:0/3302251234 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbdf40ccf70 con 0x7fbdf8100b70 2024-05-10T04:39:44.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.697+0000 7fbdfd164700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbde40a12e0 0x7fbde40a37f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8069bd0 tx=0x7fbde800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:44.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.701+0000 7fbdeeffd700 1 -- 172.21.15.150:0/3302251234 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbdf40962b0 con 0x7fbdf8100b70 2024-05-10T04:39:44.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:44 smithi161 ceph-mon[46912]: from='client.15645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:44.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:44 smithi161 ceph-mon[46912]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.902+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdf810c550 con 0x7fbdf8100b70 2024-05-10T04:39:44.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.903+0000 7fbdeeffd700 1 -- 172.21.15.150:0/3302251234 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdf4017020 con 0x7fbdf8100b70 2024-05-10T04:39:44.903 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:39:44.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:44 smithi181 ceph-mon[47314]: from='client.15645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:44.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:44 smithi181 ceph-mon[47314]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbde40a12e0 msgr2=0x7fbde40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbde40a12e0 0x7fbde40a37f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8069bd0 tx=0x7fbde800a380 comp rx=0 tx=0).stop 2024-05-10T04:39:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8100b70 msgr2=0x7fbdf806a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:44.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8100b70 0x7fbdf806a230 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400d950 tx=0x7fbdf400dc60 comp rx=0 tx=0).stop 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 shutdown_connections 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdf8102630 0x7fbdf819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbdf8100b70 0x7fbdf806a230 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbde40a12e0 0x7fbde40a37f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 --2- 172.21.15.150:0/3302251234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf8101c90 0x7fbdf819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 >> 172.21.15.150:0/3302251234 conn(0x7fbdf80fc180 msgr2=0x7fbdf8104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 shutdown_connections 2024-05-10T04:39:44.906 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:44.905+0000 7fbdffbc9700 1 -- 172.21.15.150:0/3302251234 wait complete. 2024-05-10T04:39:44.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:44 smithi150 ceph-mon[42404]: from='client.15645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:44.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:44 smithi150 ceph-mon[42404]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-10T04:39:44.918 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:39:45.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.054+0000 7f6537d1a700 1 -- 172.21.15.150:0/555475133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 msgr2=0x7f6530102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.054+0000 7f6537d1a700 1 --2- 172.21.15.150:0/555475133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f6530102020 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f652c009990 tx=0x7f652c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 -- 172.21.15.150:0/555475133 shutdown_connections 2024-05-10T04:39:45.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 --2- 172.21.15.150:0/555475133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65301025b0 0x7f6530108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 --2- 172.21.15.150:0/555475133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6530100af0 0x7f6530100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 --2- 172.21.15.150:0/555475133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f6530102020 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 -- 172.21.15.150:0/555475133 >> 172.21.15.150:0/555475133 conn(0x7f65300fc0e0 msgr2=0x7f65300fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 -- 172.21.15.150:0/555475133 shutdown_connections 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 -- 172.21.15.150:0/555475133 wait complete. 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 Processor -- start 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 -- start start 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6530100af0 0x7f6530073f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f653019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65301025b0 0x7f653019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.055+0000 7f6537d1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653019f960 con 0x7f6530101c10 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6537d1a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653019fad0 con 0x7f6530100af0 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6537d1a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653019fc90 con 0x7f65301025b0 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f653019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f653019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48062/0 (socket says 172.21.15.150:48062) 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 -- 172.21.15.150:0/709922222 learned_addr learned my addr 172.21.15.150:0/709922222 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65362b7700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65301025b0 0x7f653019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6535ab6700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6530100af0 0x7f6530073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 -- 172.21.15.150:0/709922222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65301025b0 msgr2=0x7f653019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65301025b0 0x7f653019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 -- 172.21.15.150:0/709922222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6530100af0 msgr2=0x7f6530073f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6530100af0 0x7f6530073f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 -- 172.21.15.150:0/709922222 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c0095f0 con 0x7f6530101c10 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f65352b5700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f653019cbf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f652c005b40 tx=0x7f652c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6526ffd700 1 -- 172.21.15.150:0/709922222 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c01d070 con 0x7f6530101c10 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6526ffd700 1 -- 172.21.15.150:0/709922222 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f652c00fb80 con 0x7f6530101c10 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653019ff10 con 0x7f6530101c10 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6526ffd700 1 -- 172.21.15.150:0/709922222 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c0173f0 con 0x7f6530101c10 2024-05-10T04:39:45.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.056+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65301a02d0 con 0x7f6530101c10 2024-05-10T04:39:45.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.057+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6530107d00 con 0x7f6530101c10 2024-05-10T04:39:45.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.058+0000 7f6526ffd700 1 -- 172.21.15.150:0/709922222 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f652c00fcf0 con 0x7f6530101c10 2024-05-10T04:39:45.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.059+0000 7f6526ffd700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f651c0a12e0 0x7f651c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.059+0000 7f6526ffd700 1 -- 172.21.15.150:0/709922222 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f652c0cdf20 con 0x7f6530101c10 2024-05-10T04:39:45.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.059+0000 7f6535ab6700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f651c0a12e0 0x7f651c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.060+0000 7f6535ab6700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f651c0a12e0 0x7f651c0a37f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6520005950 tx=0x7f65200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:45.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.064+0000 7f6526ffd700 1 -- 172.21.15.150:0/709922222 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f652c05dd60 con 0x7f6530101c10 2024-05-10T04:39:45.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.258+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f653010c2f0 con 0x7f6530101c10 2024-05-10T04:39:45.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.259+0000 7f6526ffd700 1 -- 172.21.15.150:0/709922222 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f652c097270 con 0x7f6530101c10 2024-05-10T04:39:45.259 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:45.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f651c0a12e0 msgr2=0x7f651c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f651c0a12e0 0x7f651c0a37f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6520005950 tx=0x7f65200058e0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 msgr2=0x7f653019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f653019cbf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f652c005b40 tx=0x7f652c00f690 comp rx=0 tx=0).stop 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 shutdown_connections 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f651c0a12e0 0x7f651c0a37f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65301025b0 0x7f653019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6530100af0 0x7f6530073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 --2- 172.21.15.150:0/709922222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6530101c10 0x7f653019cbf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 >> 172.21.15.150:0/709922222 conn(0x7f65300fc0e0 msgr2=0x7f6530104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 shutdown_connections 2024-05-10T04:39:45.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.261+0000 7f6537d1a700 1 -- 172.21.15.150:0/709922222 wait complete. 2024-05-10T04:39:45.274 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:39:45.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.410+0000 7f995752c700 1 -- 172.21.15.150:0/1794424854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950102030 msgr2=0x7f9950102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.410+0000 7f995752c700 1 --2- 172.21.15.150:0/1794424854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950102030 0x7f9950102460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9940009990 tx=0x7f9940009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.410+0000 7f995752c700 1 -- 172.21.15.150:0/1794424854 shutdown_connections 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.410+0000 7f995752c700 1 --2- 172.21.15.150:0/1794424854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9950101680 0x7f9950108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.410+0000 7f995752c700 1 --2- 172.21.15.150:0/1794424854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950102030 0x7f9950102460 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.410+0000 7f995752c700 1 --2- 172.21.15.150:0/1794424854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9950100d30 0x7f9950101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.410+0000 7f995752c700 1 -- 172.21.15.150:0/1794424854 >> 172.21.15.150:0/1794424854 conn(0x7f99500fb8f0 msgr2=0x7f99500fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 -- 172.21.15.150:0/1794424854 shutdown_connections 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 -- 172.21.15.150:0/1794424854 wait complete. 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 Processor -- start 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 -- start start 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9950100d30 0x7f9950196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950101680 0x7f9950197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9950102030 0x7f995019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019e930 con 0x7f9950101680 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019eaa0 con 0x7f9950102030 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f995752c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019ec60 con 0x7f9950100d30 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.411+0000 7f9954ac7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950101680 0x7f9950197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950101680 0x7f9950197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48082/0 (socket says 172.21.15.150:48082) 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 -- 172.21.15.150:0/3591039781 learned_addr learned my addr 172.21.15.150:0/3591039781 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f99552c8700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9950100d30 0x7f9950196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9955ac9700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9950102030 0x7f995019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 -- 172.21.15.150:0/3591039781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9950100d30 msgr2=0x7f9950196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9950100d30 0x7f9950196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 -- 172.21.15.150:0/3591039781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9950102030 msgr2=0x7f995019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9950102030 0x7f995019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 -- 172.21.15.150:0/3591039781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400095f0 con 0x7f9950101680 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f9954ac7700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950101680 0x7f9950197390 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9c0 tx=0x7f994c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f99467fc700 1 -- 172.21.15.150:0/3591039781 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0097b0 con 0x7f9950101680 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995019ef40 con 0x7f9950101680 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f99467fc700 1 -- 172.21.15.150:0/3591039781 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f994c004cf0 con 0x7f9950101680 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f99467fc700 1 -- 172.21.15.150:0/3591039781 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c005700 con 0x7f9950101680 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.412+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995019f360 con 0x7f9950101680 2024-05-10T04:39:45.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.413+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9950107e30 con 0x7f9950101680 2024-05-10T04:39:45.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.415+0000 7f99467fc700 1 -- 172.21.15.150:0/3591039781 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f994c00cbd0 con 0x7f9950101680 2024-05-10T04:39:45.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.415+0000 7f99467fc700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f993c0a1260 0x7f993c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.415+0000 7f99467fc700 1 -- 172.21.15.150:0/3591039781 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f994c014070 con 0x7f9950101680 2024-05-10T04:39:45.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.415+0000 7f99552c8700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f993c0a1260 0x7f993c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.416+0000 7f99552c8700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f993c0a1260 0x7f993c0a3770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9940006010 tx=0x7f994001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:45.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.419+0000 7f99467fc700 1 -- 172.21.15.150:0/3591039781 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f994c09a110 con 0x7f9950101680 2024-05-10T04:39:45.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.615+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f995010c310 con 0x7f9950101680 2024-05-10T04:39:45.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.616+0000 7f99467fc700 1 -- 172.21.15.150:0/3591039781 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f994c099ca0 con 0x7f9950101680 2024-05-10T04:39:45.616 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:39:45.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f993c0a1260 msgr2=0x7f993c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f993c0a1260 0x7f993c0a3770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9940006010 tx=0x7f994001a040 comp rx=0 tx=0).stop 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950101680 msgr2=0x7f9950197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950101680 0x7f9950197390 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9c0 tx=0x7f994c00ed80 comp rx=0 tx=0).stop 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 shutdown_connections 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9950102030 0x7f995019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f993c0a1260 0x7f993c0a3770 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9950100d30 0x7f9950196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 --2- 172.21.15.150:0/3591039781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9950101680 0x7f9950197390 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 >> 172.21.15.150:0/3591039781 conn(0x7f99500fb8f0 msgr2=0x7f9950104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 shutdown_connections 2024-05-10T04:39:45.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.618+0000 7f995752c700 1 -- 172.21.15.150:0/3591039781 wait complete. 2024-05-10T04:39:45.630 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:45 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1599985732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:45.630 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:45 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3302251234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:45.630 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:45 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/709922222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:45.631 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:45 smithi150 ceph-mon[42404]: pgmap v275: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-10T04:39:45.632 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:39:45.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 -- 172.21.15.150:0/3893347819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 msgr2=0x7f33b41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 --2- 172.21.15.150:0/3893347819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b41006d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f33a400aaa0 tx=0x7f33a400adb0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 -- 172.21.15.150:0/3893347819 shutdown_connections 2024-05-10T04:39:45.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 --2- 172.21.15.150:0/3893347819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33b4101d80 0x7f33b4108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.770 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 --2- 172.21.15.150:0/3893347819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b41006d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 --2- 172.21.15.150:0/3893347819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b41013e0 0x7f33b41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 -- 172.21.15.150:0/3893347819 >> 172.21.15.150:0/3893347819 conn(0x7f33b40fb8f0 msgr2=0x7f33b40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.767+0000 7f33ba771700 1 -- 172.21.15.150:0/3893347819 shutdown_connections 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 -- 172.21.15.150:0/3893347819 wait complete. 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 Processor -- start 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 -- start start 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b41013e0 0x7f33b4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33b4101d80 0x7f33b419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b419fa30 con 0x7f33b41002c0 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b419fba0 con 0x7f33b4101d80 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.768+0000 7f33ba771700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b419fd60 con 0x7f33b41013e0 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b4197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48108/0 (socket says 172.21.15.150:48108) 2024-05-10T04:39:45.771 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 -- 172.21.15.150:0/2465273296 learned_addr learned my addr 172.21.15.150:0/2465273296 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b37fe700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b41013e0 0x7f33b4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b8d0e700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33b4101d80 0x7f33b419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 -- 172.21.15.150:0/2465273296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b41013e0 msgr2=0x7f33b4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b41013e0 0x7f33b4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 -- 172.21.15.150:0/2465273296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33b4101d80 msgr2=0x7f33b419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33b4101d80 0x7f33b419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 -- 172.21.15.150:0/2465273296 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a400a700 con 0x7f33b41002c0 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b37fe700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b41013e0 0x7f33b4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:45.772 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b3fff700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b4197f50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006010 tx=0x7f33a400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:45.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b17fa700 1 -- 172.21.15.150:0/2465273296 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4005350 con 0x7f33b41002c0 2024-05-10T04:39:45.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b17fa700 1 -- 172.21.15.150:0/2465273296 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33a40054b0 con 0x7f33b41002c0 2024-05-10T04:39:45.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b419ffe0 con 0x7f33b41002c0 2024-05-10T04:39:45.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33b17fa700 1 -- 172.21.15.150:0/2465273296 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4016660 con 0x7f33b41002c0 2024-05-10T04:39:45.773 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.769+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b41a0350 con 0x7f33b41002c0 2024-05-10T04:39:45.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.770+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b40678b0 con 0x7f33b41002c0 2024-05-10T04:39:45.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.771+0000 7f33b17fa700 1 -- 172.21.15.150:0/2465273296 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f33a40167c0 con 0x7f33b41002c0 2024-05-10T04:39:45.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.772+0000 7f33b17fa700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f339c0a1260 0x7f339c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:45.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.772+0000 7f33b17fa700 1 -- 172.21.15.150:0/2465273296 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f33a4012070 con 0x7f33b41002c0 2024-05-10T04:39:45.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.772+0000 7f33b37fe700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f339c0a1260 0x7f339c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:45.774 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.773+0000 7f33b37fe700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f339c0a1260 0x7f339c0a3770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005950 tx=0x7f33a8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:45.776 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.776+0000 7f33b17fa700 1 -- 172.21.15.150:0/2465273296 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f33a409c820 con 0x7f33b41002c0 2024-05-10T04:39:45.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:45 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1599985732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:45.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:45 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3302251234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:45.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:45 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/709922222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:45.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:45 smithi161 ceph-mon[46912]: pgmap v275: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-10T04:39:45.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:45 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1599985732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:45.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:45 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3302251234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:39:45.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:45 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/709922222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:39:45.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:45 smithi181 ceph-mon[47314]: pgmap v275: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-05-10T04:39:45.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.974+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f33b41a06c0 con 0x7f33b41002c0 2024-05-10T04:39:45.975 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.975+0000 7f33b17fa700 1 -- 172.21.15.150:0/2465273296 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f33a409c3b0 con 0x7f33b41002c0 2024-05-10T04:39:45.975 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:39:45.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f339c0a1260 msgr2=0x7f339c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f339c0a1260 0x7f339c0a3770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005950 tx=0x7f33a8005f50 comp rx=0 tx=0).stop 2024-05-10T04:39:45.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 msgr2=0x7f33b4197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b4197f50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006010 tx=0x7f33a400cda0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 shutdown_connections 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33b4101d80 0x7f33b419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33b41002c0 0x7f33b4197f50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f339c0a1260 0x7f339c0a3770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 --2- 172.21.15.150:0/2465273296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b41013e0 0x7f33b4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 >> 172.21.15.150:0/2465273296 conn(0x7f33b40fb8f0 msgr2=0x7f33b4106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 shutdown_connections 2024-05-10T04:39:45.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:45.977+0000 7f33ba771700 1 -- 172.21.15.150:0/2465273296 wait complete. 2024-05-10T04:39:45.990 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:39:46.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 -- 172.21.15.150:0/4201517307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8105020 msgr2=0x7f58a8107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 --2- 172.21.15.150:0/4201517307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8105020 0x7f58a8107440 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5898009990 tx=0x7f5898009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 -- 172.21.15.150:0/4201517307 shutdown_connections 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 --2- 172.21.15.150:0/4201517307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8107980 0x7f58a8109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 --2- 172.21.15.150:0/4201517307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a81026c0 0x7f58a8104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 --2- 172.21.15.150:0/4201517307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8105020 0x7f58a8107440 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 -- 172.21.15.150:0/4201517307 >> 172.21.15.150:0/4201517307 conn(0x7f58a80fc0c0 msgr2=0x7f58a80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 -- 172.21.15.150:0/4201517307 shutdown_connections 2024-05-10T04:39:46.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.124+0000 7f58ad186700 1 -- 172.21.15.150:0/4201517307 wait complete. 2024-05-10T04:39:46.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 Processor -- start 2024-05-10T04:39:46.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 -- start start 2024-05-10T04:39:46.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a81026c0 0x7f58a8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8105020 0x7f58a8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8107980 0x7f58a819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a819fab0 con 0x7f58a8107980 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a819fc20 con 0x7f58a81026c0 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58ad186700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a819fde0 con 0x7f58a8105020 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58a759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8107980 0x7f58a819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58a759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8107980 0x7f58a819f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48130/0 (socket says 172.21.15.150:48130) 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58a759e700 1 -- 172.21.15.150:0/803283823 learned_addr learned my addr 172.21.15.150:0/803283823 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58a6d9d700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a81026c0 0x7f58a8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.125+0000 7f58a659c700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8105020 0x7f58a8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a759e700 1 -- 172.21.15.150:0/803283823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8105020 msgr2=0x7f58a8198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a759e700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8105020 0x7f58a8198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a759e700 1 -- 172.21.15.150:0/803283823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a81026c0 msgr2=0x7f58a8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a759e700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a81026c0 0x7f58a8197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a759e700 1 -- 172.21.15.150:0/803283823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980095f0 con 0x7f58a8107980 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a659c700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8105020 0x7f58a8198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a6d9d700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a81026c0 0x7f58a8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58a759e700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8107980 0x7f58a819f4a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58a000ef90 tx=0x7f58a000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f588ffff700 1 -- 172.21.15.150:0/803283823 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0018070 con 0x7f58a8107980 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a81a0060 con 0x7f58a8107980 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f588ffff700 1 -- 172.21.15.150:0/803283823 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a000f040 con 0x7f58a8107980 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f588ffff700 1 -- 172.21.15.150:0/803283823 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0003ea0 con 0x7f58a8107980 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.126+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a81a0520 con 0x7f58a8107980 2024-05-10T04:39:46.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.127+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a80fdc70 con 0x7f58a8107980 2024-05-10T04:39:46.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.128+0000 7f588ffff700 1 -- 172.21.15.150:0/803283823 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f58a000cb00 con 0x7f58a8107980 2024-05-10T04:39:46.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.129+0000 7f588ffff700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58900a11b0 0x7f58900a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.129+0000 7f588ffff700 1 -- 172.21.15.150:0/803283823 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f58a00cd380 con 0x7f58a8107980 2024-05-10T04:39:46.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.129+0000 7f58a6d9d700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58900a11b0 0x7f58900a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.129+0000 7f58a6d9d700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58900a11b0 0x7f58900a36c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f589c005fd0 tx=0x7f589c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.132 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.132+0000 7f588ffff700 1 -- 172.21.15.150:0/803283823 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f58a00966d0 con 0x7f58a8107980 2024-05-10T04:39:46.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.376+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58a80678b0 con 0x7f58a8107980 2024-05-10T04:39:46.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.377+0000 7f588ffff700 1 -- 172.21.15.150:0/803283823 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f58a0099cf0 con 0x7f58a8107980 2024-05-10T04:39:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:46.378 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:39:46.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58900a11b0 msgr2=0x7f58900a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58900a11b0 0x7f58900a36c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f589c005fd0 tx=0x7f589c005f60 comp rx=0 tx=0).stop 2024-05-10T04:39:46.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8107980 msgr2=0x7f58a819f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8107980 0x7f58a819f4a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58a000ef90 tx=0x7f58a000c450 comp rx=0 tx=0).stop 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 shutdown_connections 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8107980 0x7f58a819f4a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a81026c0 0x7f58a8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58900a11b0 0x7f58900a36c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 --2- 172.21.15.150:0/803283823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a8105020 0x7f58a8198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 >> 172.21.15.150:0/803283823 conn(0x7f58a80fc0c0 msgr2=0x7f58a81046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 shutdown_connections 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.380+0000 7f58ad186700 1 -- 172.21.15.150:0/803283823 wait complete. 2024-05-10T04:39:46.381 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:39:46.394 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:39:46.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.527+0000 7f9b27330700 1 -- 172.21.15.150:0/1363758793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20100ac0 msgr2=0x7f9b20100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.527+0000 7f9b27330700 1 --2- 172.21.15.150:0/1363758793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20100ac0 0x7f9b20100ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009990 tx=0x7f9b1c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.527+0000 7f9b27330700 1 -- 172.21.15.150:0/1363758793 shutdown_connections 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.527+0000 7f9b27330700 1 --2- 172.21.15.150:0/1363758793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b20102580 0x7f9b20108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.527+0000 7f9b27330700 1 --2- 172.21.15.150:0/1363758793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20100ac0 0x7f9b20100ed0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.527+0000 7f9b27330700 1 --2- 172.21.15.150:0/1363758793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b20101be0 0x7f9b20101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.527+0000 7f9b27330700 1 -- 172.21.15.150:0/1363758793 >> 172.21.15.150:0/1363758793 conn(0x7f9b200fc0d0 msgr2=0x7f9b200fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.528+0000 7f9b27330700 1 -- 172.21.15.150:0/1363758793 shutdown_connections 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.528+0000 7f9b27330700 1 -- 172.21.15.150:0/1363758793 wait complete. 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.528+0000 7f9b27330700 1 Processor -- start 2024-05-10T04:39:46.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.528+0000 7f9b27330700 1 -- start start 2024-05-10T04:39:46.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b20100ac0 0x7f9b20196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20101be0 0x7f9b201973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b20102580 0x7f9b2019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019e910 con 0x7f9b20101be0 2024-05-10T04:39:46.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019ea50 con 0x7f9b20102580 2024-05-10T04:39:46.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019ebe0 con 0x7f9b20100ac0 2024-05-10T04:39:46.532 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20101be0 0x7f9b201973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20101be0 0x7f9b201973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48152/0 (socket says 172.21.15.150:48152) 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 -- 172.21.15.150:0/677513850 learned_addr learned my addr 172.21.15.150:0/677513850 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b258cd700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b20102580 0x7f9b2019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b250cc700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b20100ac0 0x7f9b20196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 -- 172.21.15.150:0/677513850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b20100ac0 msgr2=0x7f9b20196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b20100ac0 0x7f9b20196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 -- 172.21.15.150:0/677513850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b20102580 msgr2=0x7f9b2019e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b20102580 0x7f9b2019e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 -- 172.21.15.150:0/677513850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0095f0 con 0x7f9b20101be0 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b250cc700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b20100ac0 0x7f9b20196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b258cd700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b20102580 0x7f9b2019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b248cb700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20101be0 0x7f9b201973b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c9e0 tx=0x7f9b1400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b127fc700 1 -- 172.21.15.150:0/677513850 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14007ac0 con 0x7f9b20101be0 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2019eec0 con 0x7f9b20101be0 2024-05-10T04:39:46.533 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b127fc700 1 -- 172.21.15.150:0/677513850 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b14007c20 con 0x7f9b20101be0 2024-05-10T04:39:46.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b127fc700 1 -- 172.21.15.150:0/677513850 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14018660 con 0x7f9b20101be0 2024-05-10T04:39:46.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.529+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2019f2f0 con 0x7f9b20101be0 2024-05-10T04:39:46.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.530+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b20107cd0 con 0x7f9b20101be0 2024-05-10T04:39:46.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.532+0000 7f9b127fc700 1 -- 172.21.15.150:0/677513850 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9b1400f450 con 0x7f9b20101be0 2024-05-10T04:39:46.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.532+0000 7f9b127fc700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b0c0a11b0 0x7f9b0c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.533+0000 7f9b250cc700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b0c0a11b0 0x7f9b0c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.533+0000 7f9b127fc700 1 -- 172.21.15.150:0/677513850 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9b140ccfc0 con 0x7f9b20101be0 2024-05-10T04:39:46.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.533+0000 7f9b250cc700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b0c0a11b0 0x7f9b0c0a36c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c000c00 tx=0x7f9b1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.536+0000 7f9b127fc700 1 -- 172.21.15.150:0/677513850 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9b14096280 con 0x7f9b20101be0 2024-05-10T04:39:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:46 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3591039781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:46 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2465273296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:46 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/803283823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:46.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.737+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b2010c2a0 con 0x7f9b20101be0 2024-05-10T04:39:46.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.739+0000 7f9b127fc700 1 -- 172.21.15.150:0/677513850 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24520 (secure 0 0 0) 0x7f9b140998a0 con 0x7f9b20101be0 2024-05-10T04:39:46.739 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:39:46.739 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":88,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:39:12.434464+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:39:46.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b0c0a11b0 msgr2=0x7f9b0c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b0c0a11b0 0x7f9b0c0a36c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c000c00 tx=0x7f9b1c005fb0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20101be0 msgr2=0x7f9b201973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20101be0 0x7f9b201973b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c9e0 tx=0x7f9b1400cda0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 shutdown_connections 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b20102580 0x7f9b2019e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9b0c0a11b0 0x7f9b0c0a36c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b20100ac0 0x7f9b20196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 --2- 172.21.15.150:0/677513850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b20101be0 0x7f9b201973b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 >> 172.21.15.150:0/677513850 conn(0x7f9b200fc0d0 msgr2=0x7f9b20104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 shutdown_connections 2024-05-10T04:39:46.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.741+0000 7f9b27330700 1 -- 172.21.15.150:0/677513850 wait complete. 2024-05-10T04:39:46.755 DEBUG:tasks.fwd_scrub:Yielding 2024-05-10T04:39:46.755 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-10T04:39:46.773 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:39:46.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-10T04:39:46.774 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-10T04:39:46.774 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:39:46.774 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:39:46.776 INFO:teuthology.run_tasks:Running task workunit... 2024-05-10T04:39:46.788 INFO:tasks.workunit:Pulling workunits from ref 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:39:46.788 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-10T04:39:46.788 INFO:tasks.workunit:timeout=3h 2024-05-10T04:39:46.788 INFO:tasks.workunit:cleanup=True 2024-05-10T04:39:46.788 DEBUG:teuthology.orchestra.run.smithi150:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-10 04:38:36.206783024 +0000 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-10 04:38:36.206783024 +0000 2024-05-10T04:39:46.806 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-10 04:39:37.558334305 +0000 2024-05-10T04:39:46.807 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-10 04:38:36.206783024 +0000 2024-05-10T04:39:46.807 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-10T04:39:46.807 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-10T04:39:46.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:46 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3591039781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:46.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:46 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2465273296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:46.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:46 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/803283823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:46.889 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 50174015a7e79f951b988949abe236791f52d4da 2024-05-10T04:39:46.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:46 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3591039781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:39:46.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:46 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2465273296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:39:46.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:46 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/803283823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:39:46.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 -- 172.21.15.150:0/2359251746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc105040 msgr2=0x7f50cc107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/2359251746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc105040 0x7f50cc107460 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009990 tx=0x7f50bc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 -- 172.21.15.150:0/2359251746 shutdown_connections 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/2359251746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc1079a0 0x7f50cc109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/2359251746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc1026e0 0x7f50cc104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/2359251746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc105040 0x7f50cc107460 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 -- 172.21.15.150:0/2359251746 >> 172.21.15.150:0/2359251746 conn(0x7f50cc0fc0e0 msgr2=0x7f50cc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 -- 172.21.15.150:0/2359251746 shutdown_connections 2024-05-10T04:39:46.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.917+0000 7f50d1eeb700 1 -- 172.21.15.150:0/2359251746 wait complete. 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 Processor -- start 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 -- start start 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc1026e0 0x7f50cc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc105040 0x7f50cc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc1079a0 0x7f50cc19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19f9c0 con 0x7f50cc1079a0 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19fb30 con 0x7f50cc105040 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50d1eeb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19fcf0 con 0x7f50cc1026e0 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.918+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc1079a0 0x7f50cc19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc1079a0 0x7f50cc19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48174/0 (socket says 172.21.15.150:48174) 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 -- 172.21.15.150:0/781396616 learned_addr learned my addr 172.21.15.150:0/781396616 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50caffd700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc105040 0x7f50cc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc1026e0 0x7f50cc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 -- 172.21.15.150:0/781396616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc1026e0 msgr2=0x7f50cc197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc1026e0 0x7f50cc197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 -- 172.21.15.150:0/781396616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc105040 msgr2=0x7f50cc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc105040 0x7f50cc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 -- 172.21.15.150:0/781396616 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0095f0 con 0x7f50cc1079a0 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc1026e0 0x7f50cc197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50cbfff700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc1079a0 0x7f50cc19f3b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f50c400ada0 tx=0x7f50c400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50c8ff9700 1 -- 172.21.15.150:0/781396616 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c400f070 con 0x7f50cc1079a0 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50c8ff9700 1 -- 172.21.15.150:0/781396616 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c4007b00 con 0x7f50cc1079a0 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc19ffd0 con 0x7f50cc1079a0 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50c8ff9700 1 -- 172.21.15.150:0/781396616 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c40053a0 con 0x7f50cc1079a0 2024-05-10T04:39:46.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.919+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc1a03f0 con 0x7f50cc1079a0 2024-05-10T04:39:46.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.922+0000 7f50c8ff9700 1 -- 172.21.15.150:0/781396616 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f50c4005500 con 0x7f50cc1079a0 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.922+0000 7f50c8ff9700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f50b80a1260 0x7f50b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.922+0000 7f50c8ff9700 1 -- 172.21.15.150:0/781396616 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50c40cc270 con 0x7f50cc1079a0 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.922+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f50b80a1260 0x7f50b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f50b80a1260 0x7f50b80a3770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005fd0 tx=0x7f50b4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 Processor -- start 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 -- start start 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f7720 0x55e7626f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e7626f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76261e1d0 con 0x55e7626f8f40 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76261bb90 con 0x55e7626f7720 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.923+0000 7f50e222fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e762617480 con 0x55e7626f85f0 2024-05-10T04:39:46.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48176/0 (socket says 172.21.15.150:48176) 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 -- 172.21.15.150:0/2840352492 learned_addr learned my addr 172.21.15.150:0/2840352492 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f7720 0x55e7626f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50caffd700 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e7626f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 -- 172.21.15.150:0/2840352492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 msgr2=0x55e7626f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e7626f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 -- 172.21.15.150:0/2840352492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f7720 msgr2=0x55e7626f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f7720 0x55e7626f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 -- 172.21.15.150:0/2840352492 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e762498430 con 0x55e7626f8f40 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50cbfff700 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ff840 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f50c400a340 tx=0x7f50c401af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6644a776b1b64e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50c1ffb700 1 -- 172.21.15.150:0/2840352492 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c409d160 con 0x55e7626f8f40 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50c1ffb700 1 -- 172.21.15.150:0/2840352492 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c40980d0 con 0x55e7626f8f40 2024-05-10T04:39:46.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.924+0000 7f50c1ffb700 1 -- 172.21.15.150:0/2840352492 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c409a100 con 0x55e7626f8f40 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 -- 172.21.15.150:0/2840352492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 msgr2=0x55e7626ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ff840 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f50c400a340 tx=0x7f50c401af70 comp rx=0 tx=0).stop 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 -- 172.21.15.150:0/2840352492 shutdown_connections 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ff840 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f7720 0x55e7626f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 --2- 172.21.15.150:0/2840352492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e7626f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 -- 172.21.15.150:0/2840352492 >> 172.21.15.150:0/2840352492 conn(0x55e762659ae0 msgr2=0x55e7626f2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 -- 172.21.15.150:0/2840352492 shutdown_connections 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.925+0000 7f50e222fb80 1 -- 172.21.15.150:0/2840352492 wait complete. 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 -- start start 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 Processor -- start 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f7720 0x55e7627017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e762701d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e762708c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7625b35e0 con 0x55e7626f7720 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7625b3280 con 0x55e7626f8f40 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e76249a720 con 0x55e7626f85f0 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50e222fb80 10 client.0 _ll_register_callbacks cb 0x55e7625bcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50cb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f7720 0x55e7627017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50cb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f7720 0x55e7627017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48178/0 (socket says 172.21.15.150:48178) 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.926+0000 7f50cb7fe700 1 -- 172.21.15.150:0/3287895622 learned_addr learned my addr 172.21.15.150:0/3287895622 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:46.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50caffd700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e762701d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cbfff700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e762708c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cb7fe700 1 -- 172.21.15.150:0/3287895622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 msgr2=0x55e762701d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e762701d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cb7fe700 1 -- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 msgr2=0x55e762708c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e762708c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cb7fe700 1 -- 172.21.15.150:0/3287895622 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc1a03f0 con 0x55e7626f7720 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cbfff700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e762708c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f7720 0x55e7627017c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f50b400f7d0 tx=0x7f50b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db574e8714724d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50aaffd700 1 -- 172.21.15.150:0/3287895622 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b401d640 con 0x55e7626f7720 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50aaffd700 1 -- 172.21.15.150:0/3287895622 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50b401d7a0 con 0x55e7626f7720 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7625702f0 con 0x55e7626f7720 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50aaffd700 1 -- 172.21.15.150:0/3287895622 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b401d960 con 0x55e7626f7720 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50e222fb80 10 client.15705 fetch_fsmap learned FSMap version 29 2024-05-10T04:39:46.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.927+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e762444be0 con 0x55e7626f7720 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50aaffd700 1 -- 172.21.15.150:0/3287895622 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7625702f0 con 0x55e7626f7720 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50aaffd700 1 -- 172.21.15.150:0/3287895622 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50b4026c40 con 0x55e7626f7720 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50e222fb80 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50e222fb80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50e222fb80 20 client.15705 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50e222fb80 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e76270d4f0 0x55e76270f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50e222fb80 4 client.15705 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7626c6800 con 0x55e76270d4f0 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.928+0000 7f50caffd700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e76270d4f0 0x55e76270f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50aaffd700 10 client.15705 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50caffd700 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e76270d4f0 0x55e76270f940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7b71b486deec9a server_cookie=ee24a28995c71ea8 in_seq=0 out_seq=0 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50aaffd700 1 -- 172.21.15.150:0/3287895622 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e7626c6800 con 0x55e76270d4f0 2024-05-10T04:39:46.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50aaffd700 10 client.15705 handle_command_reply: tid=0 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50e222fb80 1 client.15705 shutdown 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f7720 msgr2=0x55e7627017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50e222fb80 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f7720 0x55e7627017c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f50b400f7d0 tx=0x7f50b4004750 comp rx=0 tx=0).stop 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 shutdown_connections 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50e222fb80 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e76270d4f0 0x55e76270f940 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50e222fb80 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e762708c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.929+0000 7f50e222fb80 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f7720 0x55e7627017c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.930+0000 7f50e222fb80 1 --2- 172.21.15.150:0/3287895622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f85f0 0x55e762701d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.930+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 >> 172.21.15.150:0/3287895622 conn(0x55e762659ae0 msgr2=0x55e7626f2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.930+0000 7f50aaffd700 0 client.15705 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.930+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 shutdown_connections 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.930+0000 7f50e222fb80 1 -- 172.21.15.150:0/3287895622 wait complete. 2024-05-10T04:39:46.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.930+0000 7f50e222fb80 20 client.15705 trim_cache size 0 max 16384 2024-05-10T04:39:46.936 INFO:tasks.workunit.client.0.smithi150.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 Processor -- start 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 -- start start 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e76270fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f85f0 0x55e762708c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7626f81e0 con 0x55e7626f8f40 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7626fce30 con 0x55e7626f85f0 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50e222fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7626ee230 con 0x55e7626f7720 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50cb7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e76270fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48182/0 (socket says 172.21.15.150:48182) 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.949+0000 7f50cbfff700 1 -- 172.21.15.150:0/534499997 learned_addr learned my addr 172.21.15.150:0/534499997 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:46.951 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50caffd700 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f85f0 0x55e762708c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50cb7fe700 1 -- 172.21.15.150:0/534499997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f85f0 msgr2=0x55e762708c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f85f0 0x55e762708c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50cb7fe700 1 -- 172.21.15.150:0/534499997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 msgr2=0x55e7626ffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50cb7fe700 1 -- 172.21.15.150:0/534499997 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7626ee4d0 con 0x55e7626f7720 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50cbfff700 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e76270fc10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005950 tx=0x7f50b401c5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb4908de82a9391d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50c27fc700 1 -- 172.21.15.150:0/534499997 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400fb90 con 0x55e7626f7720 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50c27fc700 1 -- 172.21.15.150:0/534499997 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50b401c6a0 con 0x55e7626f7720 2024-05-10T04:39:46.952 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.950+0000 7f50c27fc700 1 -- 172.21.15.150:0/534499997 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b40052a0 con 0x55e7626f7720 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 -- 172.21.15.150:0/534499997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 msgr2=0x55e76270fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e76270fc10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005950 tx=0x7f50b401c5a0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 -- 172.21.15.150:0/534499997 shutdown_connections 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f8f40 0x55e7626ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e76270fc10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 --2- 172.21.15.150:0/534499997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f85f0 0x55e762708c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 -- 172.21.15.150:0/534499997 >> 172.21.15.150:0/534499997 conn(0x55e762659ae0 msgr2=0x55e76270f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 -- 172.21.15.150:0/534499997 shutdown_connections 2024-05-10T04:39:46.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.951+0000 7f50e222fb80 1 -- 172.21.15.150:0/534499997 wait complete. 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 -- start start 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 Processor -- start 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e762708840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f85f0 0x55e762706fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e7627074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e762709010 con 0x55e7626f85f0 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e762707a20 con 0x55e7626f8f40 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e762707be0 con 0x55e7626f7720 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50e222fb80 10 client.0 _ll_register_callbacks cb 0x55e7625bcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50caffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f85f0 0x55e762706fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50caffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f85f0 0x55e762706fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48192/0 (socket says 172.21.15.150:48192) 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50caffd700 1 -- 172.21.15.150:0/282221784 learned_addr learned my addr 172.21.15.150:0/282221784 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50cbfff700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e7627074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.952+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e762708840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50caffd700 1 -- 172.21.15.150:0/282221784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 msgr2=0x55e762708840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50caffd700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e762708840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50caffd700 1 -- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 msgr2=0x55e7627074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50caffd700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e7627074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50caffd700 1 -- 172.21.15.150:0/282221784 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e762498430 con 0x55e7626f85f0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50cbfff700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e7627074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50caffd700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f85f0 0x55e762706fa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f50bc003730 tx=0x7f50bc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14af082c8bc7793b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50aaffd700 1 -- 172.21.15.150:0/282221784 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0052b0 con 0x55e7626f85f0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50aaffd700 1 -- 172.21.15.150:0/282221784 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50bc005410 con 0x55e7626f85f0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50c40980d0 con 0x55e7626f85f0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50aaffd700 1 -- 172.21.15.150:0/282221784 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50bc0055d0 con 0x55e7626f85f0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 10 client.15711 fetch_fsmap learned FSMap version 29 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e762444be0 con 0x55e7626f85f0 2024-05-10T04:39:46.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50aaffd700 1 -- 172.21.15.150:0/282221784 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c40980d0 con 0x55e7626f85f0 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50aaffd700 1 -- 172.21.15.150:0/282221784 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50bc0099c0 con 0x55e7626f85f0 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 10 client.15711 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 20 client.15711 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e762764f90 0x55e7627673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 4 client.15711 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.953+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e76260ff40 con 0x55e762764f90 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.954+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e762764f90 0x55e7627673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.954+0000 7f50cb7fe700 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e762764f90 0x55e7627673b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4dbbfa81aa349a server_cookie=c27013128e2b52c8 in_seq=0 out_seq=0 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.954+0000 7f50aaffd700 10 client.15711 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50aaffd700 1 -- 172.21.15.150:0/282221784 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e76260ff40 con 0x55e762764f90 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50aaffd700 10 client.15711 handle_command_reply: tid=0 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 client.15711 shutdown 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f85f0 msgr2=0x55e762706fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f85f0 0x55e762706fa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f50bc003730 tx=0x7f50bc01a040 comp rx=0 tx=0).stop 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 shutdown_connections 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e762764f90 0x55e7627673b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7626f8f40 0x55e7627074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50aaffd700 0 client.15711 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7626f7720 0x55e762708840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 --2- 172.21.15.150:0/282221784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e7626f85f0 0x55e762706fa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.955+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 >> 172.21.15.150:0/282221784 conn(0x55e762659ae0 msgr2=0x55e7626ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 shutdown_connections 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50e222fb80 1 -- 172.21.15.150:0/282221784 wait complete. 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50e222fb80 20 client.15711 trim_cache size 0 max 16384 2024-05-10T04:39:46.961 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:39:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "637a5966-4582-439f-8cc6-edab3fc86bd2", 2024-05-10T04:39:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:39:46.962 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f50b80a1260 msgr2=0x7f50b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f50b80a1260 0x7f50b80a3770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005fd0 tx=0x7f50b4005dc0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc1079a0 msgr2=0x7f50cc19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc1079a0 0x7f50cc19f3b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f50c400ada0 tx=0x7f50c400c330 comp rx=0 tx=0).stop 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 shutdown_connections 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50cc1079a0 0x7f50cc19f3b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f50b80a1260 0x7f50b80a3770 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc1026e0 0x7f50cc197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 --2- 172.21.15.150:0/781396616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50cc105040 0x7f50cc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 >> 172.21.15.150:0/781396616 conn(0x7f50cc0fc0e0 msgr2=0x7f50cc103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:46.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 shutdown_connections 2024-05-10T04:39:46.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:46.956+0000 7f50d1eeb700 1 -- 172.21.15.150:0/781396616 wait complete. 2024-05-10T04:39:46.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:39:47.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 -- 172.21.15.150:0/4271064489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98102580 msgr2=0x7f8c98108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 --2- 172.21.15.150:0/4271064489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98102580 0x7f8c98108e30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000b0a0 tx=0x7f8c9000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 -- 172.21.15.150:0/4271064489 shutdown_connections 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 --2- 172.21.15.150:0/4271064489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98102580 0x7f8c98108e30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 --2- 172.21.15.150:0/4271064489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c98100ac0 0x7f8c98100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 --2- 172.21.15.150:0/4271064489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98101be0 0x7f8c98101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 -- 172.21.15.150:0/4271064489 >> 172.21.15.150:0/4271064489 conn(0x7f8c980fc0d0 msgr2=0x7f8c980fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 -- 172.21.15.150:0/4271064489 shutdown_connections 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.148+0000 7f8c9f500700 1 -- 172.21.15.150:0/4271064489 wait complete. 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 Processor -- start 2024-05-10T04:39:47.151 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 -- start start 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98100ac0 0x7f8c98196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c98101be0 0x7f8c98197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98102580 0x7f8c9819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9819e9d0 con 0x7f8c98100ac0 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9819eb40 con 0x7f8c98102580 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9f500700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9819ed00 con 0x7f8c98101be0 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98100ac0 0x7f8c98196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98100ac0 0x7f8c98196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48212/0 (socket says 172.21.15.150:48212) 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 -- 172.21.15.150:0/33171120 learned_addr learned my addr 172.21.15.150:0/33171120 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c98101be0 0x7f8c98197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 -- 172.21.15.150:0/33171120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c98101be0 msgr2=0x7f8c98197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.152 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c98101be0 0x7f8c98197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9da9d700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98102580 0x7f8c9819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 -- 172.21.15.150:0/33171120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98102580 msgr2=0x7f8c9819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98102580 0x7f8c9819e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.149+0000 7f8c9d29c700 1 -- 172.21.15.150:0/33171120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90009d00 con 0x7f8c98100ac0 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.150+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98100ac0 0x7f8c98196ef0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800d950 tx=0x7f8c8800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.150+0000 7f8c8e7fc700 1 -- 172.21.15.150:0/33171120 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88004220 con 0x7f8c98100ac0 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.150+0000 7f8c8e7fc700 1 -- 172.21.15.150:0/33171120 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c8800de30 con 0x7f8c98100ac0 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.150+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9819efe0 con 0x7f8c98100ac0 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.150+0000 7f8c8e7fc700 1 -- 172.21.15.150:0/33171120 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88003cb0 con 0x7f8c98100ac0 2024-05-10T04:39:47.153 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.150+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9819f410 con 0x7f8c98100ac0 2024-05-10T04:39:47.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.153+0000 7f8c8e7fc700 1 -- 172.21.15.150:0/33171120 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f8c88004380 con 0x7f8c98100ac0 2024-05-10T04:39:47.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.153+0000 7f8c8e7fc700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8c840a1260 0x7f8c840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8c840a1260 0x7f8c840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8c8e7fc700 1 -- 172.21.15.150:0/33171120 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c880cc9d0 con 0x7f8c98100ac0 2024-05-10T04:39:47.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 Processor -- start 2024-05-10T04:39:47.161 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 -- start start 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8c840a1260 0x7f8c840a3770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8c98197f90 tx=0x7f8c940094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b6174d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6175b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b6099150 con 0x5652b6175780 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b6096b10 con 0x5652b61748b0 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8caf844b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b6092400 con 0x5652b61760d0 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8c9ca9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6175b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.154+0000 7f8c9ca9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6175b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48218/0 (socket says 172.21.15.150:48218) 2024-05-10T04:39:47.162 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1126334452 learned_addr learned my addr 172.21.15.150:0/1126334452 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9da9d700 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b6174d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1126334452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 msgr2=0x5652b617c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1126334452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 msgr2=0x5652b6174d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b6174d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1126334452 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652b5f133b0 con 0x5652b6175780 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b6174d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6175b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400ec50 tx=0x7f8c94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32900d5454a432e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c837fe700 1 -- 172.21.15.150:0/1126334452 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94011b90 con 0x5652b6175780 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c837fe700 1 -- 172.21.15.150:0/1126334452 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c94011cf0 con 0x5652b6175780 2024-05-10T04:39:47.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.155+0000 7f8c837fe700 1 -- 172.21.15.150:0/1126334452 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94010730 con 0x5652b6175780 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 -- 172.21.15.150:0/1126334452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 msgr2=0x5652b6175b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6175b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400ec50 tx=0x7f8c94011000 comp rx=0 tx=0).stop 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 -- 172.21.15.150:0/1126334452 shutdown_connections 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b6174d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1126334452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6175b90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 -- 172.21.15.150:0/1126334452 >> 172.21.15.150:0/1126334452 conn(0x5652b60d4a60 msgr2=0x5652b616cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 -- 172.21.15.150:0/1126334452 shutdown_connections 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 -- 172.21.15.150:0/1126334452 wait complete. 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.156+0000 7f8caf844b80 1 -- start start 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 1 Processor -- start 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b617e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b617ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b6185cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b602e560 con 0x5652b6175780 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b602e200 con 0x5652b61748b0 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b5f156a0 con 0x5652b61760d0 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8caf844b80 10 client.0 _ll_register_callbacks cb 0x5652b6037b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8c9ca9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b617ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8c9ca9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b617ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48226/0 (socket says 172.21.15.150:48226) 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1508345901 learned_addr learned my addr 172.21.15.150:0/1508345901 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b617e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8c9da9d700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b6185cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.157+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1508345901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 msgr2=0x5652b6185cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b6185cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 msgr2=0x5652b617e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b617e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/1508345901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9400e900 con 0x5652b6175780 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b617e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c9da9d700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b6185cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b617ed60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097c0 tx=0x7f8c9401e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5915713eb9275b30 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c80ff9700 1 -- 172.21.15.150:0/1508345901 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9401ec60 con 0x5652b6175780 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c80ff9700 1 -- 172.21.15.150:0/1508345901 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c9402e070 con 0x5652b6175780 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652b5feb270 con 0x5652b6175780 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c80ff9700 1 -- 172.21.15.150:0/1508345901 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c940104b0 con 0x5652b6175780 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8caf844b80 10 client.15723 fetch_fsmap learned FSMap version 29 2024-05-10T04:39:47.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5652b5ebfb60 con 0x5652b6175780 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c80ff9700 1 -- 172.21.15.150:0/1508345901 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652b5feb270 con 0x5652b6175780 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8c80ff9700 1 -- 172.21.15.150:0/1508345901 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c94033690 con 0x5652b6175780 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8caf844b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8caf844b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.158+0000 7f8caf844b80 20 client.15723 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.159+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b618a4c0 0x5652b618c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.159+0000 7f8caf844b80 4 client.15723 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.159+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652b6141780 con 0x5652b618a4c0 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.159+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b618a4c0 0x5652b618c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.159+0000 7f8c80ff9700 10 client.15723 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.159+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b618a4c0 0x5652b618c910 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58318a3c58104c0d server_cookie=5c81bb13203c73a0 in_seq=0 out_seq=0 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8c80ff9700 1 -- 172.21.15.150:0/1508345901 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652b6141780 con 0x5652b618a4c0 2024-05-10T04:39:47.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8c80ff9700 10 client.15723 handle_command_reply: tid=0 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 client.15723 shutdown 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 msgr2=0x5652b617ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b617ed60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097c0 tx=0x7f8c9401e4f0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 shutdown_connections 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b618a4c0 0x5652b618c910 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b6185cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61748b0 0x5652b617e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8c80ff9700 0 client.15723 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 --2- 172.21.15.150:0/1508345901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b617ed60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 >> 172.21.15.150:0/1508345901 conn(0x5652b60d4a60 msgr2=0x5652b616cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 shutdown_connections 2024-05-10T04:39:47.168 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 1 -- 172.21.15.150:0/1508345901 wait complete. 2024-05-10T04:39:47.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.160+0000 7f8caf844b80 20 client.15723 trim_cache size 0 max 16384 2024-05-10T04:39:47.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 Processor -- start 2024-05-10T04:39:47.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 -- start start 2024-05-10T04:39:47.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b61748b0 0x5652b61911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b6175780 0x5652b617e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b616e770 con 0x5652b61748b0 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b618a110 con 0x5652b6175780 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8caf844b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b618cb30 con 0x5652b61760d0 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8c9d29c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b61748b0 0x5652b61911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8c9d29c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b61748b0 0x5652b61911e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48236/0 (socket says 172.21.15.150:48236) 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.179+0000 7f8c9d29c700 1 -- 172.21.15.150:0/370618875 learned_addr learned my addr 172.21.15.150:0/370618875 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b6175780 0x5652b617e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9da9d700 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9d29c700 1 -- 172.21.15.150:0/370618875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 msgr2=0x5652b617ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9d29c700 1 -- 172.21.15.150:0/370618875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b6175780 msgr2=0x5652b617e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b6175780 0x5652b617e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9d29c700 1 -- 172.21.15.150:0/370618875 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652b6058560 con 0x5652b61748b0 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9da9d700 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b61748b0 0x5652b61911e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009520 tx=0x7f8c88000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd33c91ab0e4c8aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c83fff700 1 -- 172.21.15.150:0/370618875 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c880b22b0 con 0x5652b61748b0 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c83fff700 1 -- 172.21.15.150:0/370618875 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c88097d50 con 0x5652b61748b0 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.180+0000 7f8c83fff700 1 -- 172.21.15.150:0/370618875 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88099c10 con 0x5652b61748b0 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 -- 172.21.15.150:0/370618875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b61748b0 msgr2=0x5652b61911e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b61748b0 0x5652b61911e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009520 tx=0x7f8c88000bc0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 -- 172.21.15.150:0/370618875 shutdown_connections 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61760d0 0x5652b617ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b61748b0 0x5652b61911e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.187 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 --2- 172.21.15.150:0/370618875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b6175780 0x5652b617e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 -- 172.21.15.150:0/370618875 >> 172.21.15.150:0/370618875 conn(0x5652b60d4a60 msgr2=0x5652b61789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 -- 172.21.15.150:0/370618875 shutdown_connections 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 -- 172.21.15.150:0/370618875 wait complete. 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 -- start start 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.181+0000 7f8caf844b80 1 Processor -- start 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61748b0 0x5652b61839a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6183ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61760d0 0x5652b6184e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b6185350 con 0x5652b6175780 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b61854c0 con 0x5652b61760d0 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b6185680 con 0x5652b61748b0 2024-05-10T04:39:47.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 10 client.0 _ll_register_callbacks cb 0x5652b6037b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6183ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6183ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:48244/0 (socket says 172.21.15.150:48244) 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/4079966654 learned_addr learned my addr 172.21.15.150:0/4079966654 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61748b0 0x5652b61839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9da9d700 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61760d0 0x5652b6184e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/4079966654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61748b0 msgr2=0x5652b61839a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61748b0 0x5652b61839a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61760d0 msgr2=0x5652b6184e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61760d0 0x5652b6184e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 -- 172.21.15.150:0/4079966654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9819f410 con 0x5652b6175780 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c9ca9b700 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6183ee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c94033af0 tx=0x7f8c94033730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2394acbd5d9cd91 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:47.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c80ff9700 1 -- 172.21.15.150:0/4079966654 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9401ee70 con 0x5652b6175780 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8c80ff9700 1 -- 172.21.15.150:0/4079966654 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c9402e040 con 0x5652b6175780 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.182+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c94011cf0 con 0x5652b6175780 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8c80ff9700 1 -- 172.21.15.150:0/4079966654 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c9401e610 con 0x5652b6175780 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 10 client.15729 fetch_fsmap learned FSMap version 29 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5652b5f17cd0 con 0x5652b6175780 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8c80ff9700 1 -- 172.21.15.150:0/4079966654 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c94011cf0 con 0x5652b6175780 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8c80ff9700 1 -- 172.21.15.150:0/4079966654 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c94027c10 con 0x5652b6175780 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 10 client.15729 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:39:47.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 20 client.15729 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b61e0af0 0x5652b61e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 4 client.15729 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652b608aec0 con 0x5652b61e0af0 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.183+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b61e0af0 0x5652b61e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8c9d29c700 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b61e0af0 0x5652b61e2f10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25eeb1979be08680 server_cookie=25ee91ed8ba3615c in_seq=0 out_seq=0 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8c80ff9700 10 client.15729 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8c80ff9700 1 -- 172.21.15.150:0/4079966654 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5652b608aec0 con 0x5652b61e0af0 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8c80ff9700 10 client.15729 handle_command_reply: tid=0 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 client.15729 shutdown 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 msgr2=0x5652b6183ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6183ee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c94033af0 tx=0x7f8c94033730 comp rx=0 tx=0).stop 2024-05-10T04:39:47.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 shutdown_connections 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5652b61e0af0 0x5652b61e2f10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652b61760d0 0x5652b6184e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8c80ff9700 0 client.15729 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5652b61748b0 0x5652b61839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 --2- 172.21.15.150:0/4079966654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5652b6175780 0x5652b6183ee0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 >> 172.21.15.150:0/4079966654 conn(0x5652b60d4a60 msgr2=0x5652b61789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 shutdown_connections 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 1 -- 172.21.15.150:0/4079966654 wait complete. 2024-05-10T04:39:47.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.184+0000 7f8caf844b80 20 client.15729 trim_cache size 0 max 16384 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: "637a5966-4582-439f-8cc6-edab3fc86bd2": { 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "637a5966-4582-439f-8cc6-edab3fc86bd2", 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:39:47.193 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:39:47.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8c840a1260 msgr2=0x7f8c840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8c840a1260 0x7f8c840a3770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8c98197f90 tx=0x7f8c940094f0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98100ac0 msgr2=0x7f8c98196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:47.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98100ac0 0x7f8c98196ef0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800d950 tx=0x7f8c8800dd10 comp rx=0 tx=0).stop 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 shutdown_connections 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98102580 0x7f8c9819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c98100ac0 0x7f8c98196ef0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8c840a1260 0x7f8c840a3770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 --2- 172.21.15.150:0/33171120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c98101be0 0x7f8c98197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 >> 172.21.15.150:0/33171120 conn(0x7f8c980fc0d0 msgr2=0x7f8c98104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 shutdown_connections 2024-05-10T04:39:47.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:39:47.185+0000 7f8c9f500700 1 -- 172.21.15.150:0/33171120 wait complete. 2024-05-10T04:39:47.198 INFO:tasks.cephfs.filesystem:scrub status for tag:637a5966-4582-439f-8cc6-edab3fc86bd2 - {'path': '/', 'tag': '637a5966-4582-439f-8cc6-edab3fc86bd2', 'options': 'recursive,force'} 2024-05-10T04:39:47.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:47 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/677513850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:47.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:47 smithi161 ceph-mon[46912]: pgmap v276: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-10T04:39:47.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:47 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/677513850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:47.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:47 smithi181 ceph-mon[47314]: pgmap v276: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-10T04:39:47.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:47 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/677513850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:39:47.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:47 smithi150 ceph-mon[42404]: pgmap v276: 129 pgs: 129 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-10T04:39:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:48 smithi161 ceph-mon[46912]: scrub queued for path: / 2024-05-10T04:39:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:48 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:39:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:48 smithi161 ceph-mon[46912]: scrub summary: active paths [/] 2024-05-10T04:39:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:48 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:39:48.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:48 smithi181 ceph-mon[47314]: scrub queued for path: / 2024-05-10T04:39:48.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:48 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:39:48.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:48 smithi181 ceph-mon[47314]: scrub summary: active paths [/] 2024-05-10T04:39:48.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:48 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:39:48.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:48 smithi150 ceph-mon[42404]: scrub queued for path: / 2024-05-10T04:39:48.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:48 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:39:48.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:48 smithi150 ceph-mon[42404]: scrub summary: active paths [/] 2024-05-10T04:39:48.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:48 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:39:49.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:49 smithi161 ceph-mon[46912]: pgmap v277: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:39:49.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:49 smithi181 ceph-mon[47314]: pgmap v277: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:39:49.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:49 smithi150 ceph-mon[42404]: pgmap v277: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:39:52.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:52 smithi181 ceph-mon[47314]: pgmap v278: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-10T04:39:52.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:52 smithi150 ceph-mon[42404]: pgmap v278: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-10T04:39:52.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:52 smithi161 ceph-mon[46912]: pgmap v278: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-10T04:39:54.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:54 smithi181 ceph-mon[47314]: pgmap v279: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:39:54.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:54 smithi150 ceph-mon[42404]: pgmap v279: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:39:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:54 smithi161 ceph-mon[46912]: pgmap v279: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:39:56.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:56 smithi181 ceph-mon[47314]: pgmap v280: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-10T04:39:56.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:56 smithi150 ceph-mon[42404]: pgmap v280: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-10T04:39:56.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:56 smithi161 ceph-mon[46912]: pgmap v280: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-10T04:39:58.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:58 smithi181 ceph-mon[47314]: pgmap v281: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T04:39:58.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:39:58 smithi181 ceph-mon[47314]: scrub summary: idle 2024-05-10T04:39:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:58 smithi150 ceph-mon[42404]: pgmap v281: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T04:39:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:39:58 smithi150 ceph-mon[42404]: scrub summary: idle 2024-05-10T04:39:58.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:58 smithi161 ceph-mon[46912]: pgmap v281: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-10T04:39:58.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:39:58 smithi161 ceph-mon[46912]: scrub summary: idle 2024-05-10T04:39:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 -- 172.21.15.150:0/82667864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0100d20 msgr2=0x7f6ee0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/82667864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0100d20 0x7f6ee0101130 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009990 tx=0x7f6ed0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 -- 172.21.15.150:0/82667864 shutdown_connections 2024-05-10T04:39:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/82667864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee0101670 0x7f6ee0108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/82667864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee0102020 0x7f6ee0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/82667864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0100d20 0x7f6ee0101130 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 -- 172.21.15.150:0/82667864 >> 172.21.15.150:0/82667864 conn(0x7f6ee00fb8e0 msgr2=0x7f6ee00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 -- 172.21.15.150:0/82667864 shutdown_connections 2024-05-10T04:39:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.953+0000 7f6ee53b1700 1 -- 172.21.15.150:0/82667864 wait complete. 2024-05-10T04:39:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.954+0000 7f6ee53b1700 1 Processor -- start 2024-05-10T04:39:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ee53b1700 1 -- start start 2024-05-10T04:39:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ee53b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee0100d20 0x7f6ee019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ee53b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0101670 0x7f6ee019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ee53b1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee0102020 0x7f6ee01a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ee53b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee01a1bd0 con 0x7f6ee0101670 2024-05-10T04:39:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ee53b1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee01a1d10 con 0x7f6ee0102020 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ee53b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee01a1ea0 con 0x7f6ee0100d20 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0101670 0x7f6ee019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0101670 0x7f6ee019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55436/0 (socket says 172.21.15.150:55436) 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 -- 172.21.15.150:0/2700009431 learned_addr learned my addr 172.21.15.150:0/2700009431 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6edeffd700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee0100d20 0x7f6ee019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee0102020 0x7f6ee01a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 -- 172.21.15.150:0/2700009431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee0100d20 msgr2=0x7f6ee019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee0100d20 0x7f6ee019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 -- 172.21.15.150:0/2700009431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee0102020 msgr2=0x7f6ee01a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee0102020 0x7f6ee01a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6ede7fc700 1 -- 172.21.15.150:0/2700009431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed00095f0 con 0x7f6ee0101670 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.955+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee0102020 0x7f6ee01a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.956+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0101670 0x7f6ee019a700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0003730 tx=0x7f6ed000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.956+0000 7f6ec7fff700 1 -- 172.21.15.150:0/2700009431 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed001d070 con 0x7f6ee0101670 2024-05-10T04:39:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.956+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee01a2120 con 0x7f6ee0101670 2024-05-10T04:39:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.956+0000 7f6ec7fff700 1 -- 172.21.15.150:0/2700009431 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed0005510 con 0x7f6ee0101670 2024-05-10T04:39:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.956+0000 7f6ec7fff700 1 -- 172.21.15.150:0/2700009431 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0004bc0 con 0x7f6ee0101670 2024-05-10T04:39:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.956+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee01a2540 con 0x7f6ee0101670 2024-05-10T04:39:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.959+0000 7f6ec7fff700 1 -- 172.21.15.150:0/2700009431 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f6ed0004d40 con 0x7f6ee0101670 2024-05-10T04:39:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.959+0000 7f6ec7fff700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6ec80a11b0 0x7f6ec80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6edeffd700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6ec80a11b0 0x7f6ec80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ec7fff700 1 -- 172.21.15.150:0/2700009431 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ed00cde70 con 0x7f6ee0101670 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 Processor -- start 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6edeffd700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6ec80a11b0 0x7f6ec80a36c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4005950 tx=0x7f6ed40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 -- start start 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c846a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846a3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c846aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c845c90f0 con 0x563c846a3050 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c845c6ab0 con 0x563c846a39a0 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c845c23a0 con 0x563c846a2180 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ede7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.960+0000 7f6ede7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846a3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55446/0 (socket says 172.21.15.150:55446) 2024-05-10T04:39:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ede7fc700 1 -- 172.21.15.150:0/3842178900 learned_addr learned my addr 172.21.15.150:0/3842178900 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6edeffd700 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c846a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c846aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ede7fc700 1 -- 172.21.15.150:0/3842178900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 msgr2=0x563c846a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c846a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ede7fc700 1 -- 172.21.15.150:0/3842178900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 msgr2=0x563c846aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c846aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ede7fc700 1 -- 172.21.15.150:0/3842178900 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c84443350 con 0x563c846a3050 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c846aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846a3460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6ed009ab80 tx=0x7f6ed009a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbfe19bc9ed7f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ec57fa700 1 -- 172.21.15.150:0/3842178900 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed009ae80 con 0x563c846a3050 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ec57fa700 1 -- 172.21.15.150:0/3842178900 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed009be90 con 0x563c846a3050 2024-05-10T04:39:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.961+0000 7f6ec57fa700 1 -- 172.21.15.150:0/3842178900 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed009b300 con 0x563c846a3050 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/3842178900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 msgr2=0x563c846a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846a3460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6ed009ab80 tx=0x7f6ed009a860 comp rx=0 tx=0).stop 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/3842178900 shutdown_connections 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c846aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c846a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/3842178900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846a3460 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/3842178900 >> 172.21.15.150:0/3842178900 conn(0x563c84604a00 msgr2=0x563c8469ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/3842178900 shutdown_connections 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/3842178900 wait complete. 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.962+0000 7f6ef56f5b80 1 -- start start 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 1 Processor -- start 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a2180 0x563c846abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a3050 0x563c846ac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a39a0 0x563c846b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8455e500 con 0x563c846a2180 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8455e1a0 con 0x563c846a3050 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c84445640 con 0x563c846a39a0 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ef56f5b80 10 client.0 _ll_register_callbacks cb 0x563c84567ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a2180 0x563c846abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a2180 0x563c846abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55448/0 (socket says 172.21.15.150:55448) 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 -- 172.21.15.150:0/711629961 learned_addr learned my addr 172.21.15.150:0/711629961 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a3050 0x563c846ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a39a0 0x563c846b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 -- 172.21.15.150:0/711629961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a39a0 msgr2=0x563c846b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a39a0 0x563c846b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 -- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a3050 msgr2=0x563c846ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a3050 0x563c846ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6edeffd700 1 -- 172.21.15.150:0/711629961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee01a2540 con 0x563c846a2180 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.963+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a3050 0x563c846ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6edeffd700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a2180 0x563c846abdf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400fff0 tx=0x7f6ed4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638a8f82ad7ef192 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ec2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/711629961 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4004230 con 0x563c846a2180 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/711629961 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ed4004390 con 0x563c846a2180 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c8451b210 con 0x563c846a2180 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/711629961 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ed4004550 con 0x563c846a2180 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 10 client.15741 fetch_fsmap learned FSMap version 29 2024-05-10T04:39:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563c843efb00 con 0x563c846a2180 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/711629961 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c8451b210 con 0x563c846a2180 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/711629961 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ed401f790 con 0x563c846a2180 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 20 client.15741 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c846b7a90 0x563c846b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 4 client.15741 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.964+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c84671720 con 0x563c846b7a90 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.965+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c846b7a90 0x563c846b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.965+0000 7f6ec2ff5700 10 client.15741 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.965+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c846b7a90 0x563c846b9ee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e38a77b9a2283d server_cookie=cf6ccce4a24f8cce in_seq=0 out_seq=0 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/711629961 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c84671720 con 0x563c846b7a90 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ec2ff5700 10 client.15741 handle_command_reply: tid=0 2024-05-10T04:39:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 client.15741 shutdown 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a2180 msgr2=0x563c846abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a2180 0x563c846abdf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400fff0 tx=0x7f6ed4004750 comp rx=0 tx=0).stop 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 shutdown_connections 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c846b7a90 0x563c846b9ee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a39a0 0x563c846b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a2180 0x563c846abdf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ec2ff5700 0 client.15741 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/711629961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a3050 0x563c846ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 >> 172.21.15.150:0/711629961 conn(0x563c84604a00 msgr2=0x563c8469ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 shutdown_connections 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/711629961 wait complete. 2024-05-10T04:39:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.966+0000 7f6ef56f5b80 20 client.15741 trim_cache size 0 max 16384 2024-05-10T04:39:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 Processor -- start 2024-05-10T04:39:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 -- start start 2024-05-10T04:39:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c84698d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c8469ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c846992b0 con 0x563c846a3050 2024-05-10T04:39:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c846bf080 con 0x563c846a39a0 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8469cf80 con 0x563c846a2180 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55450/0 (socket says 172.21.15.150:55450) 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 -- 172.21.15.150:0/830151508 learned_addr learned my addr 172.21.15.150:0/830151508 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6edeffd700 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c84698d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 -- 172.21.15.150:0/830151508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 msgr2=0x563c84698d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c8469ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c84698d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 -- 172.21.15.150:0/830151508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 msgr2=0x563c8469ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c8469ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6ede7fc700 1 -- 172.21.15.150:0/830151508 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c84588500 con 0x563c846a3050 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.981+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c8469ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846b3250 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00978f0 tx=0x7f6ed000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c19d3e256e48573 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ec5ffb700 1 -- 172.21.15.150:0/830151508 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0099220 con 0x563c846a3050 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ec5ffb700 1 -- 172.21.15.150:0/830151508 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed009a370 con 0x563c846a3050 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ec5ffb700 1 -- 172.21.15.150:0/830151508 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed009b340 con 0x563c846a3050 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/830151508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 msgr2=0x563c846b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846b3250 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00978f0 tx=0x7f6ed000ff70 comp rx=0 tx=0).stop 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/830151508 shutdown_connections 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a39a0 0x563c8469ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a2180 0x563c84698d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/830151508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a3050 0x563c846b3250 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/830151508 >> 172.21.15.150:0/830151508 conn(0x563c84604a00 msgr2=0x563c846a6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/830151508 shutdown_connections 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.982+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/830151508 wait complete. 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 -- start start 2024-05-10T04:39:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 Processor -- start 2024-05-10T04:39:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a2180 0x563c846b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a3050 0x563c846b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a39a0 0x563c846b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c846b1fc0 con 0x563c846a39a0 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c846b2130 con 0x563c846a2180 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c846b22f0 con 0x563c846a3050 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6ef56f5b80 10 client.0 _ll_register_callbacks cb 0x563c84567ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6edf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a39a0 0x563c846b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6edf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a39a0 0x563c846b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55458/0 (socket says 172.21.15.150:55458) 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.983+0000 7f6edf7fe700 1 -- 172.21.15.150:0/2494993883 learned_addr learned my addr 172.21.15.150:0/2494993883 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6edeffd700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a2180 0x563c846b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a3050 0x563c846b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6edf7fe700 1 -- 172.21.15.150:0/2494993883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a3050 msgr2=0x563c846b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a3050 0x563c846b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6edf7fe700 1 -- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a2180 msgr2=0x563c846b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a2180 0x563c846b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6edf7fe700 1 -- 172.21.15.150:0/2494993883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed00b2040 con 0x563c846a39a0 2024-05-10T04:39:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ede7fc700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a3050 0x563c846b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6edf7fe700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a39a0 0x563c846b1a80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8011a70 tx=0x7f6ed8011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e22ab22d3c79ef server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ec2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/2494993883 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8004bb0 con 0x563c846a39a0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/2494993883 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ed800c040 con 0x563c846a39a0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ed009be90 con 0x563c846a39a0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/2494993883 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ed8004d10 con 0x563c846a39a0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ef56f5b80 10 client.15747 fetch_fsmap learned FSMap version 29 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563c84447c70 con 0x563c846a39a0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.984+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/2494993883 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed009be90 con 0x563c846a39a0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/2494993883 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ed801b030 con 0x563c846a39a0 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ef56f5b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ef56f5b80 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ef56f5b80 20 client.15747 populate_metadata read hostname 'smithi150' 2024-05-10T04:39:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c8470e4a0 0x563c847108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:39:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ef56f5b80 4 client.15747 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:39:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c845bae60 con 0x563c8470e4a0 2024-05-10T04:39:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6edeffd700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c8470e4a0 0x563c847108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:39:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6ec2ff5700 10 client.15747 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:39:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.985+0000 7f6edeffd700 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c8470e4a0 0x563c847108c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816105eb0c15472f server_cookie=d48c533dbf2c346f in_seq=0 out_seq=0 2024-05-10T04:39:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ec2ff5700 1 -- 172.21.15.150:0/2494993883 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c845bae60 con 0x563c8470e4a0 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ec2ff5700 10 client.15747 handle_command_reply: tid=0 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 client.15747 shutdown 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a39a0 msgr2=0x563c846b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a39a0 0x563c846b1a80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8011a70 tx=0x7f6ed8011d80 comp rx=0 tx=0).stop 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 shutdown_connections 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563c8470e4a0 0x563c847108c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c846a39a0 0x563c846b1a80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563c846a2180 0x563c846b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ec2ff5700 0 client.15747 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 --2- 172.21.15.150:0/2494993883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563c846a3050 0x563c846b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 >> 172.21.15.150:0/2494993883 conn(0x563c84604a00 msgr2=0x563c846a6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 shutdown_connections 2024-05-10T04:40:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 1 -- 172.21.15.150:0/2494993883 wait complete. 2024-05-10T04:40:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.998+0000 7f6ef56f5b80 20 client.15747 trim_cache size 0 max 16384 2024-05-10T04:40:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:40:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:40:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:40:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6ec80a11b0 msgr2=0x7f6ec80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6ec80a11b0 0x7f6ec80a36c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4005950 tx=0x7f6ed40058e0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0101670 msgr2=0x7f6ee019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0101670 0x7f6ee019a700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0003730 tx=0x7f6ed000f720 comp rx=0 tx=0).stop 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 shutdown_connections 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee0102020 0x7f6ee01a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f6ec80a11b0 0x7f6ec80a36c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee0100d20 0x7f6ee019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 --2- 172.21.15.150:0/2700009431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ee0101670 0x7f6ee019a700 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 >> 172.21.15.150:0/2700009431 conn(0x7f6ee00fb8e0 msgr2=0x7f6ee00fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 shutdown_connections 2024-05-10T04:40:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:39:59.999+0000 7f6ee53b1700 1 -- 172.21.15.150:0/2700009431 wait complete. 2024-05-10T04:40:00.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:00 smithi181 ceph-mon[47314]: pgmap v282: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-10T04:40:00.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:00 smithi181 ceph-mon[47314]: overall HEALTH_OK 2024-05-10T04:40:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:00 smithi150 ceph-mon[42404]: pgmap v282: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-10T04:40:00.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:00 smithi150 ceph-mon[42404]: overall HEALTH_OK 2024-05-10T04:40:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:00 smithi161 ceph-mon[46912]: pgmap v282: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-05-10T04:40:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:00 smithi161 ceph-mon[46912]: overall HEALTH_OK 2024-05-10T04:40:02.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:02 smithi150 ceph-mon[42404]: pgmap v283: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-10T04:40:02.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:02 smithi161 ceph-mon[46912]: pgmap v283: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-10T04:40:02.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:02 smithi181 ceph-mon[47314]: pgmap v283: 129 pgs: 129 active+clean; 658 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-05-10T04:40:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:04 smithi150 ceph-mon[42404]: pgmap v284: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-10T04:40:04.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:04 smithi161 ceph-mon[46912]: pgmap v284: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-10T04:40:04.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:04 smithi181 ceph-mon[47314]: pgmap v284: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-10T04:40:06.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:06 smithi150 ceph-mon[42404]: pgmap v285: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-10T04:40:06.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:06 smithi161 ceph-mon[46912]: pgmap v285: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-10T04:40:06.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:06 smithi181 ceph-mon[47314]: pgmap v285: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-05-10T04:40:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:08 smithi181 ceph-mon[47314]: pgmap v286: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-10T04:40:08.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:08 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:40:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:08 smithi150 ceph-mon[42404]: pgmap v286: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-10T04:40:08.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:08 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:40:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:08 smithi161 ceph-mon[46912]: pgmap v286: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-05-10T04:40:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:08 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:40:10.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: pgmap v287: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:40:10.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: pgmap v287: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-05-10T04:40:10.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:40:10.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:40:10.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: pgmap v287: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:40:10.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:40:12.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:12 smithi181 ceph-mon[47314]: pgmap v288: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-10T04:40:12.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-10T04:40:12.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-05-10T04:40:12.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-05-10T04:40:12.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-10T04:40:12.655 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-10T04:40:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:12 smithi150 ceph-mon[42404]: pgmap v288: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-10T04:40:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-10T04:40:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-05-10T04:40:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-05-10T04:40:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-10T04:40:12.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-10T04:40:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:12 smithi161 ceph-mon[46912]: pgmap v288: 129 pgs: 129 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-05-10T04:40:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-10T04:40:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-05-10T04:40:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-05-10T04:40:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-10T04:40:12.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-10T04:40:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 -- 172.21.15.150:0/1212190499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764102580 msgr2=0x7f9764109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 --2- 172.21.15.150:0/1212190499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764102580 0x7f9764109000 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b0a0 tx=0x7f975c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 -- 172.21.15.150:0/1212190499 shutdown_connections 2024-05-10T04:40:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 --2- 172.21.15.150:0/1212190499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764102580 0x7f9764109000 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 --2- 172.21.15.150:0/1212190499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764101040 0x7f9764101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 --2- 172.21.15.150:0/1212190499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9764101a20 0x7f9764101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 -- 172.21.15.150:0/1212190499 >> 172.21.15.150:0/1212190499 conn(0x7f97640fb840 msgr2=0x7f97640fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:13.134 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 -- 172.21.15.150:0/1212190499 shutdown_connections 2024-05-10T04:40:13.134 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.131+0000 7f9768f89700 1 -- 172.21.15.150:0/1212190499 wait complete. 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 Processor -- start 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 -- start start 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9764101040 0x7f9764100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764101a20 0x7f976419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764102580 0x7f976419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976419faa0 con 0x7f9764101a20 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976419fc10 con 0x7f9764101040 2024-05-10T04:40:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.132+0000 7f9768f89700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976419fdd0 con 0x7f9764102580 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764101a20 0x7f976419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764101a20 0x7f976419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55416/0 (socket says 172.21.15.150:55416) 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 -- 172.21.15.150:0/2249365557 learned_addr learned my addr 172.21.15.150:0/2249365557 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9762d9d700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764102580 0x7f976419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f976259c700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9764101040 0x7f9764100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 -- 172.21.15.150:0/2249365557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764102580 msgr2=0x7f976419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764102580 0x7f976419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 -- 172.21.15.150:0/2249365557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9764101040 msgr2=0x7f9764100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9764101040 0x7f9764100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 -- 172.21.15.150:0/2249365557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c009d00 con 0x7f9764101a20 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f976259c700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9764101040 0x7f9764100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764101a20 0x7f976419ce10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f975400d950 tx=0x7f975400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f97537fe700 1 -- 172.21.15.150:0/2249365557 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754009b40 con 0x7f9764101a20 2024-05-10T04:40:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97641a00b0 con 0x7f9764101a20 2024-05-10T04:40:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f97537fe700 1 -- 172.21.15.150:0/2249365557 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9754009ca0 con 0x7f9764101a20 2024-05-10T04:40:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f97537fe700 1 -- 172.21.15.150:0/2249365557 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400f620 con 0x7f9764101a20 2024-05-10T04:40:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.133+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97641a0610 con 0x7f9764101a20 2024-05-10T04:40:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.136+0000 7f97537fe700 1 -- 172.21.15.150:0/2249365557 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f975400f780 con 0x7f9764101a20 2024-05-10T04:40:13.146 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.136+0000 7f97537fe700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f974c0a11b0 0x7f974c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.136+0000 7f97537fe700 1 -- 172.21.15.150:0/2249365557 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f97540cd1a0 con 0x7f9764101a20 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.136+0000 7f976259c700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f974c0a11b0 0x7f974c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f976259c700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f974c0a11b0 0x7f974c0a36c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9758005fd0 tx=0x7f9758005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 Processor -- start 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 -- start start 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e8680 0x5569010e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e9090 0x5569010e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690100a0f0 con 0x5569010e7980 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556901007ab0 con 0x5569010e8680 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.137+0000 7f97792cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569010033a0 con 0x5569010e9090 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ef6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55420/0 (socket says 172.21.15.150:55420) 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 -- 172.21.15.150:0/3878277882 learned_addr learned my addr 172.21.15.150:0/3878277882 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f976259c700 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e8680 0x5569010e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9761d9b700 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e9090 0x5569010e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 -- 172.21.15.150:0/3878277882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e9090 msgr2=0x5569010e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e9090 0x5569010e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 -- 172.21.15.150:0/3878277882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e8680 msgr2=0x5569010e8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e8680 0x5569010e8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 -- 172.21.15.150:0/3878277882 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556900e84350 con 0x5569010e7980 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f976259c700 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e8680 0x5569010e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9762d9d700 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ef6e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b0a0 tx=0x7f975c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15f037a444228a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9750ff9700 1 -- 172.21.15.150:0/3878277882 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00e050 con 0x5569010e7980 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9750ff9700 1 -- 172.21.15.150:0/3878277882 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f975c004a50 con 0x5569010e7980 2024-05-10T04:40:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.138+0000 7f9750ff9700 1 -- 172.21.15.150:0/3878277882 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c005550 con 0x5569010e7980 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 -- 172.21.15.150:0/3878277882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 msgr2=0x5569010ef6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ef6e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b0a0 tx=0x7f975c009800 comp rx=0 tx=0).stop 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 -- 172.21.15.150:0/3878277882 shutdown_connections 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ef6e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e8680 0x5569010e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 --2- 172.21.15.150:0/3878277882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e9090 0x5569010e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 -- 172.21.15.150:0/3878277882 >> 172.21.15.150:0/3878277882 conn(0x556901045a00 msgr2=0x5569010dda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 -- 172.21.15.150:0/3878277882 shutdown_connections 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.139+0000 7f97792cdb80 1 -- 172.21.15.150:0/3878277882 wait complete. 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 -- start start 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 Processor -- start 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556900f9f500 con 0x5569010e8680 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556900f9f1a0 con 0x5569010e9090 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556900e86640 con 0x5569010e7980 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f97792cdb80 10 client.0 _ll_register_callbacks cb 0x556900fa8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55428/0 (socket says 172.21.15.150:55428) 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f9761d9b700 1 -- 172.21.15.150:0/2794399792 learned_addr learned my addr 172.21.15.150:0/2794399792 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f976259c700 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.140+0000 7f9762d9d700 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9761d9b700 1 -- 172.21.15.150:0/2794399792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 msgr2=0x5569010f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9761d9b700 1 -- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 msgr2=0x5569010f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9761d9b700 1 -- 172.21.15.150:0/2794399792 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97641098c0 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f1b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f97540965f0 tx=0x7f975409a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970f89ef1e159294 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9749ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9749ffb700 1 -- 172.21.15.150:0/2794399792 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754010660 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9749ffb700 1 -- 172.21.15.150:0/2794399792 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f975409df20 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556900f5c210 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9749ffb700 1 -- 172.21.15.150:0/2794399792 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97540a2570 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 10 client.15759 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556900e30b00 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9749ffb700 1 -- 172.21.15.150:0/2794399792 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556900f5c210 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f9749ffb700 1 -- 172.21.15.150:0/2794399792 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f97540a20e0 con 0x5569010e8680 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 20 client.15759 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569010fd290 0x5569010ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 4 client.15759 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.141+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569010b2720 con 0x5569010fd290 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.142+0000 7f976259c700 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569010fd290 0x5569010ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.142+0000 7f9749ffb700 10 client.15759 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.142+0000 7f976259c700 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569010fd290 0x5569010ff6e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=189887dfa99f41a2 server_cookie=3862d5e0e004b32d in_seq=0 out_seq=0 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f9749ffb700 1 -- 172.21.15.150:0/2794399792 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569010b2720 con 0x5569010fd290 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f9749ffb700 10 client.15759 handle_command_reply: tid=0 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 client.15759 shutdown 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 msgr2=0x5569010f1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f1b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f97540965f0 tx=0x7f975409a870 comp rx=0 tx=0).stop 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 shutdown_connections 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569010fd290 0x5569010ff6e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f9749ffb700 0 client.15759 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2794399792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f1b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 >> 172.21.15.150:0/2794399792 conn(0x556901045a00 msgr2=0x5569010dda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 shutdown_connections 2024-05-10T04:40:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 1 -- 172.21.15.150:0/2794399792 wait complete. 2024-05-10T04:40:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.143+0000 7f97792cdb80 20 client.15759 trim_cache size 0 max 16384 2024-05-10T04:40:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 Processor -- start 2024-05-10T04:40:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 -- start start 2024-05-10T04:40:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e8680 0x5569010ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569010e7dc0 con 0x5569010e7980 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569010dec60 con 0x5569010e9090 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f97792cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569010ef690 con 0x5569010e8680 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.158+0000 7f976259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55438/0 (socket says 172.21.15.150:55438) 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 -- 172.21.15.150:0/1855931158 learned_addr learned my addr 172.21.15.150:0/1855931158 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f9761d9b700 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e8680 0x5569010ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f9762d9d700 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 -- 172.21.15.150:0/1855931158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e8680 msgr2=0x5569010ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e8680 0x5569010ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 -- 172.21.15.150:0/1855931158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 msgr2=0x5569010f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 -- 172.21.15.150:0/1855931158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556900fc9500 con 0x5569010e7980 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f9761d9b700 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e8680 0x5569010ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f9762d9d700 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f976259c700 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ec330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9764068390 tx=0x7f9758013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2b3581ece2d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f97517fa700 1 -- 172.21.15.150:0/1855931158 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758023040 con 0x5569010e7980 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f97517fa700 1 -- 172.21.15.150:0/1855931158 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f975801d430 con 0x5569010e7980 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.159+0000 7f97517fa700 1 -- 172.21.15.150:0/1855931158 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800f680 con 0x5569010e7980 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 -- 172.21.15.150:0/1855931158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 msgr2=0x5569010ec330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ec330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9764068390 tx=0x7f9758013000 comp rx=0 tx=0).stop 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 -- 172.21.15.150:0/1855931158 shutdown_connections 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e7980 0x5569010ec330 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 --2- 172.21.15.150:0/1855931158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e8680 0x5569010ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 -- 172.21.15.150:0/1855931158 >> 172.21.15.150:0/1855931158 conn(0x556901045a00 msgr2=0x5569010f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 -- 172.21.15.150:0/1855931158 shutdown_connections 2024-05-10T04:40:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 -- 172.21.15.150:0/1855931158 wait complete. 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 -- start start 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 Processor -- start 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.160+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569010f8540 con 0x5569010e8680 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569010f86b0 con 0x5569010e9090 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569010f8870 con 0x5569010e7980 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 10 client.0 _ll_register_callbacks cb 0x556900fa8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55444/0 (socket says 172.21.15.150:55444) 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 -- 172.21.15.150:0/2908781103 learned_addr learned my addr 172.21.15.150:0/2908781103 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9762d9d700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f976259c700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 -- 172.21.15.150:0/2908781103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 msgr2=0x5569010f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 -- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 msgr2=0x5569010f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 -- 172.21.15.150:0/2908781103 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975800dbe0 con 0x5569010e8680 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f976259c700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9761d9b700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f7120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9754009520 tx=0x7f9754010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e04b26854dd673 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9749ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9749ffb700 1 -- 172.21.15.150:0/2908781103 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97540995e0 con 0x5569010e8680 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9749ffb700 1 -- 172.21.15.150:0/2908781103 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f975409df20 con 0x5569010e8680 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f975c004a50 con 0x5569010e8680 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f9749ffb700 1 -- 172.21.15.150:0/2908781103 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9754099740 con 0x5569010e8680 2024-05-10T04:40:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 10 client.15765 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.161+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556900e88c70 con 0x5569010e8680 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f9749ffb700 1 -- 172.21.15.150:0/2908781103 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f975c004a50 con 0x5569010e8680 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f9749ffb700 1 -- 172.21.15.150:0/2908781103 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f97540b81f0 con 0x5569010e8680 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f97792cdb80 10 client.15765 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f97792cdb80 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f97792cdb80 20 client.15765 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569011544b0 0x5569011568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f97792cdb80 4 client.15765 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556900ffbe60 con 0x5569011544b0 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f976259c700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569011544b0 0x5569011568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f9749ffb700 10 client.15765 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.162+0000 7f976259c700 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569011544b0 0x5569011568d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a77b4d8e825ae87 server_cookie=c2a42e67c737b88f in_seq=0 out_seq=0 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f9749ffb700 1 -- 172.21.15.150:0/2908781103 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556900ffbe60 con 0x5569011544b0 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f9749ffb700 10 client.15765 handle_command_reply: tid=0 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 client.15765 shutdown 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 msgr2=0x5569010f7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f7120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9754009520 tx=0x7f9754010690 comp rx=0 tx=0).stop 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 shutdown_connections 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5569011544b0 0x5569011568d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5569010e9090 0x5569010f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f9749ffb700 0 client.15765 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5569010e7980 0x5569010f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 --2- 172.21.15.150:0/2908781103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5569010e8680 0x5569010f7120 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 >> 172.21.15.150:0/2908781103 conn(0x556901045a00 msgr2=0x5569010ef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 shutdown_connections 2024-05-10T04:40:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 1 -- 172.21.15.150:0/2908781103 wait complete. 2024-05-10T04:40:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.166+0000 7f97792cdb80 20 client.15765 trim_cache size 0 max 16384 2024-05-10T04:40:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:40:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:40:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:40:13.172 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f974c0a11b0 msgr2=0x7f974c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f974c0a11b0 0x7f974c0a36c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9758005fd0 tx=0x7f9758005dc0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764101a20 msgr2=0x7f976419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764101a20 0x7f976419ce10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f975400d950 tx=0x7f975400dd10 comp rx=0 tx=0).stop 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 shutdown_connections 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f974c0a11b0 0x7f974c0a36c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764102580 0x7f976419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9764101040 0x7f9764100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 --2- 172.21.15.150:0/2249365557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9764101a20 0x7f976419ce10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 >> 172.21.15.150:0/2249365557 conn(0x7f97640fb840 msgr2=0x7f9764104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 shutdown_connections 2024-05-10T04:40:13.174 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:13.167+0000 7f9768f89700 1 -- 172.21.15.150:0/2249365557 wait complete. 2024-05-10T04:40:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-10T04:40:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-05-10T04:40:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-05-10T04:40:13.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-10T04:40:13.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:13 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-10T04:40:13.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:13 smithi150 ceph-mon[42404]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T04:40:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-10T04:40:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-05-10T04:40:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-05-10T04:40:13.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-10T04:40:13.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:13 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-10T04:40:13.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:13 smithi161 ceph-mon[46912]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T04:40:13.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-10T04:40:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-05-10T04:40:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-05-10T04:40:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-10T04:40:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:13 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-10T04:40:13.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:13 smithi181 ceph-mon[47314]: osdmap e89: 12 total, 12 up, 12 in 2024-05-10T04:40:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:14 smithi150 ceph-mon[42404]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 11 op/s 2024-05-10T04:40:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:14 smithi150 ceph-mon[42404]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-10T04:40:14.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:14 smithi150 ceph-mon[42404]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T04:40:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:14 smithi161 ceph-mon[46912]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 11 op/s 2024-05-10T04:40:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:14 smithi161 ceph-mon[46912]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-10T04:40:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:14 smithi161 ceph-mon[46912]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T04:40:14.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:14 smithi181 ceph-mon[47314]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 11 op/s 2024-05-10T04:40:14.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:14 smithi181 ceph-mon[47314]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-10T04:40:14.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:14 smithi181 ceph-mon[47314]: osdmap e90: 12 total, 12 up, 12 in 2024-05-10T04:40:16.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:16 smithi161 ceph-mon[46912]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-05-10T04:40:16.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:16 smithi181 ceph-mon[47314]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-05-10T04:40:16.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:16 smithi150 ceph-mon[42404]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 op/s 2024-05-10T04:40:17.199 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:40:17.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 -- 172.21.15.150:0/4084200163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c0ffc20 msgr2=0x7fb92c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 --2- 172.21.15.150:0/4084200163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c0ffc20 0x7fb92c100030 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009990 tx=0x7fb91c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 -- 172.21.15.150:0/4084200163 shutdown_connections 2024-05-10T04:40:17.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 --2- 172.21.15.150:0/4084200163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb92c100570 0x7fb92c10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 --2- 172.21.15.150:0/4084200163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92c1031e0 0x7fb92c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 --2- 172.21.15.150:0/4084200163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c0ffc20 0x7fb92c100030 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.353 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 -- 172.21.15.150:0/4084200163 >> 172.21.15.150:0/4084200163 conn(0x7fb92c0fb8f0 msgr2=0x7fb92c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 -- 172.21.15.150:0/4084200163 shutdown_connections 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.349+0000 7fb932719700 1 -- 172.21.15.150:0/4084200163 wait complete. 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 Processor -- start 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 -- start start 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb92c0ffc20 0x7fb92c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c100570 0x7fb92c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92c1031e0 0x7fb92c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c19f9a0 con 0x7fb92c100570 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c19fb10 con 0x7fb92c0ffc20 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb932719700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c19fcd0 con 0x7fb92c1031e0 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c100570 0x7fb92c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c100570 0x7fb92c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55298/0 (socket says 172.21.15.150:55298) 2024-05-10T04:40:17.354 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1634157061 learned_addr learned my addr 172.21.15.150:0/1634157061 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.350+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb92c0ffc20 0x7fb92c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb930cb6700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92c1031e0 0x7fb92c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1634157061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92c1031e0 msgr2=0x7fb92c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92c1031e0 0x7fb92c19f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1634157061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb92c0ffc20 msgr2=0x7fb92c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb92c0ffc20 0x7fb92c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1634157061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c0095f0 con 0x7fb92c100570 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb92c0ffc20 0x7fb92c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c100570 0x7fb92c198400 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0095c0 tx=0x7fb91c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb9297fa700 1 -- 172.21.15.150:0/1634157061 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c01d070 con 0x7fb92c100570 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb9297fa700 1 -- 172.21.15.150:0/1634157061 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb91c022470 con 0x7fb92c100570 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92c19ff50 con 0x7fb92c100570 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb9297fa700 1 -- 172.21.15.150:0/1634157061 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c017650 con 0x7fb92c100570 2024-05-10T04:40:17.355 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.351+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c1a0310 con 0x7fb92c100570 2024-05-10T04:40:17.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.353+0000 7fb9297fa700 1 -- 172.21.15.150:0/1634157061 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fb91c0177b0 con 0x7fb92c100570 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.354+0000 7fb9297fa700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb9140a12e0 0x7fb9140a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.354+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb9140a12e0 0x7fb9140a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.354+0000 7fb9297fa700 1 -- 172.21.15.150:0/1634157061 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb91c0ce0f0 con 0x7fb92c100570 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.354+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb9140a12e0 0x7fb9140a37f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb920005eb0 tx=0x7fb920005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 Processor -- start 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 -- start start 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f87ef150 con 0x55e4f88c9570 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f87ecb10 con 0x55e4f88c9ec0 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb942a5db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f87e8400 con 0x55e4f88c86a0 2024-05-10T04:40:17.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55310/0 (socket says 172.21.15.150:55310) 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1523569052 learned_addr learned my addr 172.21.15.150:0/1523569052 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb930cb6700 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1523569052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 msgr2=0x55e4f88c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1523569052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 msgr2=0x55e4f88d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.355+0000 7fb92b7fe700 1 -- 172.21.15.150:0/1523569052 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f86693b0 con 0x55e4f88c9570 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.356+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88c9980 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0a3020 tx=0x7fb91c09abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bf06d0f3b7a3a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.356+0000 7fb9127fc700 1 -- 172.21.15.150:0/1523569052 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0a3550 con 0x55e4f88c9570 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.356+0000 7fb9127fc700 1 -- 172.21.15.150:0/1523569052 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb91c0cb950 con 0x55e4f88c9570 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.356+0000 7fb9127fc700 1 -- 172.21.15.150:0/1523569052 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c022b20 con 0x55e4f88c9570 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.356+0000 7fb942a5db80 1 -- 172.21.15.150:0/1523569052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 msgr2=0x55e4f88c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.356+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88c9980 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0a3020 tx=0x7fb91c09abf0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 -- 172.21.15.150:0/1523569052 shutdown_connections 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1523569052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88c9980 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 -- 172.21.15.150:0/1523569052 >> 172.21.15.150:0/1523569052 conn(0x55e4f882aa60 msgr2=0x55e4f88c2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 -- 172.21.15.150:0/1523569052 shutdown_connections 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 -- 172.21.15.150:0/1523569052 wait complete. 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 -- start start 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 Processor -- start 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.357+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb942a5db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f8784560 con 0x55e4f88c86a0 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb942a5db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f8784200 con 0x55e4f88c9ec0 2024-05-10T04:40:17.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb942a5db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f866b6a0 con 0x55e4f88c9570 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb942a5db80 10 client.0 _ll_register_callbacks cb 0x55e4f878db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88d23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55312/0 (socket says 172.21.15.150:55312) 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 -- 172.21.15.150:0/238763226 learned_addr learned my addr 172.21.15.150:0/238763226 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb930cb6700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 -- 172.21.15.150:0/238763226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 msgr2=0x55e4f88d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 -- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 msgr2=0x55e4f88d9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 -- 172.21.15.150:0/238763226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c1a0310 con 0x55e4f88c86a0 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb930cb6700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:17.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb92bfff700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88d23b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb92000ff60 tx=0x7fb920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f094ab019e7f7f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb90b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb90b7fe700 1 -- 172.21.15.150:0/238763226 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92001f430 con 0x55e4f88c86a0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb90b7fe700 1 -- 172.21.15.150:0/238763226 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb92001f590 con 0x55e4f88c86a0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.358+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4f8741270 con 0x55e4f88c86a0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb90b7fe700 1 -- 172.21.15.150:0/238763226 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92001f750 con 0x55e4f88c86a0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 10 client.15777 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e4f8615b60 con 0x55e4f88c86a0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb90b7fe700 1 -- 172.21.15.150:0/238763226 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4f8741270 con 0x55e4f88c86a0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb90b7fe700 1 -- 172.21.15.150:0/238763226 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb920025020 con 0x55e4f88c86a0 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 20 client.15777 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f88e0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.367 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 4 client.15777 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4f8897780 con 0x55e4f88de0d0 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.359+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f88e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.360+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f88e0520 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4481fbb251602452 server_cookie=befe305648390d9b in_seq=0 out_seq=0 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.360+0000 7fb90b7fe700 10 client.15777 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.360+0000 7fb90b7fe700 1 -- 172.21.15.150:0/238763226 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4f8897780 con 0x55e4f88de0d0 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.360+0000 7fb90b7fe700 10 client.15777 handle_command_reply: tid=0 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.360+0000 7fb942a5db80 1 client.15777 shutdown 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.360+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 msgr2=0x55e4f88d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.360+0000 7fb942a5db80 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88d23b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb92000ff60 tx=0x7fb920004750 comp rx=0 tx=0).stop 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 shutdown_connections 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f88e0520 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88d23b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb90b7fe700 0 client.15777 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 --2- 172.21.15.150:0/238763226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 >> 172.21.15.150:0/238763226 conn(0x55e4f882aa60 msgr2=0x55e4f88c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 shutdown_connections 2024-05-10T04:40:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 1 -- 172.21.15.150:0/238763226 wait complete. 2024-05-10T04:40:17.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.361+0000 7fb942a5db80 20 client.15777 trim_cache size 0 max 16384 2024-05-10T04:40:17.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 Processor -- start 2024-05-10T04:40:17.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 -- start start 2024-05-10T04:40:17.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f88c9320 con 0x55e4f88c86a0 2024-05-10T04:40:17.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f88d3200 con 0x55e4f88c9ec0 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb942a5db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f88cdcd0 con 0x55e4f88c9570 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55314/0 (socket says 172.21.15.150:55314) 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 -- 172.21.15.150:0/1142523427 learned_addr learned my addr 172.21.15.150:0/1142523427 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb930cb6700 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 -- 172.21.15.150:0/1142523427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 msgr2=0x55e4f88d9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 -- 172.21.15.150:0/1142523427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 msgr2=0x55e4f88cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.378+0000 7fb92bfff700 1 -- 172.21.15.150:0/1142523427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f87ae560 con 0x55e4f88c86a0 2024-05-10T04:40:17.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb92bfff700 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88cd250 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb920010aa0 tx=0x7fb920010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e48d2d40528fda server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb912ffd700 1 -- 172.21.15.150:0/1142523427 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920010d10 con 0x55e4f88c86a0 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb912ffd700 1 -- 172.21.15.150:0/1142523427 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb920010890 con 0x55e4f88c86a0 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb912ffd700 1 -- 172.21.15.150:0/1142523427 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920035e40 con 0x55e4f88c86a0 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 -- 172.21.15.150:0/1142523427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 msgr2=0x55e4f88cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88cd250 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb920010aa0 tx=0x7fb920010c40 comp rx=0 tx=0).stop 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 -- 172.21.15.150:0/1142523427 shutdown_connections 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c86a0 0x55e4f88cd250 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 --2- 172.21.15.150:0/1142523427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c9570 0x55e4f88d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 -- 172.21.15.150:0/1142523427 >> 172.21.15.150:0/1142523427 conn(0x55e4f882aa60 msgr2=0x55e4f88c3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 -- 172.21.15.150:0/1142523427 shutdown_connections 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.379+0000 7fb942a5db80 1 -- 172.21.15.150:0/1142523427 wait complete. 2024-05-10T04:40:17.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 -- start start 2024-05-10T04:40:17.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 Processor -- start 2024-05-10T04:40:17.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f88d8a00 con 0x55e4f88c9570 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f88d8b70 con 0x55e4f88c9ec0 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f88d8d30 con 0x55e4f88c86a0 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb942a5db80 10 client.0 _ll_register_callbacks cb 0x55e4f878db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88d75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55322/0 (socket says 172.21.15.150:55322) 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.380+0000 7fb92b7fe700 1 -- 172.21.15.150:0/907811922 learned_addr learned my addr 172.21.15.150:0/907811922 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb930cb6700 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb92bfff700 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb92b7fe700 1 -- 172.21.15.150:0/907811922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 msgr2=0x55e4f88d70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88d70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb92b7fe700 1 -- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 msgr2=0x55e4f88d84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb92b7fe700 1 -- 172.21.15.150:0/907811922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4f8615b60 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb92b7fe700 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88d75e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0cb810 tx=0x7fb91c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da06945a63ea6df server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb90b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb90b7fe700 1 -- 172.21.15.150:0/907811922 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0a10b0 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb90b7fe700 1 -- 172.21.15.150:0/907811922 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb91c0a1210 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb91c0cb950 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb90b7fe700 1 -- 172.21.15.150:0/907811922 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb91c022900 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb942a5db80 10 client.15783 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e4f866dcd0 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb90b7fe700 1 -- 172.21.15.150:0/907811922 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb91c0b7020 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.381+0000 7fb90b7fe700 1 -- 172.21.15.150:0/907811922 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb91c0a30c0 con 0x55e4f88c9570 2024-05-10T04:40:17.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb942a5db80 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb942a5db80 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb942a5db80 20 client.15783 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb942a5db80 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f8936a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb942a5db80 4 client.15783 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e4f87e0ec0 con 0x55e4f88de0d0 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb92bfff700 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f8936a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb90b7fe700 10 client.15783 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb92bfff700 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f8936a60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac62b39a5699712 server_cookie=7ef6004a1543ff83 in_seq=0 out_seq=0 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb90b7fe700 1 -- 172.21.15.150:0/907811922 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e4f87e0ec0 con 0x55e4f88de0d0 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb90b7fe700 10 client.15783 handle_command_reply: tid=0 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.382+0000 7fb942a5db80 1 client.15783 shutdown 2024-05-10T04:40:17.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 msgr2=0x55e4f88d75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88d75e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0cb810 tx=0x7fb91c009290 comp rx=0 tx=0).stop 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 shutdown_connections 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55e4f88de0d0 0x55e4f8936a60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4f88c9ec0 0x55e4f88d84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb90b7fe700 0 client.15783 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4f88c86a0 0x55e4f88d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 --2- 172.21.15.150:0/907811922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4f88c9570 0x55e4f88d75e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 >> 172.21.15.150:0/907811922 conn(0x55e4f882aa60 msgr2=0x55e4f88cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 shutdown_connections 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 1 -- 172.21.15.150:0/907811922 wait complete. 2024-05-10T04:40:17.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb942a5db80 20 client.15783 trim_cache size 0 max 16384 2024-05-10T04:40:17.392 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:40:17.392 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:40:17.392 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:40:17.392 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb9140a12e0 msgr2=0x7fb9140a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb9140a12e0 0x7fb9140a37f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb920005eb0 tx=0x7fb920005e40 comp rx=0 tx=0).stop 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c100570 msgr2=0x7fb92c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c100570 0x7fb92c198400 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0095c0 tx=0x7fb91c00f690 comp rx=0 tx=0).stop 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 shutdown_connections 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb9140a12e0 0x7fb9140a37f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb92c1031e0 0x7fb92c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb92c0ffc20 0x7fb92c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 --2- 172.21.15.150:0/1634157061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c100570 0x7fb92c198400 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 >> 172.21.15.150:0/1634157061 conn(0x7fb92c0fb8f0 msgr2=0x7fb92c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 shutdown_connections 2024-05-10T04:40:17.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.383+0000 7fb932719700 1 -- 172.21.15.150:0/1634157061 wait complete. 2024-05-10T04:40:17.397 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:40:17.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:40:17.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.529+0000 7f796e138700 1 -- 172.21.15.150:0/2689266697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968101be0 msgr2=0x7f7968101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.529+0000 7f796e138700 1 --2- 172.21.15.150:0/2689266697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968101be0 0x7f7968101ff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f795c009990 tx=0x7f795c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 -- 172.21.15.150:0/2689266697 shutdown_connections 2024-05-10T04:40:17.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 --2- 172.21.15.150:0/2689266697 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7968102580 0x7f7968108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 --2- 172.21.15.150:0/2689266697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7968100ac0 0x7f7968100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.534 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 --2- 172.21.15.150:0/2689266697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968101be0 0x7f7968101ff0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 -- 172.21.15.150:0/2689266697 >> 172.21.15.150:0/2689266697 conn(0x7f79680fc0d0 msgr2=0x7f79680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 -- 172.21.15.150:0/2689266697 shutdown_connections 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 -- 172.21.15.150:0/2689266697 wait complete. 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 Processor -- start 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.530+0000 7f796e138700 1 -- start start 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f796e138700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7968100ac0 0x7f7968068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f796e138700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7968101be0 0x7f796819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f796e138700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968102580 0x7f796819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f796e138700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796819fa80 con 0x7f7968102580 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f796e138700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796819fbf0 con 0x7f7968100ac0 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f796e138700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796819fdb0 con 0x7f7968101be0 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968102580 0x7f796819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968102580 0x7f796819f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55330/0 (socket says 172.21.15.150:55330) 2024-05-10T04:40:17.535 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 -- 172.21.15.150:0/1023454815 learned_addr learned my addr 172.21.15.150:0/1023454815 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7966ffd700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7968101be0 0x7f796819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f79677fe700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7968100ac0 0x7f7968068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 -- 172.21.15.150:0/1023454815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7968101be0 msgr2=0x7f796819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7968101be0 0x7f796819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 -- 172.21.15.150:0/1023454815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7968100ac0 msgr2=0x7f7968068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7968100ac0 0x7f7968068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 -- 172.21.15.150:0/1023454815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c0095f0 con 0x7f7968102580 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7967fff700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968102580 0x7f796819f330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f796000ef30 tx=0x7f796000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.531+0000 7f7964ff9700 1 -- 172.21.15.150:0/1023454815 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960018070 con 0x7f7968102580 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.532+0000 7f7964ff9700 1 -- 172.21.15.150:0/1023454815 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f796000f040 con 0x7f7968102580 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.532+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796819ffd0 con 0x7f7968102580 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.532+0000 7f7964ff9700 1 -- 172.21.15.150:0/1023454815 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000cd00 con 0x7f7968102580 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.532+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79681a03f0 con 0x7f7968102580 2024-05-10T04:40:17.536 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.533+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968107cd0 con 0x7f7968102580 2024-05-10T04:40:17.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.534+0000 7f7964ff9700 1 -- 172.21.15.150:0/1023454815 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f7960003a00 con 0x7f7968102580 2024-05-10T04:40:17.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.535+0000 7f7964ff9700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f79500a11b0 0x7f79500a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.535+0000 7f79677fe700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f79500a11b0 0x7f79500a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.535+0000 7f7964ff9700 1 -- 172.21.15.150:0/1023454815 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f79600cd290 con 0x7f7968102580 2024-05-10T04:40:17.537 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.535+0000 7f79677fe700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f79500a11b0 0x7f79500a36c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7968101ae0 tx=0x7f7958009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.538+0000 7f7964ff9700 1 -- 172.21.15.150:0/1023454815 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f79600965b0 con 0x7f7968102580 2024-05-10T04:40:17.781 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.780+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79680694d0 con 0x7f7968102580 2024-05-10T04:40:17.782 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.780+0000 7f7964ff9700 1 -- 172.21.15.150:0/1023454815 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f7960099bd0 con 0x7f7968102580 2024-05-10T04:40:17.782 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:40:17.782 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:40:17.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.782+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f79500a11b0 msgr2=0x7f79500a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f79500a11b0 0x7f79500a36c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7968101ae0 tx=0x7f7958009260 comp rx=0 tx=0).stop 2024-05-10T04:40:17.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968102580 msgr2=0x7f796819f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968102580 0x7f796819f330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f796000ef30 tx=0x7f796000c3a0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 shutdown_connections 2024-05-10T04:40:17.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968102580 0x7f796819f330 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7968100ac0 0x7f7968068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f79500a11b0 0x7f79500a36c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 --2- 172.21.15.150:0/1023454815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7968101be0 0x7f796819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 >> 172.21.15.150:0/1023454815 conn(0x7f79680fc0d0 msgr2=0x7f7968104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 shutdown_connections 2024-05-10T04:40:17.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.783+0000 7f796e138700 1 -- 172.21.15.150:0/1023454815 wait complete. 2024-05-10T04:40:17.785 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:40:17.797 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T04:40:17.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 -- 172.21.15.150:0/24833914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 msgr2=0x7fce68109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 --2- 172.21.15.150:0/24833914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce68109050 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce6000b0a0 tx=0x7fce6000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 -- 172.21.15.150:0/24833914 shutdown_connections 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 --2- 172.21.15.150:0/24833914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce68109050 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 --2- 172.21.15.150:0/24833914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce68101090 0x7fce681014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 --2- 172.21.15.150:0/24833914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce68101a70 0x7fce68101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 -- 172.21.15.150:0/24833914 >> 172.21.15.150:0/24833914 conn(0x7fce680fb8b0 msgr2=0x7fce680fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 -- 172.21.15.150:0/24833914 shutdown_connections 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 -- 172.21.15.150:0/24833914 wait complete. 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.931+0000 7fce6d434700 1 Processor -- start 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce6d434700 1 -- start start 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce6d434700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce68101090 0x7fce68196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce6d434700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce68101a70 0x7fce68197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce6d434700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce6819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce6d434700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6819e9e0 con 0x7fce68102590 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce6d434700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6819eb50 con 0x7fce68101090 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce6d434700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6819ed10 con 0x7fce68101a70 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce6819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce6819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55346/0 (socket says 172.21.15.150:55346) 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 -- 172.21.15.150:0/479198439 learned_addr learned my addr 172.21.15.150:0/479198439 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce66ffd700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce68101090 0x7fce68196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce667fc700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce68101a70 0x7fce68197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 -- 172.21.15.150:0/479198439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce68101a70 msgr2=0x7fce68197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce68101a70 0x7fce68197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 -- 172.21.15.150:0/479198439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce68101090 msgr2=0x7fce68196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce68101090 0x7fce68196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 -- 172.21.15.150:0/479198439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60009d00 con 0x7fce68102590 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce66ffd700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce68101090 0x7fce68196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce667fc700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce68101a70 0x7fce68197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.932+0000 7fce677fe700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce6819e3d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce60006070 tx=0x7fce60004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.933+0000 7fce4ffff700 1 -- 172.21.15.150:0/479198439 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6000e040 con 0x7fce68102590 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.933+0000 7fce4ffff700 1 -- 172.21.15.150:0/479198439 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6000cec0 con 0x7fce68102590 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.933+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6819ef90 con 0x7fce68102590 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.933+0000 7fce4ffff700 1 -- 172.21.15.150:0/479198439 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60004020 con 0x7fce68102590 2024-05-10T04:40:17.937 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.933+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6819f360 con 0x7fce68102590 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.935+0000 7fce4ffff700 1 -- 172.21.15.150:0/479198439 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fce60004180 con 0x7fce68102590 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.935+0000 7fce4ffff700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fce500a1260 0x7fce500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.935+0000 7fce4ffff700 1 -- 172.21.15.150:0/479198439 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fce600ce680 con 0x7fce68102590 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.936+0000 7fce66ffd700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fce500a1260 0x7fce500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.936+0000 7fce66ffd700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fce500a1260 0x7fce500a3770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fce58005950 tx=0x7fce580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.936+0000 7fce7d778b80 1 Processor -- start 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.936+0000 7fce7d778b80 1 -- start start 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c55af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c56980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.944 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce7d778b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3b7c150 con 0x5598e3c56570 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce7d778b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3b79b10 con 0x5598e3c56ec0 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce7d778b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3b75400 con 0x5598e3c556a0 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c56980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c56980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55352/0 (socket says 172.21.15.150:55352) 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 -- 172.21.15.150:0/3994645700 learned_addr learned my addr 172.21.15.150:0/3994645700 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce677fe700 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce66ffd700 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c55af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 -- 172.21.15.150:0/3994645700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 msgr2=0x5598e3c55af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c55af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 -- 172.21.15.150:0/3994645700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 msgr2=0x5598e3c5d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.945 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 -- 172.21.15.150:0/3994645700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598e39f63b0 con 0x5598e3c56570 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce667fc700 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c56980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bd00 tx=0x7fce5c00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec78109dcb392ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce4d7fa700 1 -- 172.21.15.150:0/3994645700 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c00fda0 con 0x5598e3c56570 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce4d7fa700 1 -- 172.21.15.150:0/3994645700 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce5c00be50 con 0x5598e3c56570 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.937+0000 7fce4d7fa700 1 -- 172.21.15.150:0/3994645700 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c011800 con 0x5598e3c56570 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 -- 172.21.15.150:0/3994645700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 msgr2=0x5598e3c56980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c56980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bd00 tx=0x7fce5c00f640 comp rx=0 tx=0).stop 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 -- 172.21.15.150:0/3994645700 shutdown_connections 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c55af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3994645700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c56980 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 -- 172.21.15.150:0/3994645700 >> 172.21.15.150:0/3994645700 conn(0x5598e3bb7a60 msgr2=0x5598e3c4ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 -- 172.21.15.150:0/3994645700 shutdown_connections 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.938+0000 7fce7d778b80 1 -- 172.21.15.150:0/3994645700 wait complete. 2024-05-10T04:40:17.946 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 -- start start 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 Processor -- start 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c556a0 0x5598e3c5f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56570 0x5598e3c5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c66ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3b11560 con 0x5598e3c556a0 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3b11200 con 0x5598e3c56570 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e39f86a0 con 0x5598e3c56ec0 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce7d778b80 10 client.0 _ll_register_callbacks cb 0x5598e3b1ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c556a0 0x5598e3c5f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c556a0 0x5598e3c5f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55360/0 (socket says 172.21.15.150:55360) 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 -- 172.21.15.150:0/1915223633 learned_addr learned my addr 172.21.15.150:0/1915223633 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce677fe700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c66ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce667fc700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56570 0x5598e3c5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 -- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 msgr2=0x5598e3c66ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.947 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c66ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 -- 172.21.15.150:0/1915223633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56570 msgr2=0x5598e3c5fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56570 0x5598e3c5fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce66ffd700 1 -- 172.21.15.150:0/1915223633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c00b960 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.939+0000 7fce667fc700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56570 0x5598e3c5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce66ffd700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c556a0 0x5598e3c5f700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce58010000 tx=0x7fce58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a07c0b77ddcfe7d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce4aff5700 1 -- 172.21.15.150:0/1915223633 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58004220 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce4aff5700 1 -- 172.21.15.150:0/1915223633 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fce58004380 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598e3ace270 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce4aff5700 1 -- 172.21.15.150:0/1915223633 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce58004540 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 10 client.15801 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5598e39a2b60 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce4aff5700 1 -- 172.21.15.150:0/1915223633 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598e3ace270 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce4aff5700 1 -- 172.21.15.150:0/1915223633 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce58025030 con 0x5598e3c556a0 2024-05-10T04:40:17.948 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 10 client.15801 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 20 client.15801 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3c6b500 0x5598e3c6d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 4 client.15801 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.940+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598e3c24780 con 0x5598e3c6b500 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.941+0000 7fce667fc700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3c6b500 0x5598e3c6d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.941+0000 7fce667fc700 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3c6b500 0x5598e3c6d950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1425fd9fc3c48692 server_cookie=111ae5feb5780a43 in_seq=0 out_seq=0 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.941+0000 7fce4aff5700 10 client.15801 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce4aff5700 1 -- 172.21.15.150:0/1915223633 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598e3c24780 con 0x5598e3c6b500 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce4aff5700 10 client.15801 handle_command_reply: tid=0 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 client.15801 shutdown 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c556a0 msgr2=0x5598e3c5f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c556a0 0x5598e3c5f700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce58010000 tx=0x7fce58004750 comp rx=0 tx=0).stop 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 shutdown_connections 2024-05-10T04:40:17.949 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3c6b500 0x5598e3c6d950 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c66ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c556a0 0x5598e3c5f700 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce4aff5700 0 client.15801 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 --2- 172.21.15.150:0/1915223633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56570 0x5598e3c5fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 >> 172.21.15.150:0/1915223633 conn(0x5598e3bb7a60 msgr2=0x5598e3c4ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 shutdown_connections 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 1 -- 172.21.15.150:0/1915223633 wait complete. 2024-05-10T04:40:17.950 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.942+0000 7fce7d778b80 20 client.15801 trim_cache size 0 max 16384 2024-05-10T04:40:17.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 Processor -- start 2024-05-10T04:40:17.957 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 -- start start 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c50ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c66b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3c4a850 con 0x5598e3c56570 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3c5da80 con 0x5598e3c56ec0 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.955+0000 7fce7d778b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3c6d920 con 0x5598e3c556a0 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c66b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c66b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55366/0 (socket says 172.21.15.150:55366) 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 -- 172.21.15.150:0/3720416327 learned_addr learned my addr 172.21.15.150:0/3720416327 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce66ffd700 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c50ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce677fe700 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 -- 172.21.15.150:0/3720416327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 msgr2=0x5598e3c50ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.958 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c50ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 -- 172.21.15.150:0/3720416327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 msgr2=0x5598e3c5dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 -- 172.21.15.150:0/3720416327 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598e3b3b560 con 0x5598e3c56570 2024-05-10T04:40:17.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.956+0000 7fce667fc700 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c66b60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bd00 tx=0x7fce5c00b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd53612264f843d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce4dffb700 1 -- 172.21.15.150:0/3720416327 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c021040 con 0x5598e3c56570 2024-05-10T04:40:17.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce4dffb700 1 -- 172.21.15.150:0/3720416327 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce5c00b5a0 con 0x5598e3c56570 2024-05-10T04:40:17.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce4dffb700 1 -- 172.21.15.150:0/3720416327 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c025a90 con 0x5598e3c56570 2024-05-10T04:40:17.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce7d778b80 1 -- 172.21.15.150:0/3720416327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 msgr2=0x5598e3c66b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c66b60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00bd00 tx=0x7fce5c00b3a0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce7d778b80 1 -- 172.21.15.150:0/3720416327 shutdown_connections 2024-05-10T04:40:17.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c56ec0 0x5598e3c5dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c556a0 0x5598e3c50ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce7d778b80 1 --2- 172.21.15.150:0/3720416327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c66b60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.957+0000 7fce7d778b80 1 -- 172.21.15.150:0/3720416327 >> 172.21.15.150:0/3720416327 conn(0x5598e3bb7a60 msgr2=0x5598e3c4fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 -- 172.21.15.150:0/3720416327 shutdown_connections 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 -- 172.21.15.150:0/3720416327 wait complete. 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 -- start start 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 Processor -- start 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c556a0 0x5598e3c646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c64c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c65ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3c65ff0 con 0x5598e3c56570 2024-05-10T04:40:17.965 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3c66160 con 0x5598e3c556a0 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598e3c66320 con 0x5598e3c56ec0 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce7d778b80 10 client.0 _ll_register_callbacks cb 0x5598e3b1ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c64c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce667fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c64c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55374/0 (socket says 172.21.15.150:55374) 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce667fc700 1 -- 172.21.15.150:0/275773979 learned_addr learned my addr 172.21.15.150:0/275773979 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce66ffd700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c556a0 0x5598e3c646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.958+0000 7fce677fe700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c65ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce667fc700 1 -- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 msgr2=0x5598e3c65ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce667fc700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c65ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce667fc700 1 -- 172.21.15.150:0/275773979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c556a0 msgr2=0x5598e3c646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce667fc700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c556a0 0x5598e3c646e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.966 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce667fc700 1 -- 172.21.15.150:0/275773979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598e39f63b0 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce677fe700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c65ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce667fc700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c64c20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00b490 tx=0x7fce5c00faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d43f95b3e414d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce4aff5700 1 -- 172.21.15.150:0/275773979 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c033070 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce4aff5700 1 -- 172.21.15.150:0/275773979 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fce5c0258b0 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce5c00be50 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce4aff5700 1 -- 172.21.15.150:0/275773979 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce5c021050 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 10 client.15807 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5598e39facd0 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce4aff5700 1 -- 172.21.15.150:0/275773979 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce5c00be50 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce4aff5700 1 -- 172.21.15.150:0/275773979 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce5c036020 con 0x5598e3c56570 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 10 client.15807 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 10 client.15807 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:40:17.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 20 client.15807 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3cc2220 0x5598e3cc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 4 client.15807 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.959+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598e3b6dec0 con 0x5598e3cc2220 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce66ffd700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3cc2220 0x5598e3cc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce4aff5700 10 client.15807 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce66ffd700 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3cc2220 0x5598e3cc4640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=923c0c4effaa5d61 server_cookie=e88a012d56821eab in_seq=0 out_seq=0 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce4aff5700 1 -- 172.21.15.150:0/275773979 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598e3b6dec0 con 0x5598e3cc2220 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce4aff5700 10 client.15807 handle_command_reply: tid=0 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 client.15807 shutdown 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 msgr2=0x5598e3c64c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c64c20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00b490 tx=0x7fce5c00faf0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 shutdown_connections 2024-05-10T04:40:17.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x5598e3cc2220 0x5598e3cc4640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598e3c56ec0 0x5598e3c65ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce4aff5700 0 client.15807 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598e3c556a0 0x5598e3c646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 --2- 172.21.15.150:0/275773979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5598e3c56570 0x5598e3c64c20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.960+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 >> 172.21.15.150:0/275773979 conn(0x5598e3bb7a60 msgr2=0x5598e3c5dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 shutdown_connections 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce7d778b80 1 -- 172.21.15.150:0/275773979 wait complete. 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce7d778b80 20 client.15807 trim_cache size 0 max 16384 2024-05-10T04:40:17.969 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:17.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fce500a1260 msgr2=0x7fce500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fce500a1260 0x7fce500a3770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fce58005950 tx=0x7fce580058e0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 msgr2=0x7fce6819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce6819e3d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce60006070 tx=0x7fce60004d30 comp rx=0 tx=0).stop 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 shutdown_connections 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce68102590 0x7fce6819e3d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce68101090 0x7fce68196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fce500a1260 0x7fce500a3770 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 --2- 172.21.15.150:0/479198439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce68101a70 0x7fce68197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 >> 172.21.15.150:0/479198439 conn(0x7fce680fb8b0 msgr2=0x7fce68105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 shutdown_connections 2024-05-10T04:40:17.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:17.961+0000 7fce6d434700 1 -- 172.21.15.150:0/479198439 wait complete. 2024-05-10T04:40:17.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T04:40:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.110+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2627869013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc101700 msgr2=0x7fbcbc109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.110+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2627869013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc101700 0x7fbcbc109010 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400b0a0 tx=0x7fbcb400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2627869013 shutdown_connections 2024-05-10T04:40:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2627869013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc101700 0x7fbcbc109010 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2627869013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc1020a0 0x7fbcbc1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2627869013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc100db0 0x7fbcbc1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2627869013 >> 172.21.15.150:0/2627869013 conn(0x7fbcbc0fb8f0 msgr2=0x7fbcbc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2627869013 shutdown_connections 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2627869013 wait complete. 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 Processor -- start 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- start start 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc100db0 0x7fbcbc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc101700 0x7fbcbc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc1020a0 0x7fbcbc19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19fac0 con 0x7fbcbc100db0 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19fc30 con 0x7fbcbc1020a0 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.111+0000 7fbcc2cef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc19fdf0 con 0x7fbcbc101700 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc100db0 0x7fbcbc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc100db0 0x7fbcbc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55390/0 (socket says 172.21.15.150:55390) 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/2500795843 learned_addr learned my addr 172.21.15.150:0/2500795843 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc128c700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc1020a0 0x7fbcbc19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc101700 0x7fbcbc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/2500795843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc101700 msgr2=0x7fbcbc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc101700 0x7fbcbc198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/2500795843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc1020a0 msgr2=0x7fbcbc19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc1020a0 0x7fbcbc19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/2500795843 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb4009d00 con 0x7fbcbc100db0 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc101700 0x7fbcbc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc128c700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc1020a0 0x7fbcbc19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc100db0 0x7fbcbc197fe0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d980 tx=0x7fbcb000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcb9ffb700 1 -- 172.21.15.150:0/2500795843 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0009b40 con 0x7fbcbc100db0 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcb9ffb700 1 -- 172.21.15.150:0/2500795843 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb000de60 con 0x7fbcbc100db0 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc1a00d0 con 0x7fbcbc100db0 2024-05-10T04:40:18.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcb9ffb700 1 -- 172.21.15.150:0/2500795843 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000f620 con 0x7fbcbc100db0 2024-05-10T04:40:18.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.112+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc1a0630 con 0x7fbcbc100db0 2024-05-10T04:40:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.114+0000 7fbcb9ffb700 1 -- 172.21.15.150:0/2500795843 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fbcb0010460 con 0x7fbcbc100db0 2024-05-10T04:40:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.115+0000 7fbcb9ffb700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbcac0a12e0 0x7fbcac0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.115+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbcac0a12e0 0x7fbcac0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.115+0000 7fbcb9ffb700 1 -- 172.21.15.150:0/2500795843 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fbcb00cc8f0 con 0x7fbcbc100db0 2024-05-10T04:40:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbcac0a12e0 0x7fbcac0a37f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc199080 tx=0x7fbca8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 Processor -- start 2024-05-10T04:40:18.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 -- start start 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd29faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd1c6150 con 0x558cdd2a0570 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd1c3b10 con 0x558cdd2a0ec0 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.116+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd1bf400 con 0x558cdd29f6a0 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55392/0 (socket says 172.21.15.150:55392) 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2526179089 learned_addr learned my addr 172.21.15.150:0/2526179089 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd29faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcc128c700 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2526179089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 msgr2=0x558cdd29faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd29faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2526179089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 msgr2=0x558cdd2a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2526179089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cdd0403b0 con 0x558cdd2a0570 2024-05-10T04:40:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a0980 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005d30 tx=0x7fbca8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2a6f375b2ef1ae server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbca6ffd700 1 -- 172.21.15.150:0/2526179089 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8012b90 con 0x558cdd2a0570 2024-05-10T04:40:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbca6ffd700 1 -- 172.21.15.150:0/2526179089 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca8012cf0 con 0x558cdd2a0570 2024-05-10T04:40:18.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.117+0000 7fbca6ffd700 1 -- 172.21.15.150:0/2526179089 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca801e6e0 con 0x558cdd2a0570 2024-05-10T04:40:18.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2526179089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 msgr2=0x558cdd2a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a0980 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005d30 tx=0x7fbca8013000 comp rx=0 tx=0).stop 2024-05-10T04:40:18.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2526179089 shutdown_connections 2024-05-10T04:40:18.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd29faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2526179089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a0980 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2526179089 >> 172.21.15.150:0/2526179089 conn(0x558cdd201a60 msgr2=0x558cdd299fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2526179089 shutdown_connections 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.118+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2526179089 wait complete. 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 -- start start 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 Processor -- start 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd29f6a0 0x558cdd2a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd2b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd15b560 con 0x558cdd2a0570 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd15b200 con 0x558cdd29f6a0 2024-05-10T04:40:18.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd0426a0 con 0x558cdd2a0ec0 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcd3033b80 10 client.0 _ll_register_callbacks cb 0x558cdd164b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55394/0 (socket says 172.21.15.150:55394) 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2050825148 learned_addr learned my addr 172.21.15.150:0/2050825148 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.119+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd29f6a0 0x558cdd2a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcc128c700 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd2b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2050825148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 msgr2=0x558cdd2b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd2b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd29f6a0 msgr2=0x558cdd2a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd29f6a0 0x558cdd2a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcbbfff700 1 -- 172.21.15.150:0/2050825148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca80109f0 con 0x558cdd2a0570 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a9bb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005d00 tx=0x7fbca8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e044e61215c5631 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbc9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbc9bfff700 1 -- 172.21.15.150:0/2050825148 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca802f070 con 0x558cdd2a0570 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbc9bfff700 1 -- 172.21.15.150:0/2050825148 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fbca8005350 con 0x558cdd2a0570 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cdd118270 con 0x558cdd2a0570 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbc9bfff700 1 -- 172.21.15.150:0/2050825148 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca8005510 con 0x558cdd2a0570 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcd3033b80 10 client.15819 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558cdcfecb60 con 0x558cdd2a0570 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.120+0000 7fbc9bfff700 1 -- 172.21.15.150:0/2050825148 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cdd118270 con 0x558cdd2a0570 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbc9bfff700 1 -- 172.21.15.150:0/2050825148 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbca8003ba0 con 0x558cdd2a0570 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcd3033b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcd3033b80 10 client.15819 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcd3033b80 20 client.15819 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd2b51d0 0x558cdd2b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcd3033b80 4 client.15819 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cdd26e780 con 0x558cdd2b51d0 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd2b51d0 0x558cdd2b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd2b51d0 0x558cdd2b7620 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b08db6c78d9a8e6 server_cookie=5931667e0fbb7c82 in_seq=0 out_seq=0 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.121+0000 7fbc9bfff700 10 client.15819 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbc9bfff700 1 -- 172.21.15.150:0/2050825148 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558cdd26e780 con 0x558cdd2b51d0 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbc9bfff700 10 client.15819 handle_command_reply: tid=0 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 client.15819 shutdown 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 msgr2=0x558cdd2a9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a9bb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005d00 tx=0x7fbca8012bc0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 shutdown_connections 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd2b51d0 0x558cdd2b7620 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd2b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd29f6a0 0x558cdd2a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/2050825148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2a9bb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbc9bfff700 0 client.15819 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 >> 172.21.15.150:0/2050825148 conn(0x558cdd201a60 msgr2=0x558cdd299fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 shutdown_connections 2024-05-10T04:40:18.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 1 -- 172.21.15.150:0/2050825148 wait complete. 2024-05-10T04:40:18.130 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.122+0000 7fbcd3033b80 20 client.15819 trim_cache size 0 max 16384 2024-05-10T04:40:18.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.135+0000 7fbcd3033b80 1 Processor -- start 2024-05-10T04:40:18.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcd3033b80 1 -- start start 2024-05-10T04:40:18.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd2a0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd29b770 con 0x558cdd2a0570 2024-05-10T04:40:18.142 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd2b4f20 con 0x558cdd2a0ec0 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd2a3ce0 con 0x558cdd29f6a0 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55398/0 (socket says 172.21.15.150:55398) 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 -- 172.21.15.150:0/877371972 learned_addr learned my addr 172.21.15.150:0/877371972 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd2a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcc128c700 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 -- 172.21.15.150:0/877371972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 msgr2=0x558cdd2a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd2a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 -- 172.21.15.150:0/877371972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 msgr2=0x558cdd2a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 -- 172.21.15.150:0/877371972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cdd185560 con 0x558cdd2a0570 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2b0ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbca80096c0 tx=0x7fbca8038e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc07ad68c7f7a96a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbca77fe700 1 -- 172.21.15.150:0/877371972 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca802f040 con 0x558cdd2a0570 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.136+0000 7fbca77fe700 1 -- 172.21.15.150:0/877371972 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbca8028450 con 0x558cdd2a0570 2024-05-10T04:40:18.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbca77fe700 1 -- 172.21.15.150:0/877371972 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8003a00 con 0x558cdd2a0570 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 -- 172.21.15.150:0/877371972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 msgr2=0x558cdd2b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2b0ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbca80096c0 tx=0x7fbca8038e60 comp rx=0 tx=0).stop 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 -- 172.21.15.150:0/877371972 shutdown_connections 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0ec0 0x558cdd2a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd29f6a0 0x558cdd2a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/877371972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd2a0570 0x558cdd2b0ad0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 -- 172.21.15.150:0/877371972 >> 172.21.15.150:0/877371972 conn(0x558cdd201a60 msgr2=0x558cdd29fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 -- 172.21.15.150:0/877371972 shutdown_connections 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.137+0000 7fbcd3033b80 1 -- 172.21.15.150:0/877371972 wait complete. 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 -- start start 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 Processor -- start 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd29f6a0 0x558cdd2ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0570 0x558cdd2adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd309670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.144 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd2ae520 con 0x558cdd29f6a0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd309bb0 con 0x558cdd2a0570 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdd309cf0 con 0x558cdd2a0ec0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcd3033b80 10 client.0 _ll_register_callbacks cb 0x558cdd164b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd29f6a0 0x558cdd2ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd29f6a0 0x558cdd2ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55402/0 (socket says 172.21.15.150:55402) 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/641484324 learned_addr learned my addr 172.21.15.150:0/641484324 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc128c700 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd309670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/641484324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 msgr2=0x558cdd309670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd309670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0570 0x558cdd2adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0570 msgr2=0x558cdd2adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0570 0x558cdd2adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.138+0000 7fbcc0a8b700 1 -- 172.21.15.150:0/641484324 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cdcfecb60 con 0x558cdd29f6a0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcc0a8b700 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd29f6a0 0x558cdd2ad9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0099540 tx=0x7fbcb0099480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ad1ccdc7c9cb6f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbc9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbc9bfff700 1 -- 172.21.15.150:0/641484324 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00ca150 con 0x558cdd29f6a0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbc9bfff700 1 -- 172.21.15.150:0/641484324 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fbcb00b1030 con 0x558cdd29f6a0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbca8012cf0 con 0x558cdd29f6a0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbc9bfff700 1 -- 172.21.15.150:0/641484324 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb0099aa0 con 0x558cdd29f6a0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 10 client.15825 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558cdd044cd0 con 0x558cdd29f6a0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbc9bfff700 1 -- 172.21.15.150:0/641484324 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb007e080 con 0x558cdd29f6a0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbc9bfff700 1 -- 172.21.15.150:0/641484324 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcb00a1830 con 0x558cdd29f6a0 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 10 client.15825 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 20 client.15825 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd30b590 0x558cdd30d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 4 client.15825 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:18.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.139+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558cdd1b7ec0 con 0x558cdd30b590 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd30b590 0x558cdd30d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbc9bfff700 10 client.15825 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcbbfff700 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd30b590 0x558cdd30d9b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b205d0556f4cbd03 server_cookie=94a2ec6dd61705ce in_seq=0 out_seq=0 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbc9bfff700 1 -- 172.21.15.150:0/641484324 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558cdd1b7ec0 con 0x558cdd30b590 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbc9bfff700 10 client.15825 handle_command_reply: tid=0 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 client.15825 shutdown 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd29f6a0 msgr2=0x558cdd2ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd29f6a0 0x558cdd2ad9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0099540 tx=0x7fbcb0099480 comp rx=0 tx=0).stop 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 shutdown_connections 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x558cdd30b590 0x558cdd30d9b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cdd2a0ec0 0x558cdd309670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558cdd29f6a0 0x558cdd2ad9c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbc9bfff700 0 client.15825 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 --2- 172.21.15.150:0/641484324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558cdd2a0570 0x558cdd2adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 >> 172.21.15.150:0/641484324 conn(0x558cdd201a60 msgr2=0x558cdd2a42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.140+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 shutdown_connections 2024-05-10T04:40:18.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcd3033b80 1 -- 172.21.15.150:0/641484324 wait complete. 2024-05-10T04:40:18.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcd3033b80 20 client.15825 trim_cache size 0 max 16384 2024-05-10T04:40:18.148 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbcac0a12e0 msgr2=0x7fbcac0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbcac0a12e0 0x7fbcac0a37f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc199080 tx=0x7fbca8009260 comp rx=0 tx=0).stop 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc100db0 msgr2=0x7fbcbc197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc100db0 0x7fbcbc197fe0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d980 tx=0x7fbcb000dd40 comp rx=0 tx=0).stop 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 shutdown_connections 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcbc1020a0 0x7fbcbc19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcbc100db0 0x7fbcbc197fe0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbcac0a12e0 0x7fbcac0a37f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 --2- 172.21.15.150:0/2500795843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc101700 0x7fbcbc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 >> 172.21.15.150:0/2500795843 conn(0x7fbcbc0fb8f0 msgr2=0x7fbcbc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 shutdown_connections 2024-05-10T04:40:18.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.141+0000 7fbcc2cef700 1 -- 172.21.15.150:0/2500795843 wait complete. 2024-05-10T04:40:18.155 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:40:18.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 -- 172.21.15.150:0/1606201481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad4100280 msgr2=0x7fbad4100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/1606201481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad4100280 0x7fbad4100690 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009990 tx=0x7fbac4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 -- 172.21.15.150:0/1606201481 shutdown_connections 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/1606201481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad4101d40 0x7fbad41085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/1606201481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad4100280 0x7fbad4100690 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/1606201481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbad41013a0 0x7fbad41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 -- 172.21.15.150:0/1606201481 >> 172.21.15.150:0/1606201481 conn(0x7fbad40fb8b0 msgr2=0x7fbad40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 -- 172.21.15.150:0/1606201481 shutdown_connections 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.288+0000 7fbadb6c6700 1 -- 172.21.15.150:0/1606201481 wait complete. 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 Processor -- start 2024-05-10T04:40:18.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 -- start start 2024-05-10T04:40:18.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad4100280 0x7fbad4196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad41013a0 0x7fbad4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbad4101d40 0x7fbad419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad419e8b0 con 0x7fbad41013a0 2024-05-10T04:40:18.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad419e9f0 con 0x7fbad4100280 2024-05-10T04:40:18.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbadb6c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad419eb80 con 0x7fbad4101d40 2024-05-10T04:40:18.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad41013a0 0x7fbad4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad41013a0 0x7fbad4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55430/0 (socket says 172.21.15.150:55430) 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 -- 172.21.15.150:0/2254715883 learned_addr learned my addr 172.21.15.150:0/2254715883 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad9c63700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbad4101d40 0x7fbad419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad9462700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad4100280 0x7fbad4196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 -- 172.21.15.150:0/2254715883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbad4101d40 msgr2=0x7fbad419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbad4101d40 0x7fbad419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 -- 172.21.15.150:0/2254715883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad4100280 msgr2=0x7fbad4196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad4100280 0x7fbad4196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.289+0000 7fbad8c61700 1 -- 172.21.15.150:0/2254715883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac40095f0 con 0x7fbad41013a0 2024-05-10T04:40:18.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.290+0000 7fbad8c61700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad41013a0 0x7fbad4197350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbad000c9e0 tx=0x7fbad000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.290+0000 7fbaca7fc700 1 -- 172.21.15.150:0/2254715883 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad0007ac0 con 0x7fbad41013a0 2024-05-10T04:40:18.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.290+0000 7fbaca7fc700 1 -- 172.21.15.150:0/2254715883 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbad000ce80 con 0x7fbad41013a0 2024-05-10T04:40:18.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.290+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad419ee60 con 0x7fbad41013a0 2024-05-10T04:40:18.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.290+0000 7fbaca7fc700 1 -- 172.21.15.150:0/2254715883 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad0018660 con 0x7fbad41013a0 2024-05-10T04:40:18.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.290+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad419f280 con 0x7fbad41013a0 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.292+0000 7fbaca7fc700 1 -- 172.21.15.150:0/2254715883 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7fbad0004830 con 0x7fbad41013a0 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.293+0000 7fbaca7fc700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbac00a1260 0x7fbac00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.293+0000 7fbaca7fc700 1 -- 172.21.15.150:0/2254715883 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fbad00cd010 con 0x7fbad41013a0 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.293+0000 7fbad9462700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbac00a1260 0x7fbac00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.293+0000 7fbad9462700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbac00a1260 0x7fbac00a3770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbac40098e0 tx=0x7fbac401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 Processor -- start 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 -- start start 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aaceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aacf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831a9f5150 con 0x55831aacfec0 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831a9f2b10 con 0x55831aacf570 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831a9ee400 con 0x55831aace6a0 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55432/0 (socket says 172.21.15.150:55432) 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbad9c63700 1 -- 172.21.15.150:0/236109166 learned_addr learned my addr 172.21.15.150:0/236109166 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbad9462700 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aaceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.294+0000 7fbad8c61700 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aacf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbad9c63700 1 -- 172.21.15.150:0/236109166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 msgr2=0x55831aaceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbad9c63700 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aaceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbad9c63700 1 -- 172.21.15.150:0/236109166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 msgr2=0x55831aacf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbad9c63700 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aacf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbad9c63700 1 -- 172.21.15.150:0/236109166 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55831a86f3b0 con 0x55831aacfec0 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbad9462700 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aaceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbad9c63700 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad67c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbacc003f80 tx=0x7fbacc009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb6624f95097769 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbabf7fe700 1 -- 172.21.15.150:0/236109166 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc019070 con 0x55831aacfec0 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbabf7fe700 1 -- 172.21.15.150:0/236109166 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbacc00f040 con 0x55831aacfec0 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.295+0000 7fbabf7fe700 1 -- 172.21.15.150:0/236109166 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0079d0 con 0x55831aacfec0 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/236109166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 msgr2=0x55831aad67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad67c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbacc003f80 tx=0x7fbacc009ba0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/236109166 shutdown_connections 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad67c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aaceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/236109166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aacf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/236109166 >> 172.21.15.150:0/236109166 conn(0x55831aa30a60 msgr2=0x55831aac8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/236109166 shutdown_connections 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/236109166 wait complete. 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 -- start start 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.296+0000 7fbaeba0ab80 1 Processor -- start 2024-05-10T04:40:18.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aadfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831a98a560 con 0x55831aacfec0 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831a98a200 con 0x55831aacf570 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831a8716a0 con 0x55831aace6a0 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbaeba0ab80 10 client.0 _ll_register_callbacks cb 0x55831a993b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aadfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aadfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55452/0 (socket says 172.21.15.150:55452) 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 -- 172.21.15.150:0/2489778218 learned_addr learned my addr 172.21.15.150:0/2489778218 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad8c61700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9462700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 -- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 msgr2=0x55831aad86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 -- 172.21.15.150:0/2489778218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 msgr2=0x55831aad8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad9c63700 1 -- 172.21.15.150:0/2489778218 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc003bb0 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.297+0000 7fbad8c61700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbad9c63700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aadfb70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbacc004860 tx=0x7fbacc004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6908d433e15abd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbabcff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbabcff9700 1 -- 172.21.15.150:0/2489778218 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc019040 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbabcff9700 1 -- 172.21.15.150:0/2489778218 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fbacc007c20 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55831a947270 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbabcff9700 1 -- 172.21.15.150:0/2489778218 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbacc007de0 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 10 client.15837 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55831a81bb60 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbabcff9700 1 -- 172.21.15.150:0/2489778218 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55831a947270 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbabcff9700 1 -- 172.21.15.150:0/2489778218 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbacc01f5e0 con 0x55831aacfec0 2024-05-10T04:40:18.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 10 client.15837 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 10 client.15837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 20 client.15837 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831aae4260 0x55831aae66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 4 client.15837 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55831aa9d780 con 0x55831aae4260 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.298+0000 7fbad9462700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831aae4260 0x55831aae66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.299+0000 7fbabcff9700 10 client.15837 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.299+0000 7fbad9462700 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831aae4260 0x55831aae66b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b212e8172646579 server_cookie=f79bc498c8a4ef59 in_seq=0 out_seq=0 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.299+0000 7fbabcff9700 1 -- 172.21.15.150:0/2489778218 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55831aa9d780 con 0x55831aae4260 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.299+0000 7fbabcff9700 10 client.15837 handle_command_reply: tid=0 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 client.15837 shutdown 2024-05-10T04:40:18.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 msgr2=0x55831aadfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aadfb70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbacc004860 tx=0x7fbacc004890 comp rx=0 tx=0).stop 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 shutdown_connections 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831aae4260 0x55831aae66b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aadfb70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbabcff9700 0 client.15837 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2489778218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 >> 172.21.15.150:0/2489778218 conn(0x55831aa30a60 msgr2=0x55831aac8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 shutdown_connections 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2489778218 wait complete. 2024-05-10T04:40:18.309 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.300+0000 7fbaeba0ab80 20 client.15837 trim_cache size 0 max 16384 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 Processor -- start 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 -- start start 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831aacf3c0 con 0x55831aacfec0 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831aac9d20 con 0x55831aacf570 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.313+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831aae3f60 con 0x55831aace6a0 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55454/0 (socket says 172.21.15.150:55454) 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 -- 172.21.15.150:0/728533220 learned_addr learned my addr 172.21.15.150:0/728533220 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9462700 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad8c61700 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 -- 172.21.15.150:0/728533220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 msgr2=0x55831aad7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 -- 172.21.15.150:0/728533220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 msgr2=0x55831aad7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 -- 172.21.15.150:0/728533220 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55831a9b4560 con 0x55831aacfec0 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad8c61700 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbad9c63700 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad6aa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbacc003ea0 tx=0x7fbacc009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf12e05d0447d6c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbabffff700 1 -- 172.21.15.150:0/728533220 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc019070 con 0x55831aacfec0 2024-05-10T04:40:18.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbabffff700 1 -- 172.21.15.150:0/728533220 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbacc007570 con 0x55831aacfec0 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.314+0000 7fbabffff700 1 -- 172.21.15.150:0/728533220 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc015800 con 0x55831aacfec0 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/728533220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 msgr2=0x55831aad6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad6aa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbacc003ea0 tx=0x7fbacc009100 comp rx=0 tx=0).stop 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/728533220 shutdown_connections 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831aad6aa0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aace6a0 0x55831aad7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/728533220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aacf570 0x55831aad7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/728533220 >> 172.21.15.150:0/728533220 conn(0x55831aa30a60 msgr2=0x55831aad3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/728533220 shutdown_connections 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/728533220 wait complete. 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 -- start start 2024-05-10T04:40:18.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.315+0000 7fbaeba0ab80 1 Processor -- start 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aace6a0 0x55831aadf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aacf570 0x55831aadf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbaeba0ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831ab3ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831ab3b090 con 0x55831aacfec0 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831ab3b1d0 con 0x55831aace6a0 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbaeba0ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831ab3b360 con 0x55831aacf570 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbaeba0ab80 10 client.0 _ll_register_callbacks cb 0x55831a993b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831ab3ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831ab3ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55460/0 (socket says 172.21.15.150:55460) 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 -- 172.21.15.150:0/2348928770 learned_addr learned my addr 172.21.15.150:0/2348928770 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad8c61700 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aacf570 0x55831aadf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9462700 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aace6a0 0x55831aadf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 -- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aacf570 msgr2=0x55831aadf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aacf570 0x55831aadf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 -- 172.21.15.150:0/2348928770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aace6a0 msgr2=0x55831aadf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aace6a0 0x55831aadf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.316+0000 7fbad9c63700 1 -- 172.21.15.150:0/2348928770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc01fe70 con 0x55831aacfec0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbad9c63700 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831ab3ab50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbacc01f940 tx=0x7fbacc007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4175f875ca3f3c35 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbabcff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbabcff9700 1 -- 172.21.15.150:0/2348928770 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc03f070 con 0x55831aacfec0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbabcff9700 1 -- 172.21.15.150:0/2348928770 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fbacc007a20 con 0x55831aacfec0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbacc00f040 con 0x55831aacfec0 2024-05-10T04:40:18.324 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbabcff9700 1 -- 172.21.15.150:0/2348928770 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbacc019040 con 0x55831aacfec0 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 10 client.15843 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55831a873cd0 con 0x55831aacfec0 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbabcff9700 1 -- 172.21.15.150:0/2348928770 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbacc00f040 con 0x55831aacfec0 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbabcff9700 1 -- 172.21.15.150:0/2348928770 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbacc042020 con 0x55831aacfec0 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 10 client.15843 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 10 client.15843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 20 client.15843 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831ab3c810 0x55831ab3ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 4 client.15843 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.317+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55831a9e6ec0 con 0x55831ab3c810 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbad9462700 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831ab3c810 0x55831ab3ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbabcff9700 10 client.15843 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbad9462700 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831ab3c810 0x55831ab3ec30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b64ddd0c8aad1a3 server_cookie=389cae288a6d241 in_seq=0 out_seq=0 2024-05-10T04:40:18.325 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbabcff9700 1 -- 172.21.15.150:0/2348928770 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55831a9e6ec0 con 0x55831ab3c810 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbabcff9700 10 client.15843 handle_command_reply: tid=0 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 client.15843 shutdown 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 msgr2=0x55831ab3ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831ab3ab50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbacc01f940 tx=0x7fbacc007760 comp rx=0 tx=0).stop 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 shutdown_connections 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55831ab3c810 0x55831ab3ec30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55831aacfec0 0x55831ab3ab50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbabcff9700 0 client.15843 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55831aace6a0 0x55831aadf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 --2- 172.21.15.150:0/2348928770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55831aacf570 0x55831aadf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 >> 172.21.15.150:0/2348928770 conn(0x55831aa30a60 msgr2=0x55831aad7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.318+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 shutdown_connections 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbaeba0ab80 1 -- 172.21.15.150:0/2348928770 wait complete. 2024-05-10T04:40:18.326 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbaeba0ab80 20 client.15843 trim_cache size 0 max 16384 2024-05-10T04:40:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:18.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbac00a1260 msgr2=0x7fbac00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbac00a1260 0x7fbac00a3770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbac40098e0 tx=0x7fbac401a040 comp rx=0 tx=0).stop 2024-05-10T04:40:18.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad41013a0 msgr2=0x7fbad4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.327 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad41013a0 0x7fbad4197350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbad000c9e0 tx=0x7fbad000ccf0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 shutdown_connections 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fbac00a1260 0x7fbac00a3770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbad4101d40 0x7fbad419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad4100280 0x7fbad4196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 --2- 172.21.15.150:0/2254715883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbad41013a0 0x7fbad4197350 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 >> 172.21.15.150:0/2254715883 conn(0x7fbad40fb8b0 msgr2=0x7fbad4105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 shutdown_connections 2024-05-10T04:40:18.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.319+0000 7fbadb6c6700 1 -- 172.21.15.150:0/2254715883 wait complete. 2024-05-10T04:40:18.333 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 -- 172.21.15.150:0/3801665612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190101be0 msgr2=0x7f3190101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 --2- 172.21.15.150:0/3801665612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190101be0 0x7f3190101ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f318c009990 tx=0x7f318c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 -- 172.21.15.150:0/3801665612 shutdown_connections 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 --2- 172.21.15.150:0/3801665612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3190102580 0x7f3190108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 --2- 172.21.15.150:0/3801665612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3190100ac0 0x7f3190100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 --2- 172.21.15.150:0/3801665612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190101be0 0x7f3190101ff0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 -- 172.21.15.150:0/3801665612 >> 172.21.15.150:0/3801665612 conn(0x7f31900fc0d0 msgr2=0x7f31900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 -- 172.21.15.150:0/3801665612 shutdown_connections 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.469+0000 7f3198031700 1 -- 172.21.15.150:0/3801665612 wait complete. 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 Processor -- start 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 -- start start 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3190100ac0 0x7f3190068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3190101be0 0x7f319019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190102580 0x7f319019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319019f9a0 con 0x7f3190102580 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319019fb10 con 0x7f3190101be0 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3198031700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319019fcd0 con 0x7f3190100ac0 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f31965ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190102580 0x7f319019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3195dcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3190100ac0 0x7f3190068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3195dcd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3190100ac0 0x7f3190068e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:42804/0 (socket says 172.21.15.150:42804) 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f3195dcd700 1 -- 172.21.15.150:0/1166841735 learned_addr learned my addr 172.21.15.150:0/1166841735 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f31965ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190102580 0x7f319019f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55474/0 (socket says 172.21.15.150:55474) 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.470+0000 7f31955cc700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3190101be0 0x7f319019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f31965ce700 1 -- 172.21.15.150:0/1166841735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3190100ac0 msgr2=0x7f3190068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f31965ce700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3190100ac0 0x7f3190068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f31965ce700 1 -- 172.21.15.150:0/1166841735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3190101be0 msgr2=0x7f319019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f31965ce700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3190101be0 0x7f319019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f31965ce700 1 -- 172.21.15.150:0/1166841735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c0095f0 con 0x7f3190102580 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f31965ce700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190102580 0x7f319019f2a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3188007f10 tx=0x7f318800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f3186ffd700 1 -- 172.21.15.150:0/1166841735 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188009ea0 con 0x7f3190102580 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f3186ffd700 1 -- 172.21.15.150:0/1166841735 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f318800f040 con 0x7f3190102580 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319019ffb0 con 0x7f3190102580 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f3186ffd700 1 -- 172.21.15.150:0/1166841735 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188004400 con 0x7f3190102580 2024-05-10T04:40:18.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.471+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31901a03e0 con 0x7f3190102580 2024-05-10T04:40:18.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.474+0000 7f3186ffd700 1 -- 172.21.15.150:0/1166841735 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3188003a00 con 0x7f3190102580 2024-05-10T04:40:18.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.474+0000 7f31a8375b80 1 Processor -- start 2024-05-10T04:40:18.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31a8375b80 1 -- start start 2024-05-10T04:40:18.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f3186ffd700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f317c0a1260 0x7f317c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c536af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c537980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c53e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31a8375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c45d150 con 0x55aa9c537ec0 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31a8375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c45ab10 con 0x55aa9c5366a0 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31a8375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c456400 con 0x55aa9c537570 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31965ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c53e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31955cc700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f317c0a1260 0x7f317c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31965ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c53e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55476/0 (socket says 172.21.15.150:55476) 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31965ce700 1 -- 172.21.15.150:0/3420587516 learned_addr learned my addr 172.21.15.150:0/3420587516 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c536af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f3186ffd700 1 -- 172.21.15.150:0/1166841735 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f31880cc6f0 con 0x7f3190102580 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.475+0000 7f31955cc700 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c537980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f31965ce700 1 -- 172.21.15.150:0/3420587516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 msgr2=0x55aa9c537980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f31965ce700 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c537980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f31965ce700 1 -- 172.21.15.150:0/3420587516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 msgr2=0x55aa9c536af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f31965ce700 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c536af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f31965ce700 1 -- 172.21.15.150:0/3420587516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa9c2d73b0 con 0x55aa9c537ec0 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c536af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f31955cc700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f317c0a1260 0x7f317c0a3770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f318c00b2e0 tx=0x7f318c00b930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f31965ce700 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c53e7c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3188095ae0 tx=0x7f31880097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b38c46b4aede247 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f317bfff700 1 -- 172.21.15.150:0/3420587516 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188098f40 con 0x55aa9c537ec0 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f317bfff700 1 -- 172.21.15.150:0/3420587516 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3188003cb0 con 0x55aa9c537ec0 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.476+0000 7f317bfff700 1 -- 172.21.15.150:0/3420587516 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880995e0 con 0x55aa9c537ec0 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 -- 172.21.15.150:0/3420587516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 msgr2=0x55aa9c53e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c53e7c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3188095ae0 tx=0x7f31880097f0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 -- 172.21.15.150:0/3420587516 shutdown_connections 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c53e7c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c536af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3420587516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c537980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 -- 172.21.15.150:0/3420587516 >> 172.21.15.150:0/3420587516 conn(0x55aa9c498a60 msgr2=0x55aa9c530fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 -- 172.21.15.150:0/3420587516 shutdown_connections 2024-05-10T04:40:18.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 -- 172.21.15.150:0/3420587516 wait complete. 2024-05-10T04:40:18.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.477+0000 7f31a8375b80 1 -- start start 2024-05-10T04:40:18.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 1 Processor -- start 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c540640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c540b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c547ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c3f2560 con 0x55aa9c537ec0 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c3f2200 con 0x55aa9c5366a0 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c2d96a0 con 0x55aa9c537570 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31a8375b80 10 client.0 _ll_register_callbacks cb 0x55aa9c3fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31965ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c547ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31965ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c547ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55480/0 (socket says 172.21.15.150:55480) 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31965ce700 1 -- 172.21.15.150:0/3462340071 learned_addr learned my addr 172.21.15.150:0/3462340071 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c540640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.478+0000 7f31955cc700 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c540b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31965ce700 1 -- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 msgr2=0x55aa9c540b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31965ce700 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c540b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31965ce700 1 -- 172.21.15.150:0/3462340071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 msgr2=0x55aa9c540640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31965ce700 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c540640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31965ce700 1 -- 172.21.15.150:0/3462340071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3180009930 con 0x55aa9c537ec0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31965ce700 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c547ae0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f318800d7b0 tx=0x7f318800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab26eeaa656be61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31797fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31797fa700 1 -- 172.21.15.150:0/3462340071 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318801c310 con 0x55aa9c537ec0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31797fa700 1 -- 172.21.15.150:0/3462340071 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f31880b3060 con 0x55aa9c537ec0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa9c3af270 con 0x55aa9c537ec0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31797fa700 1 -- 172.21.15.150:0/3462340071 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3188099ce0 con 0x55aa9c537ec0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 10 client.15855 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f31901a03e0 con 0x55aa9c537ec0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31797fa700 1 -- 172.21.15.150:0/3462340071 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa9c3af270 con 0x55aa9c537ec0 2024-05-10T04:40:18.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31797fa700 1 -- 172.21.15.150:0/3462340071 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f31880cb070 con 0x55aa9c537ec0 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 10 client.15855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 20 client.15855 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c54bf70 0x55aa9c54e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 4 client.15855 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.479+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa9c505780 con 0x55aa9c54bf70 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.480+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c54bf70 0x55aa9c54e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.480+0000 7f31797fa700 10 client.15855 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.480+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c54bf70 0x55aa9c54e3c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c281380bd90b6dd3 server_cookie=b50ac8b759d6b270 in_seq=0 out_seq=0 2024-05-10T04:40:18.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31797fa700 1 -- 172.21.15.150:0/3462340071 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa9c505780 con 0x55aa9c54bf70 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31797fa700 10 client.15855 handle_command_reply: tid=0 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 client.15855 shutdown 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 msgr2=0x55aa9c547ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c547ae0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f318800d7b0 tx=0x7f318800d950 comp rx=0 tx=0).stop 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 shutdown_connections 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c54bf70 0x55aa9c54e3c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537ec0 0x55aa9c547ae0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c540640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3462340071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c540b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31797fa700 0 client.15855 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 >> 172.21.15.150:0/3462340071 conn(0x55aa9c498a60 msgr2=0x55aa9c530fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 shutdown_connections 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 1 -- 172.21.15.150:0/3462340071 wait complete. 2024-05-10T04:40:18.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.481+0000 7f31a8375b80 20 client.15855 trim_cache size 0 max 16384 2024-05-10T04:40:18.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 Processor -- start 2024-05-10T04:40:18.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 -- start start 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c5366a0 0x55aa9c54e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c547aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c537ec0 0x55aa9c52d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c52d6c0 con 0x55aa9c5366a0 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c532770 con 0x55aa9c537ec0 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.494+0000 7f31a8375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c536f50 con 0x55aa9c537570 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c5366a0 0x55aa9c54e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c5366a0 0x55aa9c54e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55488/0 (socket says 172.21.15.150:55488) 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 -- 172.21.15.150:0/1781263297 learned_addr learned my addr 172.21.15.150:0/1781263297 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f31955cc700 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c547aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f31965ce700 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c537ec0 0x55aa9c52d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 -- 172.21.15.150:0/1781263297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 msgr2=0x55aa9c547aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c547aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 -- 172.21.15.150:0/1781263297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c537ec0 msgr2=0x55aa9c52d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c537ec0 0x55aa9c52d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 -- 172.21.15.150:0/1781263297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa9c283b60 con 0x55aa9c5366a0 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3195dcd700 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c5366a0 0x55aa9c54e3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55aa9c54ec60 tx=0x7f3180009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ab88a3fe84c8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3184ff9700 1 -- 172.21.15.150:0/1781263297 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800040f0 con 0x55aa9c5366a0 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3184ff9700 1 -- 172.21.15.150:0/1781263297 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3180018040 con 0x55aa9c5366a0 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.495+0000 7f3184ff9700 1 -- 172.21.15.150:0/1781263297 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318001f370 con 0x55aa9c5366a0 2024-05-10T04:40:18.499 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 -- 172.21.15.150:0/1781263297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c5366a0 msgr2=0x55aa9c54e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c5366a0 0x55aa9c54e3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55aa9c54ec60 tx=0x7f3180009260 comp rx=0 tx=0).stop 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 -- 172.21.15.150:0/1781263297 shutdown_connections 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c537ec0 0x55aa9c52d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c5366a0 0x55aa9c54e3c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 --2- 172.21.15.150:0/1781263297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537570 0x55aa9c547aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 -- 172.21.15.150:0/1781263297 >> 172.21.15.150:0/1781263297 conn(0x55aa9c498a60 msgr2=0x55aa9c53fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 -- 172.21.15.150:0/1781263297 shutdown_connections 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 -- 172.21.15.150:0/1781263297 wait complete. 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 -- start start 2024-05-10T04:40:18.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.496+0000 7f31a8375b80 1 Processor -- start 2024-05-10T04:40:18.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c544ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537570 0x55aa9c545430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31a8375b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537ec0 0x55aa9c546360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31a8375b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c5468a0 con 0x55aa9c537570 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31a8375b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c546a10 con 0x55aa9c5366a0 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31a8375b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa9c546bd0 con 0x55aa9c537ec0 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31a8375b80 10 client.0 _ll_register_callbacks cb 0x55aa9c3fbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537570 0x55aa9c545430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537570 0x55aa9c545430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55500/0 (socket says 172.21.15.150:55500) 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 -- 172.21.15.150:0/3983311011 learned_addr learned my addr 172.21.15.150:0/3983311011 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31965ce700 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537ec0 0x55aa9c546360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c544ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 -- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537ec0 msgr2=0x55aa9c546360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537ec0 0x55aa9c546360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 -- 172.21.15.150:0/3983311011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 msgr2=0x55aa9c544ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c544ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 -- 172.21.15.150:0/3983311011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3180011520 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31955cc700 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537570 0x55aa9c545430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f318c009990 tx=0x7f318c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce6ce5bf652207c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.497+0000 7f31797fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31797fa700 1 -- 172.21.15.150:0/3983311011 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c02d070 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31797fa700 1 -- 172.21.15.150:0/3983311011 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f318c00be40 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3188003cb0 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31797fa700 1 -- 172.21.15.150:0/3983311011 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f318c0279c0 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 10 client.15861 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55aa9c41c560 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31797fa700 1 -- 172.21.15.150:0/3983311011 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3188003cb0 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31797fa700 1 -- 172.21.15.150:0/3983311011 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f318c0050b0 con 0x55aa9c537570 2024-05-10T04:40:18.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 10 client.15861 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 10 client.15861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 20 client.15861 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c5a2990 0x55aa9c5a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 4 client.15861 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa9c44eec0 con 0x55aa9c5a2990 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.498+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c5a2990 0x55aa9c5a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f3195dcd700 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c5a2990 0x55aa9c5a4db0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30b910f7c92e84f7 server_cookie=ff4f8401fd8d0699 in_seq=0 out_seq=0 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31797fa700 10 client.15861 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31797fa700 1 -- 172.21.15.150:0/3983311011 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa9c44eec0 con 0x55aa9c5a2990 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31797fa700 10 client.15861 handle_command_reply: tid=0 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 client.15861 shutdown 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537570 msgr2=0x55aa9c545430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537570 0x55aa9c545430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f318c009990 tx=0x7f318c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 shutdown_connections 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55aa9c5a2990 0x55aa9c5a4db0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa9c537ec0 0x55aa9c546360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa9c5366a0 0x55aa9c544ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31797fa700 0 client.15861 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 --2- 172.21.15.150:0/3983311011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa9c537570 0x55aa9c545430 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 >> 172.21.15.150:0/3983311011 conn(0x55aa9c498a60 msgr2=0x55aa9c53fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 shutdown_connections 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.499+0000 7f31a8375b80 1 -- 172.21.15.150:0/3983311011 wait complete. 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f31a8375b80 20 client.15861 trim_cache size 0 max 16384 2024-05-10T04:40:18.508 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:18.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f317c0a1260 msgr2=0x7f317c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f317c0a1260 0x7f317c0a3770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f318c00b2e0 tx=0x7f318c00b930 comp rx=0 tx=0).stop 2024-05-10T04:40:18.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190102580 msgr2=0x7f319019f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190102580 0x7f319019f2a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3188007f10 tx=0x7f318800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 shutdown_connections 2024-05-10T04:40:18.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3190102580 0x7f319019f2a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f317c0a1260 0x7f317c0a3770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3190100ac0 0x7f3190068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 --2- 172.21.15.150:0/1166841735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3190101be0 0x7f319019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 >> 172.21.15.150:0/1166841735 conn(0x7f31900fc0d0 msgr2=0x7f3190104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 shutdown_connections 2024-05-10T04:40:18.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.500+0000 7f3198031700 1 -- 172.21.15.150:0/1166841735 wait complete. 2024-05-10T04:40:18.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:40:18.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 -- 172.21.15.150:0/110879769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640102580 msgr2=0x7f3640108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 --2- 172.21.15.150:0/110879769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640102580 0x7f3640108e30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f363800b0a0 tx=0x7f363800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 -- 172.21.15.150:0/110879769 shutdown_connections 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 --2- 172.21.15.150:0/110879769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640102580 0x7f3640108e30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 --2- 172.21.15.150:0/110879769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3640100ac0 0x7f3640100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 --2- 172.21.15.150:0/110879769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3640101be0 0x7f3640101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 -- 172.21.15.150:0/110879769 >> 172.21.15.150:0/110879769 conn(0x7f36400fc0d0 msgr2=0x7f36400fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 -- 172.21.15.150:0/110879769 shutdown_connections 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 -- 172.21.15.150:0/110879769 wait complete. 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 Processor -- start 2024-05-10T04:40:18.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.648+0000 7f3646674700 1 -- start start 2024-05-10T04:40:18.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3646674700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640100ac0 0x7f3640068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3646674700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3640101be0 0x7f364019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3646674700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3640102580 0x7f364019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3646674700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364019fad0 con 0x7f3640100ac0 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3646674700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364019fc40 con 0x7f3640102580 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3646674700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364019fe00 con 0x7f3640101be0 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640100ac0 0x7f3640068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640100ac0 0x7f3640068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55516/0 (socket says 172.21.15.150:55516) 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 -- 172.21.15.150:0/1626260739 learned_addr learned my addr 172.21.15.150:0/1626260739 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363f7fe700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3640101be0 0x7f364019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3644c11700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3640102580 0x7f364019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 -- 172.21.15.150:0/1626260739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3640101be0 msgr2=0x7f364019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3640101be0 0x7f364019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 -- 172.21.15.150:0/1626260739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3640102580 msgr2=0x7f364019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3640102580 0x7f364019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 -- 172.21.15.150:0/1626260739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009d00 con 0x7f3640100ac0 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3644c11700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3640102580 0x7f364019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363ffff700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640100ac0 0x7f3640068e50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f363000e940 tx=0x7f363000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363d7fa700 1 -- 172.21.15.150:0/1626260739 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363000cbd0 con 0x7f3640100ac0 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f363d7fa700 1 -- 172.21.15.150:0/1626260739 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363000ee90 con 0x7f3640100ac0 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.649+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36401a0080 con 0x7f3640100ac0 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.650+0000 7f363d7fa700 1 -- 172.21.15.150:0/1626260739 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630003cb0 con 0x7f3640100ac0 2024-05-10T04:40:18.654 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.650+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36401a0590 con 0x7f3640100ac0 2024-05-10T04:40:18.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.652+0000 7f363d7fa700 1 -- 172.21.15.150:0/1626260739 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f3630018430 con 0x7f3640100ac0 2024-05-10T04:40:18.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.652+0000 7f363d7fa700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36280a1260 0x7f36280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f363d7fa700 1 -- 172.21.15.150:0/1626260739 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3630014070 con 0x7f3640100ac0 2024-05-10T04:40:18.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f363f7fe700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36280a1260 0x7f36280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 Processor -- start 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 -- start start 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f363f7fe700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36280a1260 0x7f36280a3770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36400687f0 tx=0x7f363400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30bab0 0x55ef6f30bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30c980 0x55ef6f30cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30d2d0 0x55ef6f313bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f22e150 con 0x55ef6f30bab0 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f22bb10 con 0x55ef6f30c980 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f36569b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f227400 con 0x55ef6f30d2d0 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f363ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30bab0 0x55ef6f30bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f363ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30bab0 0x55ef6f30bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55522/0 (socket says 172.21.15.150:55522) 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f363ffff700 1 -- 172.21.15.150:0/823491382 learned_addr learned my addr 172.21.15.150:0/823491382 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.653+0000 7f363f7fe700 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30c980 0x55ef6f30cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f3644c11700 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30d2d0 0x55ef6f313bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f363ffff700 1 -- 172.21.15.150:0/823491382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30d2d0 msgr2=0x55ef6f313bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f363ffff700 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30d2d0 0x55ef6f313bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f363ffff700 1 -- 172.21.15.150:0/823491382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30c980 msgr2=0x55ef6f30cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f363ffff700 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30c980 0x55ef6f30cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f363ffff700 1 -- 172.21.15.150:0/823491382 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef6f0a83b0 con 0x55ef6f30bab0 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f363ffff700 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30bab0 0x55ef6f30bf00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3630003730 tx=0x7f3630099ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d95d7619aac63b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f36267fc700 1 -- 172.21.15.150:0/823491382 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36300b1070 con 0x55ef6f30bab0 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f36267fc700 1 -- 172.21.15.150:0/823491382 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363000cd30 con 0x55ef6f30bab0 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.654+0000 7f36267fc700 1 -- 172.21.15.150:0/823491382 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363001f340 con 0x55ef6f30bab0 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 -- 172.21.15.150:0/823491382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30bab0 msgr2=0x55ef6f30bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30bab0 0x55ef6f30bf00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3630003730 tx=0x7f3630099ba0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 -- 172.21.15.150:0/823491382 shutdown_connections 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30d2d0 0x55ef6f313bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30bab0 0x55ef6f30bf00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 --2- 172.21.15.150:0/823491382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30c980 0x55ef6f30cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 -- 172.21.15.150:0/823491382 >> 172.21.15.150:0/823491382 conn(0x55ef6f269a60 msgr2=0x55ef6f301fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 -- 172.21.15.150:0/823491382 shutdown_connections 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 -- 172.21.15.150:0/823491382 wait complete. 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 -- start start 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.655+0000 7f36569b8b80 1 Processor -- start 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f315b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f316080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f36569b8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f1c3560 con 0x55ef6f30d2d0 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f36569b8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f1c3200 con 0x55ef6f30bab0 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f36569b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f0aa6a0 con 0x55ef6f30c980 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f36569b8b80 10 client.0 _ll_register_callbacks cb 0x55ef6f1ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55538/0 (socket says 172.21.15.150:55538) 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f363ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f315b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 -- 172.21.15.150:0/2928098596 learned_addr learned my addr 172.21.15.150:0/2928098596 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f363f7fe700 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f316080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 -- 172.21.15.150:0/2928098596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 msgr2=0x55ef6f316080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f316080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 -- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 msgr2=0x55ef6f315b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f315b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 -- 172.21.15.150:0/2928098596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36401a0590 con 0x55ef6f30d2d0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f363ffff700 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f315b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f3644c11700 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31cfe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f36380077f0 tx=0x7f3638007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd9c3f7303dd082 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.656+0000 7f361f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f361f7fe700 1 -- 172.21.15.150:0/2928098596 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800e040 con 0x55ef6f30d2d0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f361f7fe700 1 -- 172.21.15.150:0/2928098596 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3638004a50 con 0x55ef6f30d2d0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef6f180270 con 0x55ef6f30d2d0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f361f7fe700 1 -- 172.21.15.150:0/2928098596 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3638004c10 con 0x55ef6f30d2d0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 10 client.15873 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ef6f054b60 con 0x55ef6f30d2d0 2024-05-10T04:40:18.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f361f7fe700 1 -- 172.21.15.150:0/2928098596 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef6f180270 con 0x55ef6f30d2d0 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f361f7fe700 1 -- 172.21.15.150:0/2928098596 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f363800b0d0 con 0x55ef6f30d2d0 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 20 client.15873 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f321760 0x55ef6f323bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 4 client.15873 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef6f2d6780 con 0x55ef6f321760 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.657+0000 7f363ffff700 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f321760 0x55ef6f323bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.658+0000 7f361f7fe700 10 client.15873 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.658+0000 7f363ffff700 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f321760 0x55ef6f323bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddfacc1d00f54b91 server_cookie=9db3aca10748b6d8 in_seq=0 out_seq=0 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f361f7fe700 1 -- 172.21.15.150:0/2928098596 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef6f2d6780 con 0x55ef6f321760 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f361f7fe700 10 client.15873 handle_command_reply: tid=0 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 client.15873 shutdown 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 msgr2=0x55ef6f31cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31cfe0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f36380077f0 tx=0x7f3638007820 comp rx=0 tx=0).stop 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 shutdown_connections 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f321760 0x55ef6f323bb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31cfe0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f361f7fe700 0 client.15873 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f315b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2928098596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f316080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 >> 172.21.15.150:0/2928098596 conn(0x55ef6f269a60 msgr2=0x55ef6f301fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 shutdown_connections 2024-05-10T04:40:18.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 1 -- 172.21.15.150:0/2928098596 wait complete. 2024-05-10T04:40:18.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.659+0000 7f36569b8b80 20 client.15873 trim_cache size 0 max 16384 2024-05-10T04:40:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 Processor -- start 2024-05-10T04:40:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 -- start start 2024-05-10T04:40:18.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30bab0 0x55ef6f323b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30c980 0x55ef6f31cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30d2d0 0x55ef6f30c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f313f20 con 0x55ef6f30c980 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f30c650 con 0x55ef6f30d2d0 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f36569b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f30c810 con 0x55ef6f30bab0 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f363f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30c980 0x55ef6f31cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f363f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30c980 0x55ef6f31cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55544/0 (socket says 172.21.15.150:55544) 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f363f7fe700 1 -- 172.21.15.150:0/3856530369 learned_addr learned my addr 172.21.15.150:0/3856530369 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.672+0000 7f3644c11700 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30d2d0 0x55ef6f30c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f363f7fe700 1 -- 172.21.15.150:0/3856530369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30bab0 msgr2=0x55ef6f323b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f363ffff700 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30bab0 0x55ef6f323b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f363f7fe700 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30bab0 0x55ef6f323b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f363f7fe700 1 -- 172.21.15.150:0/3856530369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30d2d0 msgr2=0x55ef6f30c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f363f7fe700 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30d2d0 0x55ef6f30c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f363f7fe700 1 -- 172.21.15.150:0/3856530369 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef6f1ed560 con 0x55ef6f30c980 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f3644c11700 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30d2d0 0x55ef6f30c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f363f7fe700 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30c980 0x55ef6f31cfa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3634011760 tx=0x7f3634013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34d555847096822 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f3626ffd700 1 -- 172.21.15.150:0/3856530369 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634024070 con 0x55ef6f30c980 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f3626ffd700 1 -- 172.21.15.150:0/3856530369 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3634015070 con 0x55ef6f30c980 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.673+0000 7f3626ffd700 1 -- 172.21.15.150:0/3856530369 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363401f760 con 0x55ef6f30c980 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 -- 172.21.15.150:0/3856530369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30c980 msgr2=0x55ef6f31cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30c980 0x55ef6f31cfa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3634011760 tx=0x7f3634013000 comp rx=0 tx=0).stop 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 -- 172.21.15.150:0/3856530369 shutdown_connections 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30d2d0 0x55ef6f30c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30bab0 0x55ef6f323b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 --2- 172.21.15.150:0/3856530369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30c980 0x55ef6f31cfa0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 -- 172.21.15.150:0/3856530369 >> 172.21.15.150:0/3856530369 conn(0x55ef6f269a60 msgr2=0x55ef6f301f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 -- 172.21.15.150:0/3856530369 shutdown_connections 2024-05-10T04:40:18.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 -- 172.21.15.150:0/3856530369 wait complete. 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 -- start start 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.674+0000 7f36569b8b80 1 Processor -- start 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f31a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f31a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f31bc20 con 0x55ef6f30d2d0 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f31bd90 con 0x55ef6f30bab0 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6f31bf50 con 0x55ef6f30c980 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 10 client.0 _ll_register_callbacks cb 0x55ef6f1ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55556/0 (socket says 172.21.15.150:55556) 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 -- 172.21.15.150:0/2100953709 learned_addr learned my addr 172.21.15.150:0/2100953709 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f363ffff700 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f31a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f363f7fe700 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f31a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 -- 172.21.15.150:0/2100953709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 msgr2=0x55ef6f31a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f31a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 -- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 msgr2=0x55ef6f31a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f31a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 -- 172.21.15.150:0/2100953709 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3634011440 con 0x55ef6f30d2d0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f3644c11700 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31b6e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3638009fd0 tx=0x7f3638007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4abedec347b785 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f361f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f361f7fe700 1 -- 172.21.15.150:0/2100953709 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800e040 con 0x55ef6f30d2d0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f361f7fe700 1 -- 172.21.15.150:0/2100953709 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f363801b9e0 con 0x55ef6f30d2d0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.675+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f363000cd30 con 0x55ef6f30d2d0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f361f7fe700 1 -- 172.21.15.150:0/2100953709 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363801bba0 con 0x55ef6f30d2d0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 10 client.15879 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ef6f0accd0 con 0x55ef6f30d2d0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f361f7fe700 1 -- 172.21.15.150:0/2100953709 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363000cd30 con 0x55ef6f30d2d0 2024-05-10T04:40:18.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f361f7fe700 1 -- 172.21.15.150:0/2100953709 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3638030420 con 0x55ef6f30d2d0 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 20 client.15879 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f377c30 0x55ef6f37a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 4 client.15879 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ef6f21fec0 con 0x55ef6f377c30 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.676+0000 7f363ffff700 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f377c30 0x55ef6f37a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f361f7fe700 10 client.15879 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f363ffff700 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f377c30 0x55ef6f37a050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2b2e603b384206 server_cookie=760c6abefa38598b in_seq=0 out_seq=0 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f361f7fe700 1 -- 172.21.15.150:0/2100953709 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ef6f21fec0 con 0x55ef6f377c30 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f361f7fe700 10 client.15879 handle_command_reply: tid=0 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 client.15879 shutdown 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 msgr2=0x55ef6f31b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31b6e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3638009fd0 tx=0x7f3638007940 comp rx=0 tx=0).stop 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 shutdown_connections 2024-05-10T04:40:18.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ef6f377c30 0x55ef6f37a050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef6f30d2d0 0x55ef6f31b6e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef6f30bab0 0x55ef6f31a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f361f7fe700 0 client.15879 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 --2- 172.21.15.150:0/2100953709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ef6f30c980 0x55ef6f31a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 >> 172.21.15.150:0/2100953709 conn(0x55ef6f269a60 msgr2=0x55ef6f314110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 shutdown_connections 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 1 -- 172.21.15.150:0/2100953709 wait complete. 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.677+0000 7f36569b8b80 20 client.15879 trim_cache size 0 max 16384 2024-05-10T04:40:18.686 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:18.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36280a1260 msgr2=0x7f36280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36280a1260 0x7f36280a3770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36400687f0 tx=0x7f363400b410 comp rx=0 tx=0).stop 2024-05-10T04:40:18.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640100ac0 msgr2=0x7f3640068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:18.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640100ac0 0x7f3640068e50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f363000e940 tx=0x7f363000ec50 comp rx=0 tx=0).stop 2024-05-10T04:40:18.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 shutdown_connections 2024-05-10T04:40:18.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3640102580 0x7f364019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640100ac0 0x7f3640068e50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f36280a1260 0x7f36280a3770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 --2- 172.21.15.150:0/1626260739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3640101be0 0x7f364019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:18.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 >> 172.21.15.150:0/1626260739 conn(0x7f36400fc0d0 msgr2=0x7f3640104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:18.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 shutdown_connections 2024-05-10T04:40:18.688 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:18.678+0000 7f3646674700 1 -- 172.21.15.150:0/1626260739 wait complete. 2024-05-10T04:40:18.690 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:18 smithi150 ceph-mon[42404]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-10T04:40:18.690 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:18 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1023454815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:40:18.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:18 smithi161 ceph-mon[46912]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-10T04:40:18.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:18 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1023454815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:40:18.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:18 smithi181 ceph-mon[47314]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-10T04:40:18.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:18 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1023454815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:40:19.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:40:19.692 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:40:19.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:19 smithi150 ceph-mon[42404]: pgmap v294: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 11 op/s 2024-05-10T04:40:19.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:19 smithi150 ceph-mon[42404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-10T04:40:19.723 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:19 smithi150 ceph-mon[42404]: Cluster is now healthy 2024-05-10T04:40:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:19 smithi161 ceph-mon[46912]: pgmap v294: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 11 op/s 2024-05-10T04:40:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:19 smithi161 ceph-mon[46912]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-10T04:40:19.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:19 smithi161 ceph-mon[46912]: Cluster is now healthy 2024-05-10T04:40:19.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 -- 172.21.15.150:0/2450260566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 msgr2=0x7f44401071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 --2- 172.21.15.150:0/2450260566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f44401071f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4434009990 tx=0x7f4434009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 -- 172.21.15.150:0/2450260566 shutdown_connections 2024-05-10T04:40:19.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 --2- 172.21.15.150:0/2450260566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4440107880 0x7f4440109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 --2- 172.21.15.150:0/2450260566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4440069090 0x7f4440104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 --2- 172.21.15.150:0/2450260566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f44401071f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 -- 172.21.15.150:0/2450260566 >> 172.21.15.150:0/2450260566 conn(0x7f44400fc0d0 msgr2=0x7f44400fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 -- 172.21.15.150:0/2450260566 shutdown_connections 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 -- 172.21.15.150:0/2450260566 wait complete. 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.844+0000 7f4446035700 1 Processor -- start 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f4446035700 1 -- start start 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f4446035700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4440069090 0x7f4440197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f4446035700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f4440198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f4446035700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4440107880 0x7f444019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f4446035700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019fa10 con 0x7f4440104d10 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f4446035700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019fb80 con 0x7f4440107880 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f4446035700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019fd40 con 0x7f4440069090 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f4440198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f4440198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55574/0 (socket says 172.21.15.150:55574) 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 -- 172.21.15.150:0/1195566816 learned_addr learned my addr 172.21.15.150:0/1195566816 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443f7fe700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4440069090 0x7f4440197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443ffff700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4440107880 0x7f444019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 -- 172.21.15.150:0/1195566816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4440069090 msgr2=0x7f4440197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4440069090 0x7f4440197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 -- 172.21.15.150:0/1195566816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4440107880 msgr2=0x7f444019f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4440107880 0x7f444019f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 -- 172.21.15.150:0/1195566816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44340095f0 con 0x7f4440104d10 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.845+0000 7f443effd700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f4440198470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4434000c00 tx=0x7f4434004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.846+0000 7f443cff9700 1 -- 172.21.15.150:0/1195566816 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443401d070 con 0x7f4440104d10 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.846+0000 7f443cff9700 1 -- 172.21.15.150:0/1195566816 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4434021410 con 0x7f4440104d10 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.846+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444019ffc0 con 0x7f4440104d10 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.846+0000 7f443cff9700 1 -- 172.21.15.150:0/1195566816 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434017650 con 0x7f4440104d10 2024-05-10T04:40:19.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.846+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44401a0380 con 0x7f4440104d10 2024-05-10T04:40:19.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.848+0000 7f443cff9700 1 -- 172.21.15.150:0/1195566816 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f44340177b0 con 0x7f4440104d10 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.849+0000 7f443cff9700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44280a12e0 0x7f44280a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.849+0000 7f443cff9700 1 -- 172.21.15.150:0/1195566816 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f443400fbf0 con 0x7f4440104d10 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.849+0000 7f443f7fe700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44280a12e0 0x7f44280a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.849+0000 7f443f7fe700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44280a12e0 0x7f44280a37f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f44400694a0 tx=0x7f4430009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 Processor -- start 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 -- start start 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dc9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dcaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dd1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8cf01d0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8cedb90 con 0x55d2c8dcaf40 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f4456379b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8ce9480 con 0x55d2c8dc9720 2024-05-10T04:40:19.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dcaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dcaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55584/0 (socket says 172.21.15.150:55584) 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.850+0000 7f443effd700 1 -- 172.21.15.150:0/715430899 learned_addr learned my addr 172.21.15.150:0/715430899 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443f7fe700 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dc9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443ffff700 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dd1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443effd700 1 -- 172.21.15.150:0/715430899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 msgr2=0x55d2c8dc9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443effd700 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dc9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443effd700 1 -- 172.21.15.150:0/715430899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 msgr2=0x55d2c8dd1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443effd700 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dd1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443effd700 1 -- 172.21.15.150:0/715430899 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2c8b6a430 con 0x55d2c8dca5f0 2024-05-10T04:40:19.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443ffff700 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dd1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f443effd700 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dcaa00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4434098690 tx=0x7f4434098600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e6fdb31c9848e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f4425ffb700 1 -- 172.21.15.150:0/715430899 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443409c190 con 0x55d2c8dca5f0 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f4425ffb700 1 -- 172.21.15.150:0/715430899 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4434021e90 con 0x55d2c8dca5f0 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.851+0000 7f4425ffb700 1 -- 172.21.15.150:0/715430899 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443409c650 con 0x55d2c8dca5f0 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 -- 172.21.15.150:0/715430899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 msgr2=0x55d2c8dcaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dcaa00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4434098690 tx=0x7f4434098600 comp rx=0 tx=0).stop 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 -- 172.21.15.150:0/715430899 shutdown_connections 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dd1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dc9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 --2- 172.21.15.150:0/715430899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dcaa00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 -- 172.21.15.150:0/715430899 >> 172.21.15.150:0/715430899 conn(0x55d2c8d2bae0 msgr2=0x55d2c8dc4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 -- 172.21.15.150:0/715430899 shutdown_connections 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 -- 172.21.15.150:0/715430899 wait complete. 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.852+0000 7f4456379b80 1 -- start start 2024-05-10T04:40:19.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 1 Processor -- start 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dd33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dda860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8c855e0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8c85280 con 0x55d2c8dcaf40 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8b6c720 con 0x55d2c8dc9720 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f4456379b80 10 client.0 _ll_register_callbacks cb 0x55d2c8c8ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55592/0 (socket says 172.21.15.150:55592) 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 -- 172.21.15.150:0/363248295 learned_addr learned my addr 172.21.15.150:0/363248295 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443ffff700 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dda860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443f7fe700 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dd33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 -- 172.21.15.150:0/363248295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 msgr2=0x55d2c8dd33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dd33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 -- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 msgr2=0x55d2c8dda860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dda860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 -- 172.21.15.150:0/363248295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44401a0380 con 0x55d2c8dca5f0 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f443effd700 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd3900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4434028000 tx=0x7f4434005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f1de6884ba8271 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.853+0000 7f441effd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f441effd700 1 -- 172.21.15.150:0/363248295 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443409ffc0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f441effd700 1 -- 172.21.15.150:0/363248295 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f44340a4580 con 0x55d2c8dca5f0 2024-05-10T04:40:19.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2c8c422f0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f441effd700 1 -- 172.21.15.150:0/363248295 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f443409c190 con 0x55d2c8dca5f0 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 10 client.15891 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d2c8b16be0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f441effd700 1 -- 172.21.15.150:0/363248295 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2c8c422f0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f441effd700 1 -- 172.21.15.150:0/363248295 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f443409b0f0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 20 client.15891 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8ddeaf0 0x55d2c8de0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 4 client.15891 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2c8d98800 con 0x55d2c8ddeaf0 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.854+0000 7f443f7fe700 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8ddeaf0 0x55d2c8de0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.855+0000 7f441effd700 10 client.15891 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.855+0000 7f443f7fe700 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8ddeaf0 0x55d2c8de0f40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca113fcd3caef36 server_cookie=61c5b53deee688cf in_seq=0 out_seq=0 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.855+0000 7f441effd700 1 -- 172.21.15.150:0/363248295 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d2c8d98800 con 0x55d2c8ddeaf0 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.855+0000 7f441effd700 10 client.15891 handle_command_reply: tid=0 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.855+0000 7f4456379b80 1 client.15891 shutdown 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 msgr2=0x55d2c8dd3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd3900 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4434028000 tx=0x7f4434005910 comp rx=0 tx=0).stop 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 shutdown_connections 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8ddeaf0 0x55d2c8de0f40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dcaf40 0x55d2c8dda860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dd33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f441effd700 0 client.15891 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 --2- 172.21.15.150:0/363248295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd3900 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 >> 172.21.15.150:0/363248295 conn(0x55d2c8d2bae0 msgr2=0x55d2c8dc4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:19.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 shutdown_connections 2024-05-10T04:40:19.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 1 -- 172.21.15.150:0/363248295 wait complete. 2024-05-10T04:40:19.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.856+0000 7f4456379b80 20 client.15891 trim_cache size 0 max 16384 2024-05-10T04:40:19.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 Processor -- start 2024-05-10T04:40:19.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 -- start start 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8d98800 con 0x55d2c8dc9720 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8c422f0 con 0x55d2c8dca5f0 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f4456379b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8dc0db0 con 0x55d2c8d2bae0 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.873+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55598/0 (socket says 172.21.15.150:55598) 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 -- 172.21.15.150:0/3058485420 learned_addr learned my addr 172.21.15.150:0/3058485420 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443ffff700 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443f7fe700 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 -- 172.21.15.150:0/3058485420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8d2bae0 msgr2=0x55d2c8dce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 -- 172.21.15.150:0/3058485420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 msgr2=0x55d2c8dce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 -- 172.21.15.150:0/3058485420 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2c8caf5e0 con 0x55d2c8dc9720 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443ffff700 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f443effd700 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda820 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f443409c120 tx=0x7f443409c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9849c739c649f4d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f44267fc700 1 -- 172.21.15.150:0/3058485420 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44340ba6f0 con 0x55d2c8dc9720 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f44267fc700 1 -- 172.21.15.150:0/3058485420 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44340a4580 con 0x55d2c8dc9720 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.874+0000 7f44267fc700 1 -- 172.21.15.150:0/3058485420 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44340a20d0 con 0x55d2c8dc9720 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 -- 172.21.15.150:0/3058485420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dc9720 msgr2=0x55d2c8dda820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda820 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f443409c120 tx=0x7f443409c310 comp rx=0 tx=0).stop 2024-05-10T04:40:19.878 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 -- 172.21.15.150:0/3058485420 shutdown_connections 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 --2- 172.21.15.150:0/3058485420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda820 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 -- 172.21.15.150:0/3058485420 >> 172.21.15.150:0/3058485420 conn(0x55d2c8dc04d0 msgr2=0x55d2c8dd33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 -- 172.21.15.150:0/3058485420 shutdown_connections 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 -- 172.21.15.150:0/3058485420 wait complete. 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 -- start start 2024-05-10T04:40:19.879 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.875+0000 7f4456379b80 1 Processor -- start 2024-05-10T04:40:19.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f4456379b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dda170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.884 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f4456379b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f4456379b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f4456379b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8dd8ea0 con 0x55d2c8d2bae0 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f4456379b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8dd9010 con 0x55d2c8dca5f0 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f4456379b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2c8dd91d0 con 0x55d2c8dc9720 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f4456379b80 10 client.0 _ll_register_callbacks cb 0x55d2c8c8ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dda170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dda170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55610/0 (socket says 172.21.15.150:55610) 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 -- 172.21.15.150:0/320892971 learned_addr learned my addr 172.21.15.150:0/320892971 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:19.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443effd700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443ffff700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 -- 172.21.15.150:0/320892971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 msgr2=0x55d2c8dda6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 -- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 msgr2=0x55d2c8dd8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 -- 172.21.15.150:0/320892971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2c8b16be0 con 0x55d2c8d2bae0 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443ffff700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f443f7fe700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dda170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55d2c8de16e0 tx=0x7f4430013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2978a9debc0542d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.876+0000 7f441effd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f441effd700 1 -- 172.21.15.150:0/320892971 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430023040 con 0x55d2c8d2bae0 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f441effd700 1 -- 172.21.15.150:0/320892971 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4430004220 con 0x55d2c8d2bae0 2024-05-10T04:40:19.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2c8dc52c0 con 0x55d2c8d2bae0 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f441effd700 1 -- 172.21.15.150:0/320892971 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44300043e0 con 0x55d2c8d2bae0 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 10 client.15897 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d2c8b6ed50 con 0x55d2c8d2bae0 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f441effd700 1 -- 172.21.15.150:0/320892971 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2c8dc52c0 con 0x55d2c8d2bae0 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f441effd700 1 -- 172.21.15.150:0/320892971 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4430003b60 con 0x55d2c8d2bae0 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 20 client.15897 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8dcaf40 0x55d2c8e38230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 4 client.15897 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f4434021e90 con 0x55d2c8dcaf40 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.877+0000 7f443effd700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8dcaf40 0x55d2c8e38230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.878+0000 7f441effd700 10 client.15897 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.878+0000 7f443effd700 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8dcaf40 0x55d2c8e38230 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3ac1cc062a040 server_cookie=d0fe37da80145b86 in_seq=0 out_seq=0 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.879+0000 7f441effd700 1 -- 172.21.15.150:0/320892971 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f4434021e90 con 0x55d2c8dcaf40 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.879+0000 7f441effd700 10 client.15897 handle_command_reply: tid=0 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.879+0000 7f4456379b80 1 client.15897 shutdown 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.879+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8d2bae0 msgr2=0x55d2c8dda170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.879+0000 7f4456379b80 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dda170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55d2c8de16e0 tx=0x7f4430013040 comp rx=0 tx=0).stop 2024-05-10T04:40:19.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 shutdown_connections 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55d2c8dcaf40 0x55d2c8e38230 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2c8dca5f0 0x55d2c8dd8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2c8d2bae0 0x55d2c8dda170 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 --2- 172.21.15.150:0/320892971 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d2c8dc9720 0x55d2c8dda6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f441effd700 0 client.15897 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 >> 172.21.15.150:0/320892971 conn(0x55d2c8dc04d0 msgr2=0x55d2c8dd19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 shutdown_connections 2024-05-10T04:40:19.889 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 1 -- 172.21.15.150:0/320892971 wait complete. 2024-05-10T04:40:19.890 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4456379b80 20 client.15897 trim_cache size 0 max 16384 2024-05-10T04:40:19.890 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:40:19.890 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:40:19.890 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "4f00eadb-d9c2-4ccb-868e-ad4f2fd1202f", 2024-05-10T04:40:19.890 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:40:19.891 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:40:19.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44280a12e0 msgr2=0x7f44280a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44280a12e0 0x7f44280a37f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f44400694a0 tx=0x7f4430009260 comp rx=0 tx=0).stop 2024-05-10T04:40:19.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 msgr2=0x7f4440198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:19.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f4440198470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4434000c00 tx=0x7f4434004730 comp rx=0 tx=0).stop 2024-05-10T04:40:19.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 shutdown_connections 2024-05-10T04:40:19.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4440107880 0x7f444019f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44280a12e0 0x7f44280a37f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4440069090 0x7f4440197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 --2- 172.21.15.150:0/1195566816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4440104d10 0x7f4440198470 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:19.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 >> 172.21.15.150:0/1195566816 conn(0x7f44400fc0d0 msgr2=0x7f4440102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:19.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 shutdown_connections 2024-05-10T04:40:19.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:19.880+0000 7f4446035700 1 -- 172.21.15.150:0/1195566816 wait complete. 2024-05-10T04:40:19.894 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:40:19.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:19 smithi181 ceph-mon[47314]: pgmap v294: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 11 op/s 2024-05-10T04:40:19.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:19 smithi181 ceph-mon[47314]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-10T04:40:19.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:19 smithi181 ceph-mon[47314]: Cluster is now healthy 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 -- 172.21.15.150:0/3312040365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12701050a0 msgr2=0x7f12701074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 --2- 172.21.15.150:0/3312040365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12701050a0 0x7f12701074c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1260009990 tx=0x7f1260009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 -- 172.21.15.150:0/3312040365 shutdown_connections 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 --2- 172.21.15.150:0/3312040365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1270107a00 0x7f1270109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 --2- 172.21.15.150:0/3312040365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12700fff30 0x7f1270100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 --2- 172.21.15.150:0/3312040365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12701050a0 0x7f12701074c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 -- 172.21.15.150:0/3312040365 >> 172.21.15.150:0/3312040365 conn(0x7f12700fb880 msgr2=0x7f12700fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:20.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 -- 172.21.15.150:0/3312040365 shutdown_connections 2024-05-10T04:40:20.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.070+0000 7f12755d0700 1 -- 172.21.15.150:0/3312040365 wait complete. 2024-05-10T04:40:20.075 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 Processor -- start 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 -- start start 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12700fff30 0x7f127019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12701050a0 0x7f127019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1270107a00 0x7f12701a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701a1af0 con 0x7f1270107a00 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701a1c30 con 0x7f12700fff30 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.071+0000 7f12755d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701a1dc0 con 0x7f12701050a0 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1270107a00 0x7f12701a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.076 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1270107a00 0x7f12701a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55626/0 (socket says 172.21.15.150:55626) 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 -- 172.21.15.150:0/3404080528 learned_addr learned my addr 172.21.15.150:0/3404080528 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126effd700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12700fff30 0x7f127019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126e7fc700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12701050a0 0x7f127019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 -- 172.21.15.150:0/3404080528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12701050a0 msgr2=0x7f127019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12701050a0 0x7f127019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 -- 172.21.15.150:0/3404080528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12700fff30 msgr2=0x7f127019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12700fff30 0x7f127019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 -- 172.21.15.150:0/3404080528 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12600095f0 con 0x7f1270107a00 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f126f7fe700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1270107a00 0x7f12701a1520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1268007f10 tx=0x7f126800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:20.077 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f1257fff700 1 -- 172.21.15.150:0/3404080528 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268017070 con 0x7f1270107a00 2024-05-10T04:40:20.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f1257fff700 1 -- 172.21.15.150:0/3404080528 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f126800f040 con 0x7f1270107a00 2024-05-10T04:40:20.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12701a20a0 con 0x7f1270107a00 2024-05-10T04:40:20.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f1257fff700 1 -- 172.21.15.150:0/3404080528 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268005610 con 0x7f1270107a00 2024-05-10T04:40:20.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.072+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12701a2560 con 0x7f1270107a00 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.075+0000 7f1257fff700 1 -- 172.21.15.150:0/3404080528 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f1268004ad0 con 0x7f1270107a00 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.075+0000 7f1257fff700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12580a1260 0x7f12580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.075+0000 7f1257fff700 1 -- 172.21.15.150:0/3404080528 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f12680cc4f0 con 0x7f1270107a00 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.075+0000 7f126effd700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12580a1260 0x7f12580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f126effd700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12580a1260 0x7f12580a3770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1270103020 tx=0x7f126400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 Processor -- start 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 -- start start 2024-05-10T04:40:20.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b236a0 0x55bfc9b23af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24570 0x55bfc9b24980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9a4a150 con 0x55bfc9b236a0 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9a47b10 con 0x55bfc9b24ec0 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.076+0000 7f1285914b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9a43400 con 0x55bfc9b24570 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b236a0 0x55bfc9b23af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b236a0 0x55bfc9b23af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55636/0 (socket says 172.21.15.150:55636) 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 -- 172.21.15.150:0/2101572668 learned_addr learned my addr 172.21.15.150:0/2101572668 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126f7fe700 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126e7fc700 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24570 0x55bfc9b24980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 -- 172.21.15.150:0/2101572668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24570 msgr2=0x55bfc9b24980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24570 0x55bfc9b24980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 -- 172.21.15.150:0/2101572668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 msgr2=0x55bfc9b2b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 -- 172.21.15.150:0/2101572668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc98c43b0 con 0x55bfc9b236a0 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f126effd700 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b236a0 0x55bfc9b23af0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f126400fea0 tx=0x7f1264012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ced07b4c7a90f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f12557fa700 1 -- 172.21.15.150:0/2101572668 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1264003b10 con 0x55bfc9b236a0 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f12557fa700 1 -- 172.21.15.150:0/2101572668 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1264003c70 con 0x55bfc9b236a0 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.077+0000 7f12557fa700 1 -- 172.21.15.150:0/2101572668 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126401d660 con 0x55bfc9b236a0 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 -- 172.21.15.150:0/2101572668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b236a0 msgr2=0x55bfc9b23af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b236a0 0x55bfc9b23af0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f126400fea0 tx=0x7f1264012000 comp rx=0 tx=0).stop 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 -- 172.21.15.150:0/2101572668 shutdown_connections 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b236a0 0x55bfc9b23af0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 --2- 172.21.15.150:0/2101572668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24570 0x55bfc9b24980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 -- 172.21.15.150:0/2101572668 >> 172.21.15.150:0/2101572668 conn(0x55bfc9a85a60 msgr2=0x55bfc9b1dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 -- 172.21.15.150:0/2101572668 shutdown_connections 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 -- 172.21.15.150:0/2101572668 wait complete. 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.078+0000 7f1285914b80 1 -- start start 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 1 Processor -- start 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b2dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b34b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc99df560 con 0x55bfc9b24570 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc99df200 con 0x55bfc9b24ec0 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc98c66a0 con 0x55bfc9b236a0 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1285914b80 10 client.0 _ll_register_callbacks cb 0x55bfc99e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b2dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b2dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55652/0 (socket says 172.21.15.150:55652) 2024-05-10T04:40:20.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 -- 172.21.15.150:0/1403594526 learned_addr learned my addr 172.21.15.150:0/1403594526 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126f7fe700 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126effd700 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 -- 172.21.15.150:0/1403594526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 msgr2=0x55bfc9b2d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b2d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 -- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 msgr2=0x55bfc9b34b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b34b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 -- 172.21.15.150:0/1403594526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126400fb50 con 0x55bfc9b24570 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f126e7fc700 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b2dc10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1260009990 tx=0x7f1260009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f947e64ad9a4c484 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.079+0000 7f1252ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1252ff5700 1 -- 172.21.15.150:0/1403594526 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260005520 con 0x55bfc9b24570 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1252ff5700 1 -- 172.21.15.150:0/1403594526 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1260005680 con 0x55bfc9b24570 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfc999c270 con 0x55bfc9b24570 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1252ff5700 1 -- 172.21.15.150:0/1403594526 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1260003890 con 0x55bfc9b24570 2024-05-10T04:40:20.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 10 client.15909 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bfc9870b60 con 0x55bfc9b24570 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1252ff5700 1 -- 172.21.15.150:0/1403594526 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfc999c270 con 0x55bfc9b24570 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1252ff5700 1 -- 172.21.15.150:0/1403594526 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f126000fba0 con 0x55bfc9b24570 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 20 client.15909 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b3b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 4 client.15909 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfc9af2780 con 0x55bfc9b39420 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.080+0000 7f126effd700 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b3b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.081+0000 7f126effd700 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b3b870 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda7c7172d636306 server_cookie=93b69603a18d8a92 in_seq=0 out_seq=0 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.081+0000 7f1252ff5700 10 client.15909 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:20.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.081+0000 7f1252ff5700 1 -- 172.21.15.150:0/1403594526 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfc9af2780 con 0x55bfc9b39420 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.081+0000 7f1252ff5700 10 client.15909 handle_command_reply: tid=0 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.081+0000 7f1285914b80 1 client.15909 shutdown 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 msgr2=0x55bfc9b2dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b2dc10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1260009990 tx=0x7f1260009ee0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 shutdown_connections 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b3b870 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b34b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1252ff5700 0 client.15909 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b2d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 --2- 172.21.15.150:0/1403594526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b2dc10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 >> 172.21.15.150:0/1403594526 conn(0x55bfc9a85a60 msgr2=0x55bfc9b1dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 shutdown_connections 2024-05-10T04:40:20.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 1 -- 172.21.15.150:0/1403594526 wait complete. 2024-05-10T04:40:20.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.082+0000 7f1285914b80 20 client.15909 trim_cache size 0 max 16384 2024-05-10T04:40:20.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 Processor -- start 2024-05-10T04:40:20.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 -- start start 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b236a0 0x55bfc9b32b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b33060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9b35610 con 0x55bfc9b24570 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9b243f0 con 0x55bfc9b236a0 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f1285914b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9b1f7f0 con 0x55bfc9b24ec0 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b33060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b33060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55660/0 (socket says 172.21.15.150:55660) 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 -- 172.21.15.150:0/1584319814 learned_addr learned my addr 172.21.15.150:0/1584319814 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126effd700 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b236a0 0x55bfc9b32b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126f7fe700 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 -- 172.21.15.150:0/1584319814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24ec0 msgr2=0x55bfc9b2d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 -- 172.21.15.150:0/1584319814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b236a0 msgr2=0x55bfc9b32b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b236a0 0x55bfc9b32b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126e7fc700 1 -- 172.21.15.150:0/1584319814 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc9a09560 con 0x55bfc9b24570 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.099+0000 7f126effd700 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b236a0 0x55bfc9b32b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f126e7fc700 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b33060 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1260003730 tx=0x7f1260009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ac35312d021ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1255ffb700 1 -- 172.21.15.150:0/1584319814 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260022530 con 0x55bfc9b24570 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1255ffb700 1 -- 172.21.15.150:0/1584319814 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f126001e070 con 0x55bfc9b24570 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1255ffb700 1 -- 172.21.15.150:0/1584319814 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260024ab0 con 0x55bfc9b24570 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 -- 172.21.15.150:0/1584319814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 msgr2=0x55bfc9b33060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b33060 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1260003730 tx=0x7f1260009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 -- 172.21.15.150:0/1584319814 shutdown_connections 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b2d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b236a0 0x55bfc9b32b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 --2- 172.21.15.150:0/1584319814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b33060 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 -- 172.21.15.150:0/1584319814 >> 172.21.15.150:0/1584319814 conn(0x55bfc9a85a60 msgr2=0x55bfc9b2cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.100+0000 7f1285914b80 1 -- 172.21.15.150:0/1584319814 shutdown_connections 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 -- 172.21.15.150:0/1584319814 wait complete. 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 -- start start 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 Processor -- start 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b8bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b8e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9b333a0 con 0x55bfc9b24570 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9b8e880 con 0x55bfc9b24ec0 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc9b8e9c0 con 0x55bfc9b236a0 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f1285914b80 10 client.0 _ll_register_callbacks cb 0x55bfc99e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f126e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b8bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f126e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b8bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55672/0 (socket says 172.21.15.150:55672) 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.101+0000 7f126e7fc700 1 -- 172.21.15.150:0/4163907674 learned_addr learned my addr 172.21.15.150:0/4163907674 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126effd700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126f7fe700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b8e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126e7fc700 1 -- 172.21.15.150:0/4163907674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 msgr2=0x55bfc9b898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126e7fc700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126e7fc700 1 -- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 msgr2=0x55bfc9b8e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126e7fc700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b8e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126e7fc700 1 -- 172.21.15.150:0/4163907674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfc9870b60 con 0x55bfc9b24570 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126effd700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f126e7fc700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b8bdf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1260024490 tx=0x7f12600246c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a001d2cb329aa server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1252ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:20.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1252ff5700 1 -- 172.21.15.150:0/4163907674 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260005320 con 0x55bfc9b24570 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1252ff5700 1 -- 172.21.15.150:0/4163907674 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1260000e40 con 0x55bfc9b24570 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1264003c70 con 0x55bfc9b24570 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1252ff5700 1 -- 172.21.15.150:0/4163907674 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1260005480 con 0x55bfc9b24570 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1285914b80 10 client.15915 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bfc98c8cd0 con 0x55bfc9b24570 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.102+0000 7f1252ff5700 1 -- 172.21.15.150:0/4163907674 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1264003c70 con 0x55bfc9b24570 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1252ff5700 1 -- 172.21.15.150:0/4163907674 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12600229b0 con 0x55bfc9b24570 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1285914b80 10 client.15915 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:20.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1285914b80 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1285914b80 20 client.15915 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1285914b80 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b91fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1285914b80 4 client.15915 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bfc9a3bec0 con 0x55bfc9b39420 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f126effd700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b91fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1252ff5700 10 client.15915 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f126effd700 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b91fd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f4bb9b62c0898f server_cookie=b12fc6f22aa01ae4 in_seq=0 out_seq=0 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1252ff5700 1 -- 172.21.15.150:0/4163907674 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bfc9a3bec0 con 0x55bfc9b39420 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.103+0000 7f1252ff5700 10 client.15915 handle_command_reply: tid=0 2024-05-10T04:40:20.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 client.15915 shutdown 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 msgr2=0x55bfc9b8bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b8bdf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1260024490 tx=0x7f12600246c0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 shutdown_connections 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55bfc9b39420 0x55bfc9b91fd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bfc9b24ec0 0x55bfc9b8e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bfc9b236a0 0x55bfc9b898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 --2- 172.21.15.150:0/4163907674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bfc9b24570 0x55bfc9b8bdf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1252ff5700 0 client.15915 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:20.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 >> 172.21.15.150:0/4163907674 conn(0x55bfc9a85a60 msgr2=0x55bfc9b2bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 shutdown_connections 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 1 -- 172.21.15.150:0/4163907674 wait complete. 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f1285914b80 20 client.15915 trim_cache size 0 max 16384 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stdout: "4f00eadb-d9c2-4ccb-868e-ad4f2fd1202f": { 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:40:20.117 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "4f00eadb-d9c2-4ccb-868e-ad4f2fd1202f", 2024-05-10T04:40:20.118 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:40:20.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:40:20.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:40:20.118 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:40:20.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12580a1260 msgr2=0x7f12580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12580a1260 0x7f12580a3770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1270103020 tx=0x7f126400a380 comp rx=0 tx=0).stop 2024-05-10T04:40:20.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1270107a00 msgr2=0x7f12701a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:20.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1270107a00 0x7f12701a1520 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1268007f10 tx=0x7f126800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 shutdown_connections 2024-05-10T04:40:20.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1270107a00 0x7f12701a1520 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12700fff30 0x7f127019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12580a1260 0x7f12580a3770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 --2- 172.21.15.150:0/3404080528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12701050a0 0x7f127019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:20.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 >> 172.21.15.150:0/3404080528 conn(0x7f12700fb880 msgr2=0x7f12701017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:20.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 shutdown_connections 2024-05-10T04:40:20.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:20.104+0000 7f12755d0700 1 -- 172.21.15.150:0/3404080528 wait complete. 2024-05-10T04:40:20.120 INFO:tasks.cephfs.filesystem:scrub status for tag:4f00eadb-d9c2-4ccb-868e-ad4f2fd1202f - {'path': '/', 'tag': '4f00eadb-d9c2-4ccb-868e-ad4f2fd1202f', 'options': 'recursive,force'} 2024-05-10T04:40:21.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:20 smithi181 ceph-mon[47314]: scrub queued for path: / 2024-05-10T04:40:21.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:20 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:21.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:20 smithi181 ceph-mon[47314]: scrub summary: active paths [/] 2024-05-10T04:40:21.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:20 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:21.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:20 smithi150 ceph-mon[42404]: scrub queued for path: / 2024-05-10T04:40:21.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:20 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:21.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:20 smithi150 ceph-mon[42404]: scrub summary: active paths [/] 2024-05-10T04:40:21.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:20 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:21.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:20 smithi161 ceph-mon[46912]: scrub queued for path: / 2024-05-10T04:40:21.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:20 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:21.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:20 smithi161 ceph-mon[46912]: scrub summary: active paths [/] 2024-05-10T04:40:21.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:20 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:22.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:21 smithi181 ceph-mon[47314]: pgmap v295: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 343 B/s wr, 9 op/s 2024-05-10T04:40:22.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:21 smithi150 ceph-mon[42404]: pgmap v295: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 343 B/s wr, 9 op/s 2024-05-10T04:40:22.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:21 smithi161 ceph-mon[46912]: pgmap v295: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 343 B/s wr, 9 op/s 2024-05-10T04:40:23.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:22 smithi181 ceph-mon[47314]: scrub summary: idle 2024-05-10T04:40:23.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:22 smithi150 ceph-mon[42404]: scrub summary: idle 2024-05-10T04:40:23.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:22 smithi161 ceph-mon[46912]: scrub summary: idle 2024-05-10T04:40:24.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:23 smithi181 ceph-mon[47314]: pgmap v296: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 307 B/s wr, 11 op/s 2024-05-10T04:40:24.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:23 smithi150 ceph-mon[42404]: pgmap v296: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 307 B/s wr, 11 op/s 2024-05-10T04:40:24.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:23 smithi161 ceph-mon[46912]: pgmap v296: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 307 B/s wr, 11 op/s 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 -- 172.21.15.150:0/3240879375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254101a40 msgr2=0x7f6254101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 --2- 172.21.15.150:0/3240879375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254101a40 0x7f6254101ed0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6250009990 tx=0x7f6250009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 -- 172.21.15.150:0/3240879375 shutdown_connections 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 --2- 172.21.15.150:0/3240879375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6254102560 0x7f6254109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 --2- 172.21.15.150:0/3240879375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101060 0x7f6254101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 --2- 172.21.15.150:0/3240879375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254101a40 0x7f6254101ed0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 -- 172.21.15.150:0/3240879375 >> 172.21.15.150:0/3240879375 conn(0x7f62540fb840 msgr2=0x7f62540fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 -- 172.21.15.150:0/3240879375 shutdown_connections 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.298+0000 7f625c1d9700 1 -- 172.21.15.150:0/3240879375 wait complete. 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 Processor -- start 2024-05-10T04:40:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 -- start start 2024-05-10T04:40:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101060 0x7f6254100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6254101a40 0x7f625419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254102560 0x7f625419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625419f9f0 con 0x7f6254102560 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625419fb60 con 0x7f6254101060 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625c1d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625419fd20 con 0x7f6254101a40 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254102560 0x7f625419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254102560 0x7f625419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55682/0 (socket says 172.21.15.150:55682) 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 -- 172.21.15.150:0/65400261 learned_addr learned my addr 172.21.15.150:0/65400261 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f6259774700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6254101a40 0x7f625419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f6259f75700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101060 0x7f6254100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 -- 172.21.15.150:0/65400261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6254101a40 msgr2=0x7f625419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6254101a40 0x7f625419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 -- 172.21.15.150:0/65400261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101060 msgr2=0x7f6254100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101060 0x7f6254100740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f625a776700 1 -- 172.21.15.150:0/65400261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62500095f0 con 0x7f6254102560 2024-05-10T04:40:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.299+0000 7f6259f75700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101060 0x7f6254100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.300+0000 7f625a776700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254102560 0x7f625419f2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f624c00ef30 tx=0x7f624c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.300+0000 7f624affd700 1 -- 172.21.15.150:0/65400261 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c018070 con 0x7f6254102560 2024-05-10T04:40:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.300+0000 7f624affd700 1 -- 172.21.15.150:0/65400261 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f624c00f040 con 0x7f6254102560 2024-05-10T04:40:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.300+0000 7f624affd700 1 -- 172.21.15.150:0/65400261 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c00cdb0 con 0x7f6254102560 2024-05-10T04:40:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.300+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625419ffa0 con 0x7f6254102560 2024-05-10T04:40:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.300+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62541a03d0 con 0x7f6254102560 2024-05-10T04:40:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.302+0000 7f624affd700 1 -- 172.21.15.150:0/65400261 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f624c003a00 con 0x7f6254102560 2024-05-10T04:40:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.303+0000 7f624affd700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62400a1230 0x7f62400a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.303+0000 7f6259f75700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62400a1230 0x7f62400a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.303+0000 7f624affd700 1 -- 172.21.15.150:0/65400261 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f624c0cc9b0 con 0x7f6254102560 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 Processor -- start 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f6259f75700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62400a1230 0x7f62400a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6254068370 tx=0x7f6244005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 -- start start 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0b650 0x555ea0f0ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f0c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f13680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0e2e0f0 con 0x555ea0f0cd80 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0e2bab0 con 0x555ea0f0c430 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f626c51db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0e273a0 con 0x555ea0f0b650 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f625a776700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f13680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f625a776700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f13680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55690/0 (socket says 172.21.15.150:55690) 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f625a776700 1 -- 172.21.15.150:0/861466863 learned_addr learned my addr 172.21.15.150:0/861466863 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f6259f75700 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0b650 0x555ea0f0ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.304+0000 7f6259774700 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f0c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f625a776700 1 -- 172.21.15.150:0/861466863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0b650 msgr2=0x555ea0f0ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f625a776700 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0b650 0x555ea0f0ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f625a776700 1 -- 172.21.15.150:0/861466863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 msgr2=0x555ea0f0c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f625a776700 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f0c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f625a776700 1 -- 172.21.15.150:0/861466863 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea0ca8350 con 0x555ea0f0cd80 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f625a776700 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f13680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f624c0090a0 tx=0x7f624c099880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7152c5341f7d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f623ffff700 1 -- 172.21.15.150:0/861466863 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c004ad0 con 0x555ea0f0cd80 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f623ffff700 1 -- 172.21.15.150:0/861466863 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f624c01b320 con 0x555ea0f0cd80 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.305+0000 7f623ffff700 1 -- 172.21.15.150:0/861466863 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c097e10 con 0x555ea0f0cd80 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 -- 172.21.15.150:0/861466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 msgr2=0x555ea0f13680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f13680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f624c0090a0 tx=0x7f624c099880 comp rx=0 tx=0).stop 2024-05-10T04:40:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 -- 172.21.15.150:0/861466863 shutdown_connections 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f13680 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0b650 0x555ea0f0ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 --2- 172.21.15.150:0/861466863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f0c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 -- 172.21.15.150:0/861466863 >> 172.21.15.150:0/861466863 conn(0x555ea0e69a00 msgr2=0x555ea0f01a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 -- 172.21.15.150:0/861466863 shutdown_connections 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 -- 172.21.15.150:0/861466863 wait complete. 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 -- start start 2024-05-10T04:40:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.306+0000 7f626c51db80 1 Processor -- start 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f626c51db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f15620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f626c51db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0c430 0x555ea0f15b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f626c51db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f626c51db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0dc3500 con 0x555ea0f0c430 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f626c51db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0dc31a0 con 0x555ea0f0b650 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f626c51db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0caa640 con 0x555ea0f0cd80 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f626c51db80 10 client.0 _ll_register_callbacks cb 0x555ea0dccad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0c430 0x555ea0f15b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0c430 0x555ea0f15b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55692/0 (socket says 172.21.15.150:55692) 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 -- 172.21.15.150:0/1668619811 learned_addr learned my addr 172.21.15.150:0/1668619811 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f625a776700 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259f75700 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f15620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 -- 172.21.15.150:0/1668619811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 msgr2=0x555ea0f1cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 -- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 msgr2=0x555ea0f15620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f15620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 -- 172.21.15.150:0/1668619811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62541a03d0 con 0x555ea0f0c430 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f6259774700 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0c430 0x555ea0f15b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f625000c010 tx=0x7f6250005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be13d93d19feda0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.307+0000 7f623d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f623d7fa700 1 -- 172.21.15.150:0/1668619811 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625001d070 con 0x555ea0f0c430 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f623d7fa700 1 -- 172.21.15.150:0/1668619811 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f6250004700 con 0x555ea0f0c430 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ea0d80210 con 0x555ea0f0c430 2024-05-10T04:40:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f623d7fa700 1 -- 172.21.15.150:0/1668619811 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6250005710 con 0x555ea0f0c430 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 10 client.15927 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555ea0c54b00 con 0x555ea0f0c430 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f623d7fa700 1 -- 172.21.15.150:0/1668619811 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ea0d80210 con 0x555ea0f0c430 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f623d7fa700 1 -- 172.21.15.150:0/1668619811 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f625000f9f0 con 0x555ea0f0c430 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 10 client.15927 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 20 client.15927 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f21320 0x555ea0f23770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 4 client.15927 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ea0ed6720 con 0x555ea0f21320 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.308+0000 7f6259f75700 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f21320 0x555ea0f23770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.309+0000 7f6259f75700 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f21320 0x555ea0f23770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a10f7da17bd8596 server_cookie=cae9054be514cc98 in_seq=0 out_seq=0 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.309+0000 7f623d7fa700 10 client.15927 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f623d7fa700 1 -- 172.21.15.150:0/1668619811 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ea0ed6720 con 0x555ea0f21320 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f623d7fa700 10 client.15927 handle_command_reply: tid=0 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 client.15927 shutdown 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0c430 msgr2=0x555ea0f15b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0c430 0x555ea0f15b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f625000c010 tx=0x7f6250005900 comp rx=0 tx=0).stop 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 shutdown_connections 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f21320 0x555ea0f23770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f15620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 --2- 172.21.15.150:0/1668619811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0c430 0x555ea0f15b60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f623d7fa700 0 client.15927 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 >> 172.21.15.150:0/1668619811 conn(0x555ea0e69a00 msgr2=0x555ea0f01a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 shutdown_connections 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 1 -- 172.21.15.150:0/1668619811 wait complete. 2024-05-10T04:40:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.310+0000 7f626c51db80 20 client.15927 trim_cache size 0 max 16384 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 Processor -- start 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 -- start start 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f1d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0c430 0x555ea0f1c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0f13640 con 0x555ea0f0cd80 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0f10ed0 con 0x555ea0f0b650 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f626c51db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0f210a0 con 0x555ea0f0c430 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f155e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55704/0 (socket says 172.21.15.150:55704) 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 -- 172.21.15.150:0/2001780022 learned_addr learned my addr 172.21.15.150:0/2001780022 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f6259774700 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0c430 0x555ea0f1c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f6259f75700 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f1d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 -- 172.21.15.150:0/2001780022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0c430 msgr2=0x555ea0f1c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0c430 0x555ea0f1c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 -- 172.21.15.150:0/2001780022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 msgr2=0x555ea0f1d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f1d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f625a776700 1 -- 172.21.15.150:0/2001780022 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea0ded500 con 0x555ea0f0cd80 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f6259774700 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0c430 0x555ea0f1c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.325+0000 7f6259f75700 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f1d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f625a776700 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f155e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f624c099920 tx=0x7f624c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4899d23b330c471f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f6248ff9700 1 -- 172.21.15.150:0/2001780022 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c09a090 con 0x555ea0f0cd80 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f6248ff9700 1 -- 172.21.15.150:0/2001780022 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f624c0b2270 con 0x555ea0f0cd80 2024-05-10T04:40:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f6248ff9700 1 -- 172.21.15.150:0/2001780022 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c097b60 con 0x555ea0f0cd80 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 -- 172.21.15.150:0/2001780022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 msgr2=0x555ea0f155e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f155e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f624c099920 tx=0x7f624c0092e0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 -- 172.21.15.150:0/2001780022 shutdown_connections 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0cd80 0x555ea0f155e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0b650 0x555ea0f1d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 --2- 172.21.15.150:0/2001780022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0c430 0x555ea0f1c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 -- 172.21.15.150:0/2001780022 >> 172.21.15.150:0/2001780022 conn(0x555ea0e69a00 msgr2=0x555ea0f14b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.326+0000 7f626c51db80 1 -- 172.21.15.150:0/2001780022 shutdown_connections 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 -- 172.21.15.150:0/2001780022 wait complete. 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 -- start start 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 Processor -- start 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0b650 0x555ea0f19d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f1a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0f1b730 con 0x555ea0f0b650 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0f1b8a0 con 0x555ea0f0c430 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea0f1ba60 con 0x555ea0f0cd80 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f626c51db80 10 client.0 _ll_register_callbacks cb 0x555ea0dccad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f6259f75700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0b650 0x555ea0f19d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f6259f75700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0b650 0x555ea0f19d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55716/0 (socket says 172.21.15.150:55716) 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.327+0000 7f6259f75700 1 -- 172.21.15.150:0/424457549 learned_addr learned my addr 172.21.15.150:0/424457549 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f625a776700 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f6259774700 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f1a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f6259f75700 1 -- 172.21.15.150:0/424457549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 msgr2=0x555ea0f1b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f6259f75700 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f6259f75700 1 -- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 msgr2=0x555ea0f1a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f6259f75700 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f1a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f6259f75700 1 -- 172.21.15.150:0/424457549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ea0ca8350 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f6259f75700 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0b650 0x555ea0f19d80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x555ea0f02860 tx=0x7f62440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb2a97ad65fb331 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f623d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f623d7fa700 1 -- 172.21.15.150:0/424457549 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624401e6b0 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f623d7fa700 1 -- 172.21.15.150:0/424457549 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f6244026e40 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f624c01b320 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f623d7fa700 1 -- 172.21.15.150:0/424457549 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6244004830 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f626c51db80 10 client.15933 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555ea0cacc70 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.328+0000 7f623d7fa700 1 -- 172.21.15.150:0/424457549 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f624c01b320 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f623d7fa700 1 -- 172.21.15.150:0/424457549 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6244005020 con 0x555ea0f0b650 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f626c51db80 10 client.15933 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f626c51db80 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f626c51db80 20 client.15933 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f626c51db80 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f77b10 0x555ea0f79f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f626c51db80 4 client.15933 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:40:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ea0e1fe60 con 0x555ea0f77b10 2024-05-10T04:40:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f6259774700 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f77b10 0x555ea0f79f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f623d7fa700 10 client.15933 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.329+0000 7f6259774700 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f77b10 0x555ea0f79f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce331e53434ea0fd server_cookie=d550dcc70a5d11d2 in_seq=0 out_seq=0 2024-05-10T04:40:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f623d7fa700 1 -- 172.21.15.150:0/424457549 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ea0e1fe60 con 0x555ea0f77b10 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f623d7fa700 10 client.15933 handle_command_reply: tid=0 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 client.15933 shutdown 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0b650 msgr2=0x555ea0f19d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0b650 0x555ea0f19d80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x555ea0f02860 tx=0x7f62440047c0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 shutdown_connections 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555ea0f77b10 0x555ea0f79f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555ea0f0cd80 0x555ea0f1b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ea0f0b650 0x555ea0f19d80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f623d7fa700 0 client.15933 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 --2- 172.21.15.150:0/424457549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555ea0f0c430 0x555ea0f1a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 >> 172.21.15.150:0/424457549 conn(0x555ea0e69a00 msgr2=0x555ea0f138b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 shutdown_connections 2024-05-10T04:40:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 1 -- 172.21.15.150:0/424457549 wait complete. 2024-05-10T04:40:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.337+0000 7f626c51db80 20 client.15933 trim_cache size 0 max 16384 2024-05-10T04:40:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:40:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:40:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:40:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62400a1230 msgr2=0x7f62400a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62400a1230 0x7f62400a3740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6254068370 tx=0x7f6244005f90 comp rx=0 tx=0).stop 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254102560 msgr2=0x7f625419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254102560 0x7f625419f2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f624c00ef30 tx=0x7f624c00c3a0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 shutdown_connections 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6254102560 0x7f625419f2f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6254101060 0x7f6254100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62400a1230 0x7f62400a3740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 --2- 172.21.15.150:0/65400261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6254101a40 0x7f625419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 >> 172.21.15.150:0/65400261 conn(0x7f62540fb840 msgr2=0x7f6254105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 shutdown_connections 2024-05-10T04:40:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:26.338+0000 7f625c1d9700 1 -- 172.21.15.150:0/65400261 wait complete. 2024-05-10T04:40:26.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:26 smithi181 ceph-mon[47314]: pgmap v297: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 257 B/s wr, 10 op/s 2024-05-10T04:40:26.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:26 smithi150 ceph-mon[42404]: pgmap v297: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 257 B/s wr, 10 op/s 2024-05-10T04:40:26.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:26 smithi161 ceph-mon[46912]: pgmap v297: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 257 B/s wr, 10 op/s 2024-05-10T04:40:28.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:28 smithi150 ceph-mon[42404]: pgmap v298: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:40:28.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:28 smithi161 ceph-mon[46912]: pgmap v298: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:40:28.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:28 smithi181 ceph-mon[47314]: pgmap v298: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:40:30.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:30 smithi150 ceph-mon[42404]: pgmap v299: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-05-10T04:40:30.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:30 smithi161 ceph-mon[46912]: pgmap v299: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-05-10T04:40:30.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:30 smithi181 ceph-mon[47314]: pgmap v299: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-05-10T04:40:32.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:32 smithi161 ceph-mon[46912]: pgmap v300: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:32.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:32 smithi181 ceph-mon[47314]: pgmap v300: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:32.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:32 smithi150 ceph-mon[42404]: pgmap v300: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:34 smithi161 ceph-mon[46912]: pgmap v301: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:34.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:34 smithi181 ceph-mon[47314]: pgmap v301: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:34.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:34 smithi150 ceph-mon[42404]: pgmap v301: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:36.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:36 smithi161 ceph-mon[46912]: pgmap v302: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:36.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:36 smithi181 ceph-mon[47314]: pgmap v302: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:36.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:36 smithi150 ceph-mon[42404]: pgmap v302: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-10T04:40:38.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:38 smithi161 ceph-mon[46912]: pgmap v303: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:40:38.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:38 smithi181 ceph-mon[47314]: pgmap v303: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:40:38.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:38 smithi150 ceph-mon[42404]: pgmap v303: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-10T04:40:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 -- 172.21.15.150:0/2044886631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8102580 msgr2=0x7f9cd8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/2044886631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8102580 0x7f9cd8108e30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000b0a0 tx=0x7f9cd000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 -- 172.21.15.150:0/2044886631 shutdown_connections 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/2044886631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8102580 0x7f9cd8108e30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/2044886631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd8100ac0 0x7f9cd8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/2044886631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cd8101be0 0x7f9cd8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 -- 172.21.15.150:0/2044886631 >> 172.21.15.150:0/2044886631 conn(0x7f9cd80fc0f0 msgr2=0x7f9cd80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 -- 172.21.15.150:0/2044886631 shutdown_connections 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 -- 172.21.15.150:0/2044886631 wait complete. 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.470+0000 7f9cdf15e700 1 Processor -- start 2024-05-10T04:40:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdf15e700 1 -- start start 2024-05-10T04:40:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdf15e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8100ac0 0x7f9cd8104880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdf15e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cd8101be0 0x7f9cd8102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdf15e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd8102580 0x7f9cd8103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdf15e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8103ac0 con 0x7f9cd8100ac0 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdf15e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8103c30 con 0x7f9cd8101be0 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdf15e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8103df0 con 0x7f9cd8102580 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8100ac0 0x7f9cd8104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8100ac0 0x7f9cd8104880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43972/0 (socket says 172.21.15.150:43972) 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 -- 172.21.15.150:0/339002578 learned_addr learned my addr 172.21.15.150:0/339002578 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd8102580 0x7f9cd8103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cd8101be0 0x7f9cd8102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 -- 172.21.15.150:0/339002578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd8102580 msgr2=0x7f9cd8103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd8102580 0x7f9cd8103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 -- 172.21.15.150:0/339002578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cd8101be0 msgr2=0x7f9cd8102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cd8101be0 0x7f9cd8102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 -- 172.21.15.150:0/339002578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd0009d00 con 0x7f9cd8100ac0 2024-05-10T04:40:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.471+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8100ac0 0x7f9cd8104880 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00e940 tx=0x7f9ccc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.472+0000 7f9cd5ffb700 1 -- 172.21.15.150:0/339002578 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc004220 con 0x7f9cd8100ac0 2024-05-10T04:40:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.472+0000 7f9cd5ffb700 1 -- 172.21.15.150:0/339002578 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc00cca0 con 0x7f9cd8100ac0 2024-05-10T04:40:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.472+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd81a2370 con 0x7f9cd8100ac0 2024-05-10T04:40:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.472+0000 7f9cd5ffb700 1 -- 172.21.15.150:0/339002578 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc003c00 con 0x7f9cd8100ac0 2024-05-10T04:40:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.472+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd81a2880 con 0x7f9cd8100ac0 2024-05-10T04:40:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.475+0000 7f9cd5ffb700 1 -- 172.21.15.150:0/339002578 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 135925+0+0 (secure 0 0 0) 0x7f9ccc00c7e0 con 0x7f9cd8100ac0 2024-05-10T04:40:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.475+0000 7f9cd5ffb700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.475+0000 7f9cd5ffb700 1 -- 172.21.15.150:0/339002578 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9ccc014070 con 0x7f9cd8100ac0 2024-05-10T04:40:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.475+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8104220 tx=0x7f9cc40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 Processor -- start 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 -- start start 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94996180 0x55fe949965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe94997460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe949979a0 0x55fe9499e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe948bd0f0 con 0x55fe94996180 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe948baab0 con 0x55fe94997050 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe948b63a0 con 0x55fe949979a0 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cdcefa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94996180 0x55fe949965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cdcefa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94996180 0x55fe949965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43984/0 (socket says 172.21.15.150:43984) 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.476+0000 7f9cdcefa700 1 -- 172.21.15.150:0/3038285005 learned_addr learned my addr 172.21.15.150:0/3038285005 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe949979a0 0x55fe9499e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe94997460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cdcefa700 1 -- 172.21.15.150:0/3038285005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe949979a0 msgr2=0x55fe9499e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe949979a0 0x55fe9499e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cdcefa700 1 -- 172.21.15.150:0/3038285005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 msgr2=0x55fe94997460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe94997460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cdcefa700 1 -- 172.21.15.150:0/3038285005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe94737350 con 0x55fe94996180 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe94997460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94996180 0x55fe949965b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc099f30 tx=0x7f9ccc099e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534a422a93543bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cc2ffd700 1 -- 172.21.15.150:0/3038285005 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0b1070 con 0x55fe94996180 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cc2ffd700 1 -- 172.21.15.150:0/3038285005 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc09a6a0 con 0x55fe94996180 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.477+0000 7f9cc2ffd700 1 -- 172.21.15.150:0/3038285005 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc098340 con 0x55fe94996180 2024-05-10T04:40:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3038285005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94996180 msgr2=0x55fe949965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94996180 0x55fe949965b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc099f30 tx=0x7f9ccc099e40 comp rx=0 tx=0).stop 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3038285005 shutdown_connections 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe949979a0 0x55fe9499e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94996180 0x55fe949965b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3038285005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe94997460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3038285005 >> 172.21.15.150:0/3038285005 conn(0x55fe948f8a00 msgr2=0x55fe94990a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3038285005 shutdown_connections 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3038285005 wait complete. 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.478+0000 7f9cef4a2b80 1 -- start start 2024-05-10T04:40:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 1 Processor -- start 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe949a0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94997050 0x55fe949a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe949979a0 0x55fe949a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe94852500 con 0x55fe94997050 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe948521a0 con 0x55fe949979a0 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe94739640 con 0x55fe94996180 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cef4a2b80 10 client.0 _ll_register_callbacks cb 0x55fe9485bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94997050 0x55fe949a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94997050 0x55fe949a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43992/0 (socket says 172.21.15.150:43992) 2024-05-10T04:40:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 -- 172.21.15.150:0/2778035477 learned_addr learned my addr 172.21.15.150:0/2778035477 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe949a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe949979a0 0x55fe949a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 -- 172.21.15.150:0/2778035477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 msgr2=0x55fe949a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe949a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 -- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe949979a0 msgr2=0x55fe949a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe949979a0 0x55fe949a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.479+0000 7f9cd7fff700 1 -- 172.21.15.150:0/2778035477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd81a2880 con 0x55fe94997050 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94997050 0x55fe949a0790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40114c0 tx=0x7f9cc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd5d3a1011e6505 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2778035477 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4027670 con 0x55fe94997050 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2778035477 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9cc4017070 con 0x55fe94997050 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe9480f210 con 0x55fe94997050 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2778035477 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc4011e70 con 0x55fe94997050 2024-05-10T04:40:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 10 client.15945 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe946e3b00 con 0x55fe94997050 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2778035477 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe9480f210 con 0x55fe94997050 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2778035477 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cc401e9f0 con 0x55fe94997050 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 20 client.15945 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe949abef0 0x55fe949ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 4 client.15945 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.480+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe94965720 con 0x55fe949abef0 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.481+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe949abef0 0x55fe949ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.481+0000 7f9cb7fff700 10 client.15945 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.481+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe949abef0 0x55fe949ae340 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba54c14f3b70aab server_cookie=5ce6f82d8bc46b8b in_seq=0 out_seq=0 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2778035477 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe94965720 con 0x55fe949abef0 2024-05-10T04:40:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cb7fff700 10 client.15945 handle_command_reply: tid=0 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 client.15945 shutdown 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94997050 msgr2=0x55fe949a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94997050 0x55fe949a0790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40114c0 tx=0x7f9cc40047c0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 shutdown_connections 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe949abef0 0x55fe949ae340 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe949979a0 0x55fe949a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe949a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2778035477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe94997050 0x55fe949a0790 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cb7fff700 0 client.15945 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 >> 172.21.15.150:0/2778035477 conn(0x55fe948f8a00 msgr2=0x55fe94990a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 shutdown_connections 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2778035477 wait complete. 2024-05-10T04:40:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.482+0000 7f9cef4a2b80 20 client.15945 trim_cache size 0 max 16384 2024-05-10T04:40:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 Processor -- start 2024-05-10T04:40:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 -- start start 2024-05-10T04:40:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe9498cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe949a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe949889e0 con 0x55fe949979a0 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe94996a10 con 0x55fe94997050 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9499e2c0 con 0x55fe94996180 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cdd6fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cdd6fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44004/0 (socket says 172.21.15.150:44004) 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/3270143444 learned_addr learned my addr 172.21.15.150:0/3270143444 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe949a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe9498cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.498+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/3270143444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 msgr2=0x55fe9498cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe9498cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/3270143444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 msgr2=0x55fe949a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe949a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/3270143444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe9487c500 con 0x55fe949979a0 2024-05-10T04:40:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe949a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a0230 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0015000 tx=0x7f9cd0009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373c7db7ebd23b25 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cc37fe700 1 -- 172.21.15.150:0/3270143444 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000e040 con 0x55fe949979a0 2024-05-10T04:40:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cc37fe700 1 -- 172.21.15.150:0/3270143444 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cd0003a00 con 0x55fe949979a0 2024-05-10T04:40:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.499+0000 7f9cc37fe700 1 -- 172.21.15.150:0/3270143444 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0005440 con 0x55fe949979a0 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3270143444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 msgr2=0x55fe949a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a0230 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0015000 tx=0x7f9cd0009430 comp rx=0 tx=0).stop 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3270143444 shutdown_connections 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a0230 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94996180 0x55fe9498cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/3270143444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94997050 0x55fe949a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3270143444 >> 172.21.15.150:0/3270143444 conn(0x55fe948f8a00 msgr2=0x55fe9499f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3270143444 shutdown_connections 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/3270143444 wait complete. 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 -- start start 2024-05-10T04:40:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 Processor -- start 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.500+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94996180 0x55fe949a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94997050 0x55fe949a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe949a6840 con 0x55fe949979a0 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe949a69b0 con 0x55fe94996180 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe949a6b70 con 0x55fe94997050 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 10 client.0 _ll_register_callbacks cb 0x55fe9485bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44008/0 (socket says 172.21.15.150:44008) 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/2922252019 learned_addr learned my addr 172.21.15.150:0/2922252019 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cd7fff700 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94997050 0x55fe949a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94996180 0x55fe949a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/2922252019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94997050 msgr2=0x55fe949a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94997050 0x55fe949a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94996180 msgr2=0x55fe949a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94996180 0x55fe949a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 -- 172.21.15.150:0/2922252019 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd81098c0 con 0x55fe949979a0 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cdd6fb700 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a6300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009a20 tx=0x7f9cd000bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bff342d8fa0ee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2922252019 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000e040 con 0x55fe949979a0 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2922252019 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9cd00055d0 con 0x55fe949979a0 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ccc09a6a0 con 0x55fe949979a0 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2922252019 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cd000be90 con 0x55fe949979a0 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 10 client.15951 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.501+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe9473bc70 con 0x55fe949979a0 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2922252019 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ccc09a6a0 con 0x55fe949979a0 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2922252019 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cd000b0d0 con 0x55fe949979a0 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cef4a2b80 10 client.15951 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cef4a2b80 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cef4a2b80 20 client.15951 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe94a027c0 0x55fe94a04be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cef4a2b80 4 client.15951 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe948aee60 con 0x55fe94a027c0 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.502+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe94a027c0 0x55fe94a04be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.503+0000 7f9cb7fff700 10 client.15951 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.503+0000 7f9cdcefa700 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe94a027c0 0x55fe94a04be0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa671f5106d83c65 server_cookie=2eaf541136b3b319 in_seq=0 out_seq=0 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cb7fff700 1 -- 172.21.15.150:0/2922252019 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe948aee60 con 0x55fe94a027c0 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cb7fff700 10 client.15951 handle_command_reply: tid=0 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 client.15951 shutdown 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 msgr2=0x55fe949a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a6300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009a20 tx=0x7f9cd000bb30 comp rx=0 tx=0).stop 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 shutdown_connections 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55fe94a027c0 0x55fe94a04be0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe949979a0 0x55fe949a6300 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe94996180 0x55fe949a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 --2- 172.21.15.150:0/2922252019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fe94997050 0x55fe949a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.510 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cb7fff700 0 client.15951 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.506+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 >> 172.21.15.150:0/2922252019 conn(0x55fe948f8a00 msgr2=0x55fe9499e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 shutdown_connections 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cef4a2b80 1 -- 172.21.15.150:0/2922252019 wait complete. 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cef4a2b80 20 client.15951 trim_cache size 0 max 16384 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:40:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:40:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cc80a11b0 msgr2=0x7f9cc80a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8104220 tx=0x7f9cc40073d0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8100ac0 msgr2=0x7f9cd8104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8100ac0 0x7f9cd8104880 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00e940 tx=0x7f9ccc00ec50 comp rx=0 tx=0).stop 2024-05-10T04:40:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 shutdown_connections 2024-05-10T04:40:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9cc80a11b0 0x7f9cc80a36c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cd8102580 0x7f9cd8103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd8100ac0 0x7f9cd8104880 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 --2- 172.21.15.150:0/339002578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cd8101be0 0x7f9cd8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 >> 172.21.15.150:0/339002578 conn(0x7f9cd80fc0f0 msgr2=0x7f9cd80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 shutdown_connections 2024-05-10T04:40:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:39.507+0000 7f9cdf15e700 1 -- 172.21.15.150:0/339002578 wait complete. 2024-05-10T04:40:40.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:40 smithi161 ceph-mon[46912]: pgmap v304: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-10T04:40:40.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:40 smithi181 ceph-mon[47314]: pgmap v304: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-10T04:40:40.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:40 smithi150 ceph-mon[42404]: pgmap v304: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-05-10T04:40:41.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:41 smithi161 ceph-mon[46912]: pgmap v305: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:40:41.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:41 smithi181 ceph-mon[47314]: pgmap v305: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:40:41.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:41 smithi150 ceph-mon[42404]: pgmap v305: 129 pgs: 129 active+clean; 670 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:40:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:43 smithi161 ceph-mon[46912]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-10T04:40:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:43 smithi161 ceph-mon[46912]: pgmap v306: 129 pgs: 129 active+clean; 671 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T04:40:43.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:43 smithi181 ceph-mon[47314]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-10T04:40:43.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:43 smithi181 ceph-mon[47314]: pgmap v306: 129 pgs: 129 active+clean; 671 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T04:40:43.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:43 smithi150 ceph-mon[42404]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-10T04:40:43.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:43 smithi150 ceph-mon[42404]: pgmap v306: 129 pgs: 129 active+clean; 671 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T04:40:46.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:46 smithi150 ceph-mon[42404]: pgmap v307: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:46.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:46 smithi161 ceph-mon[46912]: pgmap v307: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:46.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:46 smithi181 ceph-mon[47314]: pgmap v307: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:48.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:48 smithi150 ceph-mon[42404]: pgmap v308: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:48 smithi161 ceph-mon[46912]: pgmap v308: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:48.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:48 smithi181 ceph-mon[47314]: pgmap v308: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:50.121 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:40:50.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.260+0000 7f9d5bec7700 1 -- 172.21.15.150:0/3131546562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54107980 msgr2=0x7f9d54109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.260+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/3131546562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54107980 0x7f9d54109da0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400b0a0 tx=0x7f9d4400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.260+0000 7f9d5bec7700 1 -- 172.21.15.150:0/3131546562 shutdown_connections 2024-05-10T04:40:50.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.260+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/3131546562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54107980 0x7f9d54109da0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.260+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/3131546562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d541026c0 0x7f9d54104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.260+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/3131546562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d54105020 0x7f9d54107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.260+0000 7f9d5bec7700 1 -- 172.21.15.150:0/3131546562 >> 172.21.15.150:0/3131546562 conn(0x7f9d540fc0e0 msgr2=0x7f9d540fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 -- 172.21.15.150:0/3131546562 shutdown_connections 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 -- 172.21.15.150:0/3131546562 wait complete. 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 Processor -- start 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 -- start start 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d541026c0 0x7f9d54198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54105020 0x7f9d541985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d54107980 0x7f9d5419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5419fb70 con 0x7f9d54105020 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5419fce0 con 0x7f9d54107980 2024-05-10T04:40:50.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.261+0000 7f9d5bec7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5419fea0 con 0x7f9d541026c0 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54105020 0x7f9d541985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54105020 0x7f9d541985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37518/0 (socket says 172.21.15.150:37518) 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 -- 172.21.15.150:0/1844846168 learned_addr learned my addr 172.21.15.150:0/1844846168 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59c63700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d541026c0 0x7f9d54198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d5a464700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d54107980 0x7f9d5419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 -- 172.21.15.150:0/1844846168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d541026c0 msgr2=0x7f9d54198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d541026c0 0x7f9d54198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 -- 172.21.15.150:0/1844846168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d54107980 msgr2=0x7f9d5419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d54107980 0x7f9d5419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 -- 172.21.15.150:0/1844846168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44009d00 con 0x7f9d54105020 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d59462700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54105020 0x7f9d541985d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e9a0 tx=0x7f9d4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d52ffd700 1 -- 172.21.15.150:0/1844846168 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48009790 con 0x7f9d54105020 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d52ffd700 1 -- 172.21.15.150:0/1844846168 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4800cbd0 con 0x7f9d54105020 2024-05-10T04:40:50.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d541a0180 con 0x7f9d54105020 2024-05-10T04:40:50.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d52ffd700 1 -- 172.21.15.150:0/1844846168 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48018680 con 0x7f9d54105020 2024-05-10T04:40:50.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.262+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d541a06e0 con 0x7f9d54105020 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.265+0000 7f9d52ffd700 1 -- 172.21.15.150:0/1844846168 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9d480187e0 con 0x7f9d54105020 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.265+0000 7f9d52ffd700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d3c0a1240 0x7f9d3c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.266+0000 7f9d59c63700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d3c0a1240 0x7f9d3c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.266+0000 7f9d52ffd700 1 -- 172.21.15.150:0/1844846168 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9d48014070 con 0x7f9d54105020 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.266+0000 7f9d59c63700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d3c0a1240 0x7f9d3c0a3750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005950 tx=0x7f9d4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.266+0000 7f9d6c20bb80 1 Processor -- start 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.266+0000 7f9d6c20bb80 1 -- start start 2024-05-10T04:40:50.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.266+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d820f4150 con 0x555d821ceec0 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d820f1b10 con 0x555d821ce570 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d820ed400 con 0x555d821cd6a0 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821d57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37530/0 (socket says 172.21.15.150:37530) 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 -- 172.21.15.150:0/1001166495 learned_addr learned my addr 172.21.15.150:0/1001166495 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d59c63700 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d59462700 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 -- 172.21.15.150:0/1001166495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 msgr2=0x555d821cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 -- 172.21.15.150:0/1001166495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 msgr2=0x555d821ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 -- 172.21.15.150:0/1001166495 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d81f6e3b0 con 0x555d821ceec0 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d59c63700 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d5a464700 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821d57c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400b780 tx=0x7f9d44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6254fa10e8886760 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d3bfff700 1 -- 172.21.15.150:0/1001166495 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400e040 con 0x555d821ceec0 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d3bfff700 1 -- 172.21.15.150:0/1001166495 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4400bab0 con 0x555d821ceec0 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.267+0000 7f9d3bfff700 1 -- 172.21.15.150:0/1001166495 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4401b6a0 con 0x555d821ceec0 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/1001166495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 msgr2=0x555d821d57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821d57c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400b780 tx=0x7f9d44009780 comp rx=0 tx=0).stop 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/1001166495 shutdown_connections 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821d57c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/1001166495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/1001166495 >> 172.21.15.150:0/1001166495 conn(0x555d8212fa60 msgr2=0x555d821c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/1001166495 shutdown_connections 2024-05-10T04:40:50.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.268+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/1001166495 wait complete. 2024-05-10T04:40:50.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 -- start start 2024-05-10T04:40:50.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 Processor -- start 2024-05-10T04:40:50.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.276 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821deaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d82089560 con 0x555d821ceec0 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d82089200 con 0x555d821ce570 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81f706a0 con 0x555d821cd6a0 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d6c20bb80 10 client.0 _ll_register_callbacks cb 0x555d82092b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d5a464700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821deaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d5a464700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821deaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37540/0 (socket says 172.21.15.150:37540) 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.269+0000 7f9d5a464700 1 -- 172.21.15.150:0/434818057 learned_addr learned my addr 172.21.15.150:0/434818057 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d59462700 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d59c63700 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d5a464700 1 -- 172.21.15.150:0/434818057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 msgr2=0x555d821d7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d5a464700 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821d7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d5a464700 1 -- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 msgr2=0x555d821d7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d5a464700 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821d7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d5a464700 1 -- 172.21.15.150:0/434818057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5410a9c0 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d5a464700 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821deaf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009c50 tx=0x7f9d44007670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35516799118bf737 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d397fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d397fa700 1 -- 172.21.15.150:0/434818057 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400e040 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d397fa700 1 -- 172.21.15.150:0/434818057 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9d4401b850 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d82046270 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d397fa700 1 -- 172.21.15.150:0/434818057 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4401ba10 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d6c20bb80 10 client.15963 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555d81f1ab60 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.270+0000 7f9d397fa700 1 -- 172.21.15.150:0/434818057 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d82046270 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d397fa700 1 -- 172.21.15.150:0/434818057 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d44019040 con 0x555d821ceec0 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d6c20bb80 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d6c20bb80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:50.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d6c20bb80 20 client.15963 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d821e2dd0 0x555d821e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d6c20bb80 4 client.15963 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d8219c780 con 0x555d821e2dd0 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d59c63700 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d821e2dd0 0x555d821e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d397fa700 10 client.15963 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.271+0000 7f9d59c63700 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d821e2dd0 0x555d821e5220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebf45d204066ef5 server_cookie=5b3ed80208b4dac5 in_seq=0 out_seq=0 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d397fa700 1 -- 172.21.15.150:0/434818057 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d8219c780 con 0x555d821e2dd0 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d397fa700 10 client.15963 handle_command_reply: tid=0 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 client.15963 shutdown 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 msgr2=0x555d821deaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821deaf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009c50 tx=0x7f9d44007670 comp rx=0 tx=0).stop 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 shutdown_connections 2024-05-10T04:40:50.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d821e2dd0 0x555d821e5220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821ceec0 0x555d821deaf0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821cd6a0 0x555d821d7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d397fa700 0 client.15963 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/434818057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ce570 0x555d821d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 >> 172.21.15.150:0/434818057 conn(0x555d8212fa60 msgr2=0x555d821c7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 shutdown_connections 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/434818057 wait complete. 2024-05-10T04:40:50.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.272+0000 7f9d6c20bb80 20 client.15963 trim_cache size 0 max 16384 2024-05-10T04:40:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 Processor -- start 2024-05-10T04:40:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 -- start start 2024-05-10T04:40:50.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821deab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d821cf310 con 0x555d821cd6a0 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d821d8720 con 0x555d821ceec0 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d821e2c40 con 0x555d821ce570 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821d2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37542/0 (socket says 172.21.15.150:37542) 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 -- 172.21.15.150:0/596306203 learned_addr learned my addr 172.21.15.150:0/596306203 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59462700 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821deab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d5a464700 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 -- 172.21.15.150:0/596306203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 msgr2=0x555d821deab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821deab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 -- 172.21.15.150:0/596306203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 msgr2=0x555d821d7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821d7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.290+0000 7f9d59c63700 1 -- 172.21.15.150:0/596306203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d820b3560 con 0x555d821cd6a0 2024-05-10T04:40:50.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.291+0000 7f9d59c63700 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821d2250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005d90 tx=0x7f9d4c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a123dec1f2f571 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.291+0000 7f9d50ff9700 1 -- 172.21.15.150:0/596306203 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c025040 con 0x555d821cd6a0 2024-05-10T04:40:50.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.291+0000 7f9d50ff9700 1 -- 172.21.15.150:0/596306203 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4c015070 con 0x555d821cd6a0 2024-05-10T04:40:50.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.291+0000 7f9d50ff9700 1 -- 172.21.15.150:0/596306203 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c028560 con 0x555d821cd6a0 2024-05-10T04:40:50.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/596306203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 msgr2=0x555d821d2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821d2250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005d90 tx=0x7f9d4c013000 comp rx=0 tx=0).stop 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/596306203 shutdown_connections 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821d7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821d2250 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/596306203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/596306203 >> 172.21.15.150:0/596306203 conn(0x555d8212fa60 msgr2=0x555d821d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/596306203 shutdown_connections 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/596306203 wait complete. 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 -- start start 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.292+0000 7f9d6c20bb80 1 Processor -- start 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821dd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d821deeb0 con 0x555d821cd6a0 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d821dd800 con 0x555d821ceec0 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d821dd9c0 con 0x555d821ce570 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 10 client.0 _ll_register_callbacks cb 0x555d82092b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821de6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37548/0 (socket says 172.21.15.150:37548) 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 -- 172.21.15.150:0/496561306 learned_addr learned my addr 172.21.15.150:0/496561306 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59462700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821dcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d5a464700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 -- 172.21.15.150:0/496561306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 msgr2=0x555d821dcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821dcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 -- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 msgr2=0x555d821dd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821dd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 -- 172.21.15.150:0/496561306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c010da0 con 0x555d821cd6a0 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59462700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:50.300 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d59c63700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821de6e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c020e50 tx=0x7f9d4c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a3a2937181d668 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d397fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d397fa700 1 -- 172.21.15.150:0/496561306 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c028910 con 0x555d821cd6a0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d397fa700 1 -- 172.21.15.150:0/496561306 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9d4c036070 con 0x555d821cd6a0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.293+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d4400bab0 con 0x555d821cd6a0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d397fa700 1 -- 172.21.15.150:0/496561306 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4c03b480 con 0x555d821cd6a0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 10 client.15969 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555d81f72cd0 con 0x555d821cd6a0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d397fa700 1 -- 172.21.15.150:0/496561306 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4400bab0 con 0x555d821cd6a0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d397fa700 1 -- 172.21.15.150:0/496561306 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d4c02e070 con 0x555d821cd6a0 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 10 client.15969 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 20 client.15969 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d8223a820 0x555d8223cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 4 client.15969 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d820e5ec0 con 0x555d8223a820 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.294+0000 7f9d59462700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d8223a820 0x555d8223cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d397fa700 10 client.15969 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d59462700 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d8223a820 0x555d8223cc40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f1fbcb57cca3e8 server_cookie=135dc776962e24a in_seq=0 out_seq=0 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d397fa700 1 -- 172.21.15.150:0/496561306 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555d820e5ec0 con 0x555d8223a820 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d397fa700 10 client.15969 handle_command_reply: tid=0 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 client.15969 shutdown 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 msgr2=0x555d821de6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821de6e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c020e50 tx=0x7f9d4c0057d0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 shutdown_connections 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x555d8223a820 0x555d8223cc40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555d821ceec0 0x555d821dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d397fa700 0 client.15969 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:50.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555d821cd6a0 0x555d821de6e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 --2- 172.21.15.150:0/496561306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555d821ce570 0x555d821dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 >> 172.21.15.150:0/496561306 conn(0x555d8212fa60 msgr2=0x555d821d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 shutdown_connections 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 1 -- 172.21.15.150:0/496561306 wait complete. 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.295+0000 7f9d6c20bb80 20 client.15969 trim_cache size 0 max 16384 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:40:50.303 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:40:50.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d3c0a1240 msgr2=0x7f9d3c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d3c0a1240 0x7f9d3c0a3750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005950 tx=0x7f9d4c00b410 comp rx=0 tx=0).stop 2024-05-10T04:40:50.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54105020 msgr2=0x7f9d541985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54105020 0x7f9d541985d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e9a0 tx=0x7f9d4800ecb0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 shutdown_connections 2024-05-10T04:40:50.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d54107980 0x7f9d5419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d3c0a1240 0x7f9d3c0a3750 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d541026c0 0x7f9d54198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 --2- 172.21.15.150:0/1844846168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d54105020 0x7f9d541985d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 >> 172.21.15.150:0/1844846168 conn(0x7f9d540fc0e0 msgr2=0x7f9d54103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 shutdown_connections 2024-05-10T04:40:50.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.296+0000 7f9d5bec7700 1 -- 172.21.15.150:0/1844846168 wait complete. 2024-05-10T04:40:50.310 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:40:50.310 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:40:50.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 -- 172.21.15.150:0/1116690115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8101c00 msgr2=0x7f58a8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 --2- 172.21.15.150:0/1116690115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8101c00 0x7f58a8102010 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5894009990 tx=0x7f5894009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 -- 172.21.15.150:0/1116690115 shutdown_connections 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 --2- 172.21.15.150:0/1116690115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a81025a0 0x7f58a8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 --2- 172.21.15.150:0/1116690115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a8100ae0 0x7f58a8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 --2- 172.21.15.150:0/1116690115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8101c00 0x7f58a8102010 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 -- 172.21.15.150:0/1116690115 >> 172.21.15.150:0/1116690115 conn(0x7f58a80fc0f0 msgr2=0x7f58a80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 -- 172.21.15.150:0/1116690115 shutdown_connections 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.445+0000 7f58af18d700 1 -- 172.21.15.150:0/1116690115 wait complete. 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 Processor -- start 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 -- start start 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8100ae0 0x7f58a806a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a8101c00 0x7f58a819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a81025a0 0x7f58a819f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a819fa40 con 0x7f58a8100ae0 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a819fbb0 con 0x7f58a8101c00 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58af18d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a819fd70 con 0x7f58a81025a0 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58acf29700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8100ae0 0x7f58a806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58acf29700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8100ae0 0x7f58a806a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37566/0 (socket says 172.21.15.150:37566) 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58acf29700 1 -- 172.21.15.150:0/1246679733 learned_addr learned my addr 172.21.15.150:0/1246679733 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58a7fff700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a8101c00 0x7f58a819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.446+0000 7f58ad72a700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a81025a0 0x7f58a819f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58acf29700 1 -- 172.21.15.150:0/1246679733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a81025a0 msgr2=0x7f58a819f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58acf29700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a81025a0 0x7f58a819f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58acf29700 1 -- 172.21.15.150:0/1246679733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a8101c00 msgr2=0x7f58a819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58acf29700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a8101c00 0x7f58a819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58acf29700 1 -- 172.21.15.150:0/1246679733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940095f0 con 0x7f58a8100ae0 2024-05-10T04:40:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58acf29700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8100ae0 0x7f58a806a550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f589c00d950 tx=0x7f589c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58a5ffb700 1 -- 172.21.15.150:0/1246679733 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c004220 con 0x7f58a8100ae0 2024-05-10T04:40:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a819fff0 con 0x7f58a8100ae0 2024-05-10T04:40:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58a5ffb700 1 -- 172.21.15.150:0/1246679733 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589c004380 con 0x7f58a8100ae0 2024-05-10T04:40:50.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58a5ffb700 1 -- 172.21.15.150:0/1246679733 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c003be0 con 0x7f58a8100ae0 2024-05-10T04:40:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.447+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a81a04c0 con 0x7f58a8100ae0 2024-05-10T04:40:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.449+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8107cf0 con 0x7f58a8100ae0 2024-05-10T04:40:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.449+0000 7f58a5ffb700 1 -- 172.21.15.150:0/1246679733 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f589c003d40 con 0x7f58a8100ae0 2024-05-10T04:40:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.450+0000 7f58a5ffb700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58980a1120 0x7f58980a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.450+0000 7f58a5ffb700 1 -- 172.21.15.150:0/1246679733 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f589c0cc6f0 con 0x7f58a8100ae0 2024-05-10T04:40:50.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.450+0000 7f58a7fff700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58980a1120 0x7f58980a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.451+0000 7f58a7fff700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58980a1120 0x7f58980a3630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58a8069ef0 tx=0x7f589401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.455+0000 7f58a5ffb700 1 -- 172.21.15.150:0/1246679733 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f589c095ae0 con 0x7f58a8100ae0 2024-05-10T04:40:50.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.719+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58a81076d0 con 0x7f58a8100ae0 2024-05-10T04:40:50.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.720+0000 7f58a5ffb700 1 -- 172.21.15.150:0/1246679733 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f589c099100 con 0x7f58a8100ae0 2024-05-10T04:40:50.721 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:40:50.721 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:40:50.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58980a1120 msgr2=0x7f58980a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58980a1120 0x7f58980a3630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58a8069ef0 tx=0x7f589401a040 comp rx=0 tx=0).stop 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8100ae0 msgr2=0x7f58a806a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8100ae0 0x7f58a806a550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f589c00d950 tx=0x7f589c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 shutdown_connections 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f58980a1120 0x7f58980a3630 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58a81025a0 0x7f58a819f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58a8100ae0 0x7f58a806a550 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.725 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 --2- 172.21.15.150:0/1246679733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58a8101c00 0x7f58a819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 >> 172.21.15.150:0/1246679733 conn(0x7f58a80fc0f0 msgr2=0x7f58a8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 shutdown_connections 2024-05-10T04:40:50.726 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.723+0000 7f58af18d700 1 -- 172.21.15.150:0/1246679733 wait complete. 2024-05-10T04:40:50.727 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:40:50.740 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:50 smithi150 ceph-mon[42404]: pgmap v309: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:50.741 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T04:40:50.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:50 smithi161 ceph-mon[46912]: pgmap v309: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:50.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 -- 172.21.15.150:0/15857528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c069090 msgr2=0x7fad7c104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 --2- 172.21.15.150:0/15857528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c069090 0x7fad7c104720 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b0a0 tx=0x7fad6c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 -- 172.21.15.150:0/15857528 shutdown_connections 2024-05-10T04:40:50.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 --2- 172.21.15.150:0/15857528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad7c107880 0x7fad7c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 --2- 172.21.15.150:0/15857528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c069090 0x7fad7c104720 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 --2- 172.21.15.150:0/15857528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad7c104d10 0x7fad7c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 -- 172.21.15.150:0/15857528 >> 172.21.15.150:0/15857528 conn(0x7fad7c0fc0d0 msgr2=0x7fad7c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 -- 172.21.15.150:0/15857528 shutdown_connections 2024-05-10T04:40:50.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.880+0000 7fad81ef6700 1 -- 172.21.15.150:0/15857528 wait complete. 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.881+0000 7fad81ef6700 1 Processor -- start 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad81ef6700 1 -- start start 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad81ef6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad7c069090 0x7fad7c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad81ef6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c104d10 0x7fad7c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad81ef6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad7c107880 0x7fad7c19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad81ef6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c19fb40 con 0x7fad7c104d10 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad81ef6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c19fcb0 con 0x7fad7c069090 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad81ef6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c19fe70 con 0x7fad7c107880 2024-05-10T04:40:50.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c104d10 0x7fad7c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c104d10 0x7fad7c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37578/0 (socket says 172.21.15.150:37578) 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 -- 172.21.15.150:0/2270048596 learned_addr learned my addr 172.21.15.150:0/2270048596 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad7c069090 0x7fad7c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7bfff700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad7c107880 0x7fad7c19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 -- 172.21.15.150:0/2270048596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad7c107880 msgr2=0x7fad7c19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad7c107880 0x7fad7c19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 -- 172.21.15.150:0/2270048596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad7c069090 msgr2=0x7fad7c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad7c069090 0x7fad7c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7affd700 1 -- 172.21.15.150:0/2270048596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c009d00 con 0x7fad7c104d10 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.882+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad7c069090 0x7fad7c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:50.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.883+0000 7fad7affd700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c104d10 0x7fad7c1985a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fad6400d950 tx=0x7fad6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.883+0000 7fad78ff9700 1 -- 172.21.15.150:0/2270048596 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400f890 con 0x7fad7c104d10 2024-05-10T04:40:50.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.883+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c1a0150 con 0x7fad7c104d10 2024-05-10T04:40:50.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.883+0000 7fad78ff9700 1 -- 172.21.15.150:0/2270048596 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad64010460 con 0x7fad7c104d10 2024-05-10T04:40:50.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.883+0000 7fad78ff9700 1 -- 172.21.15.150:0/2270048596 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400e610 con 0x7fad7c104d10 2024-05-10T04:40:50.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.883+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1a0660 con 0x7fad7c104d10 2024-05-10T04:40:50.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.885+0000 7fad78ff9700 1 -- 172.21.15.150:0/2270048596 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fad640108d0 con 0x7fad7c104d10 2024-05-10T04:40:50.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.886+0000 7fad78ff9700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fad680a11c0 0x7fad680a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.886+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fad680a11c0 0x7fad680a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.886+0000 7fad78ff9700 1 -- 172.21.15.150:0/2270048596 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fad640cca40 con 0x7fad7c104d10 2024-05-10T04:40:50.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.886+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fad680a11c0 0x7fad680a36d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000f80 tx=0x7fad6c0092f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 Processor -- start 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 -- start start 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe06a0 0x560d8bfe0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfe1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfe87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bf07150 con 0x560d8bfe1570 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bf04b10 con 0x560d8bfe1ec0 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad9223ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bf00400 con 0x560d8bfe06a0 2024-05-10T04:40:50.891 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad7affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfe1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad7affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfe1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37584/0 (socket says 172.21.15.150:37584) 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad7affd700 1 -- 172.21.15.150:0/818059421 learned_addr learned my addr 172.21.15.150:0/818059421 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe06a0 0x560d8bfe0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.888+0000 7fad7bfff700 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfe87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad7affd700 1 -- 172.21.15.150:0/818059421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe06a0 msgr2=0x560d8bfe0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad7affd700 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe06a0 0x560d8bfe0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad7affd700 1 -- 172.21.15.150:0/818059421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 msgr2=0x560d8bfe87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad7affd700 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfe87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad7affd700 1 -- 172.21.15.150:0/818059421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d8bd813b0 con 0x560d8bfe1570 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad7affd700 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfe1980 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fad640a1a40 tx=0x7fad6409a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9c706a1cc2308 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad71ffb700 1 -- 172.21.15.150:0/818059421 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640a1f70 con 0x560d8bfe1570 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad71ffb700 1 -- 172.21.15.150:0/818059421 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad64097e90 con 0x560d8bfe1570 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.889+0000 7fad71ffb700 1 -- 172.21.15.150:0/818059421 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640a1ae0 con 0x560d8bfe1570 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 -- 172.21.15.150:0/818059421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 msgr2=0x560d8bfe1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.892 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfe1980 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fad640a1a40 tx=0x7fad6409a1f0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 -- 172.21.15.150:0/818059421 shutdown_connections 2024-05-10T04:40:50.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfe87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe06a0 0x560d8bfe0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 --2- 172.21.15.150:0/818059421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfe1980 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 -- 172.21.15.150:0/818059421 >> 172.21.15.150:0/818059421 conn(0x560d8bf42a60 msgr2=0x560d8bfdafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 -- 172.21.15.150:0/818059421 shutdown_connections 2024-05-10T04:40:50.893 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 -- 172.21.15.150:0/818059421 wait complete. 2024-05-10T04:40:50.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.890+0000 7fad9223ab80 1 -- start start 2024-05-10T04:40:50.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 1 Processor -- start 2024-05-10T04:40:50.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe06a0 0x560d8bfea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfeab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8be9c560 con 0x560d8bfe1570 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8be9c200 con 0x560d8bfe06a0 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bd836a0 con 0x560d8bfe1ec0 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad9223ab80 10 client.0 _ll_register_callbacks cb 0x560d8bea5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfeab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfeab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37590/0 (socket says 172.21.15.150:37590) 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 -- 172.21.15.150:0/1816245887 learned_addr learned my addr 172.21.15.150:0/1816245887 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7bfff700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe06a0 0x560d8bfea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 -- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 msgr2=0x560d8bff1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 -- 172.21.15.150:0/1816245887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe06a0 msgr2=0x560d8bfea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe06a0 0x560d8bfea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7affd700 1 -- 172.21.15.150:0/1816245887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1a0660 con 0x560d8bfe1570 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.891+0000 7fad7bfff700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad7affd700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfeab90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fad64095ff0 tx=0x7fad64098070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51167313ed8efbb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad5affd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad5affd700 1 -- 172.21.15.150:0/1816245887 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64099f50 con 0x560d8bfe1570 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad5affd700 1 -- 172.21.15.150:0/1816245887 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fad640b2270 con 0x560d8bfe1570 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d8be59270 con 0x560d8bfe1570 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad5affd700 1 -- 172.21.15.150:0/1816245887 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad640b2430 con 0x560d8bfe1570 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 10 client.15987 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560d8bd2db60 con 0x560d8bfe1570 2024-05-10T04:40:50.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad5affd700 1 -- 172.21.15.150:0/1816245887 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d8be59270 con 0x560d8bfe1570 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad5affd700 1 -- 172.21.15.150:0/1816245887 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad64098b70 con 0x560d8bfe1570 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 10 client.15987 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 20 client.15987 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8bff5e20 0x560d8bff8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 4 client.15987 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.892+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d8bfaf780 con 0x560d8bff5e20 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.893+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8bff5e20 0x560d8bff8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.893+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8bff5e20 0x560d8bff8240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e750a53921dface2 server_cookie=5d7dd7fbebe7c957 in_seq=0 out_seq=0 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.893+0000 7fad5affd700 10 client.15987 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad5affd700 1 -- 172.21.15.150:0/1816245887 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d8bfaf780 con 0x560d8bff5e20 2024-05-10T04:40:50.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad5affd700 10 client.15987 handle_command_reply: tid=0 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 client.15987 shutdown 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 msgr2=0x560d8bfeab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfeab90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fad64095ff0 tx=0x7fad64098070 comp rx=0 tx=0).stop 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 shutdown_connections 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8bff5e20 0x560d8bff8240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad5affd700 0 client.15987 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe06a0 0x560d8bfea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 --2- 172.21.15.150:0/1816245887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe1570 0x560d8bfeab90 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 >> 172.21.15.150:0/1816245887 conn(0x560d8bf42a60 msgr2=0x560d8bfdafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 shutdown_connections 2024-05-10T04:40:50.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 1 -- 172.21.15.150:0/1816245887 wait complete. 2024-05-10T04:40:50.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.894+0000 7fad9223ab80 20 client.15987 trim_cache size 0 max 16384 2024-05-10T04:40:50.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:50 smithi181 ceph-mon[47314]: pgmap v309: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 Processor -- start 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 -- start start 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfe47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1570 0x560d8bff1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bfe22d0 con 0x560d8bfe06a0 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bfea470 con 0x560d8bfe1ec0 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad9223ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bfdc770 con 0x560d8bfe1570 2024-05-10T04:40:50.916 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfe47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfe47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37598/0 (socket says 172.21.15.150:37598) 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 -- 172.21.15.150:0/4029610457 learned_addr learned my addr 172.21.15.150:0/4029610457 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7affd700 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1570 0x560d8bff1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7bfff700 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 -- 172.21.15.150:0/4029610457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1570 msgr2=0x560d8bff1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1570 0x560d8bff1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 -- 172.21.15.150:0/4029610457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 msgr2=0x560d8bfea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 -- 172.21.15.150:0/4029610457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d8bec6560 con 0x560d8bfe06a0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.908+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfe47a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00d010 tx=0x7fad6c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd5591e1847710e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.909+0000 7fad727fc700 1 -- 172.21.15.150:0/4029610457 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02fea0 con 0x560d8bfe06a0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.909+0000 7fad727fc700 1 -- 172.21.15.150:0/4029610457 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad6c02ce30 con 0x560d8bfe06a0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.909+0000 7fad727fc700 1 -- 172.21.15.150:0/4029610457 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00e040 con 0x560d8bfe06a0 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 -- 172.21.15.150:0/4029610457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 msgr2=0x560d8bfe47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfe47a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00d010 tx=0x7fad6c01c000 comp rx=0 tx=0).stop 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 -- 172.21.15.150:0/4029610457 shutdown_connections 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1ec0 0x560d8bfea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfe47a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 --2- 172.21.15.150:0/4029610457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1570 0x560d8bff1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 -- 172.21.15.150:0/4029610457 >> 172.21.15.150:0/4029610457 conn(0x560d8bf42a60 msgr2=0x560d8bfdbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 -- 172.21.15.150:0/4029610457 shutdown_connections 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 -- 172.21.15.150:0/4029610457 wait complete. 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 -- start start 2024-05-10T04:40:50.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 Processor -- start 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.910+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1570 0x560d8bfefae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad9223ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad9223ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bff0f50 con 0x560d8bfe06a0 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad9223ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bff10c0 con 0x560d8bfe1570 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad9223ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8bff1280 con 0x560d8bfe1ec0 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad9223ab80 10 client.0 _ll_register_callbacks cb 0x560d8bea5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfef5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37604/0 (socket says 172.21.15.150:37604) 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 -- 172.21.15.150:0/795216519 learned_addr learned my addr 172.21.15.150:0/795216519 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7affd700 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1570 0x560d8bfefae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7bfff700 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 -- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 msgr2=0x560d8bff0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 -- 172.21.15.150:0/795216519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1570 msgr2=0x560d8bfefae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1570 0x560d8bfefae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 -- 172.21.15.150:0/795216519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c10a9b0 con 0x560d8bfe06a0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad7b7fe700 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfef5a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005040 tx=0x7fad6c007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeb973b767edf9e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad5affd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad5affd700 1 -- 172.21.15.150:0/795216519 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00e070 con 0x560d8bfe06a0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad5affd700 1 -- 172.21.15.150:0/795216519 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fad6c024040 con 0x560d8bfe06a0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.911+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad64097e90 con 0x560d8bfe06a0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad5affd700 1 -- 172.21.15.150:0/795216519 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad6c02fea0 con 0x560d8bfe06a0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 10 client.15993 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560d8bd85cd0 con 0x560d8bfe06a0 2024-05-10T04:40:50.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad5affd700 1 -- 172.21.15.150:0/795216519 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad64097e90 con 0x560d8bfe06a0 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad5affd700 1 -- 172.21.15.150:0/795216519 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad6c038450 con 0x560d8bfe06a0 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 10 client.15993 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 10 client.15993 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 20 client.15993 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8c04cb00 0x560d8c04ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 4 client.15993 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d8bef8ec0 con 0x560d8c04cb00 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.912+0000 7fad7affd700 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8c04cb00 0x560d8c04ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad5affd700 10 client.15993 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad7affd700 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8c04cb00 0x560d8c04ef20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=baabc6975c49baf6 server_cookie=3a014875ec7412e8 in_seq=0 out_seq=0 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad5affd700 1 -- 172.21.15.150:0/795216519 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d8bef8ec0 con 0x560d8c04cb00 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad5affd700 10 client.15993 handle_command_reply: tid=0 2024-05-10T04:40:50.921 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 client.15993 shutdown 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 msgr2=0x560d8bfef5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfef5a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005040 tx=0x7fad6c007ab0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 shutdown_connections 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x560d8c04cb00 0x560d8c04ef20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560d8bfe1ec0 0x560d8bff0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d8bfe06a0 0x560d8bfef5a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 --2- 172.21.15.150:0/795216519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8bfe1570 0x560d8bfefae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.922 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 >> 172.21.15.150:0/795216519 conn(0x560d8bf42a60 msgr2=0x560d8bfe5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad5affd700 0 client.15993 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:40:50.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 shutdown_connections 2024-05-10T04:40:50.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 1 -- 172.21.15.150:0/795216519 wait complete. 2024-05-10T04:40:50.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.913+0000 7fad9223ab80 20 client.15993 trim_cache size 0 max 16384 2024-05-10T04:40:50.923 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:50.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fad680a11c0 msgr2=0x7fad680a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fad680a11c0 0x7fad680a36d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000f80 tx=0x7fad6c0092f0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c104d10 msgr2=0x7fad7c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:50.924 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c104d10 0x7fad7c1985a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fad6400d950 tx=0x7fad6400dc60 comp rx=0 tx=0).stop 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 shutdown_connections 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fad680a11c0 0x7fad680a36d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad7c107880 0x7fad7c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad7c069090 0x7fad7c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 --2- 172.21.15.150:0/2270048596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad7c104d10 0x7fad7c1985a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 >> 172.21.15.150:0/2270048596 conn(0x7fad7c0fc0d0 msgr2=0x7fad7c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 shutdown_connections 2024-05-10T04:40:50.925 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:50.914+0000 7fad81ef6700 1 -- 172.21.15.150:0/2270048596 wait complete. 2024-05-10T04:40:50.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T04:40:51.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 -- 172.21.15.150:0/689321017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 msgr2=0x7f7e24108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/689321017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e24108630 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b0a0 tx=0x7f7e1c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 -- 172.21.15.150:0/689321017 shutdown_connections 2024-05-10T04:40:51.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/689321017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e24108630 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/689321017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e241002c0 0x7f7e241006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/689321017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e241013e0 0x7f7e241017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 -- 172.21.15.150:0/689321017 >> 172.21.15.150:0/689321017 conn(0x7f7e240fb8f0 msgr2=0x7f7e240fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 -- 172.21.15.150:0/689321017 shutdown_connections 2024-05-10T04:40:51.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 -- 172.21.15.150:0/689321017 wait complete. 2024-05-10T04:40:51.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 Processor -- start 2024-05-10T04:40:51.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.063+0000 7f7e2ad43700 1 -- start start 2024-05-10T04:40:51.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e2ad43700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e241002c0 0x7f7e240fd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e2ad43700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e241013e0 0x7f7e2419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e2ad43700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e2419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e2ad43700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419fa10 con 0x7f7e24101d80 2024-05-10T04:40:51.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e2ad43700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419fb80 con 0x7f7e241002c0 2024-05-10T04:40:51.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e2ad43700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2419fd40 con 0x7f7e241013e0 2024-05-10T04:40:51.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e2419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e2419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37630/0 (socket says 172.21.15.150:37630) 2024-05-10T04:40:51.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 -- 172.21.15.150:0/3939417956 learned_addr learned my addr 172.21.15.150:0/3939417956 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e28adf700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e241002c0 0x7f7e240fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e23fff700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e241013e0 0x7f7e2419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 -- 172.21.15.150:0/3939417956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e241013e0 msgr2=0x7f7e2419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e241013e0 0x7f7e2419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 -- 172.21.15.150:0/3939417956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e241002c0 msgr2=0x7f7e240fd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e241002c0 0x7f7e240fd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 -- 172.21.15.150:0/3939417956 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e18009520 con 0x7f7e24101d80 2024-05-10T04:40:51.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e292e0700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e2419f310 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b070 tx=0x7f7e1c0079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e21ffb700 1 -- 172.21.15.150:0/3939417956 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00e040 con 0x7f7e24101d80 2024-05-10T04:40:51.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e21ffb700 1 -- 172.21.15.150:0/3939417956 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e1c003c00 con 0x7f7e24101d80 2024-05-10T04:40:51.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e21ffb700 1 -- 172.21.15.150:0/3939417956 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c01b6c0 con 0x7f7e24101d80 2024-05-10T04:40:51.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.064+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1c009d00 con 0x7f7e24101d80 2024-05-10T04:40:51.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.065+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e241a01a0 con 0x7f7e24101d80 2024-05-10T04:40:51.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.067+0000 7f7e21ffb700 1 -- 172.21.15.150:0/3939417956 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7e1c01d470 con 0x7f7e24101d80 2024-05-10T04:40:51.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.068+0000 7f7e21ffb700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7e140a1240 0x7f7e140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.068+0000 7f7e21ffb700 1 -- 172.21.15.150:0/3939417956 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7e1c0cdc70 con 0x7f7e24101d80 2024-05-10T04:40:51.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.068+0000 7f7e28adf700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7e140a1240 0x7f7e140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 Processor -- start 2024-05-10T04:40:51.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 -- start start 2024-05-10T04:40:51.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e28adf700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7e140a1240 0x7f7e140a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7e241012e0 tx=0x7f7e18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b69af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6a570 0x55aa07b6a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07a90150 con 0x55aa07b6aec0 2024-05-10T04:40:51.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07a8db10 con 0x55aa07b696a0 2024-05-10T04:40:51.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07a89400 con 0x55aa07b6a570 2024-05-10T04:40:51.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37644/0 (socket says 172.21.15.150:37644) 2024-05-10T04:40:51.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e292e0700 1 -- 172.21.15.150:0/3722694194 learned_addr learned my addr 172.21.15.150:0/3722694194 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e28adf700 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b69af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e23fff700 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6a570 0x55aa07b6a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e292e0700 1 -- 172.21.15.150:0/3722694194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6a570 msgr2=0x55aa07b6a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e292e0700 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6a570 0x55aa07b6a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.069+0000 7f7e292e0700 1 -- 172.21.15.150:0/3722694194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 msgr2=0x55aa07b69af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.070+0000 7f7e292e0700 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b69af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.070+0000 7f7e292e0700 1 -- 172.21.15.150:0/3722694194 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0790a3b0 con 0x55aa07b6aec0 2024-05-10T04:40:51.087 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.070+0000 7f7e292e0700 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b717c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0977c0 tx=0x7f7e1c01dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06c0542a5ee450f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.070+0000 7f7e0effd700 1 -- 172.21.15.150:0/3722694194 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c09b450 con 0x55aa07b6aec0 2024-05-10T04:40:51.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.070+0000 7f7e0effd700 1 -- 172.21.15.150:0/3722694194 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e1c09ad50 con 0x55aa07b6aec0 2024-05-10T04:40:51.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.070+0000 7f7e0effd700 1 -- 172.21.15.150:0/3722694194 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c01d720 con 0x55aa07b6aec0 2024-05-10T04:40:51.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 -- 172.21.15.150:0/3722694194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 msgr2=0x55aa07b717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b717c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0977c0 tx=0x7f7e1c01dc20 comp rx=0 tx=0).stop 2024-05-10T04:40:51.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 -- 172.21.15.150:0/3722694194 shutdown_connections 2024-05-10T04:40:51.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b717c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b69af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/3722694194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6a570 0x55aa07b6a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 -- 172.21.15.150:0/3722694194 >> 172.21.15.150:0/3722694194 conn(0x55aa07acba60 msgr2=0x55aa07b63fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 -- 172.21.15.150:0/3722694194 shutdown_connections 2024-05-10T04:40:51.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 -- 172.21.15.150:0/3722694194 wait complete. 2024-05-10T04:40:51.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 -- start start 2024-05-10T04:40:51.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.071+0000 7f7e3b087b80 1 Processor -- start 2024-05-10T04:40:51.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6a570 0x55aa07b73920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b7a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07a25560 con 0x55aa07b6a570 2024-05-10T04:40:51.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07a25200 con 0x55aa07b696a0 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0790c6a0 con 0x55aa07b6aec0 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e3b087b80 10 client.0 _ll_register_callbacks cb 0x55aa07a2eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6a570 0x55aa07b73920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b7a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b7a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:38764/0 (socket says 172.21.15.150:38764) 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 -- 172.21.15.150:0/2729663292 learned_addr learned my addr 172.21.15.150:0/2729663292 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6a570 0x55aa07b73920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37648/0 (socket says 172.21.15.150:37648) 2024-05-10T04:40:51.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e28adf700 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 -- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 msgr2=0x55aa07b733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 -- 172.21.15.150:0/2729663292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6a570 msgr2=0x55aa07b73920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6a570 0x55aa07b73920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e292e0700 1 -- 172.21.15.150:0/2729663292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e241a01a0 con 0x55aa07b6aec0 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.072+0000 7f7e23fff700 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6a570 0x55aa07b73920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e292e0700 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b7a880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0990a0 tx=0x7f7e1c009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b7160eaf360f1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e03fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e03fff700 1 -- 172.21.15.150:0/2729663292 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c09b1b0 con 0x55aa07b6aec0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa079e2270 con 0x55aa07b6aec0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e03fff700 1 -- 172.21.15.150:0/2729663292 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7e1c09eb80 con 0x55aa07b6aec0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e03fff700 1 -- 172.21.15.150:0/2729663292 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e1c0a3140 con 0x55aa07b6aec0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e3b087b80 10 client.25567 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55aa078b6b60 con 0x55aa07b6aec0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e03fff700 1 -- 172.21.15.150:0/2729663292 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa079e2270 con 0x55aa07b6aec0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e03fff700 1 -- 172.21.15.150:0/2729663292 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e1c099cb0 con 0x55aa07b6aec0 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e3b087b80 10 client.25567 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e3b087b80 10 client.25567 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:40:51.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.073+0000 7f7e3b087b80 20 client.25567 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.074+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07b7eb10 0x55aa07b80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.074+0000 7f7e3b087b80 4 client.25567 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:51.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.074+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa07b38780 con 0x55aa07b7eb10 2024-05-10T04:40:51.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.074+0000 7f7e28adf700 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07b7eb10 0x55aa07b80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.074+0000 7f7e03fff700 10 client.25567 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:51.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.074+0000 7f7e28adf700 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07b7eb10 0x55aa07b80f30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8231c6fb9b38884a server_cookie=fa68a928f0d4b3d8 in_seq=0 out_seq=0 2024-05-10T04:40:51.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e03fff700 1 -- 172.21.15.150:0/2729663292 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa07b38780 con 0x55aa07b7eb10 2024-05-10T04:40:51.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e03fff700 10 client.25567 handle_command_reply: tid=0 2024-05-10T04:40:51.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 client.25567 shutdown 2024-05-10T04:40:51.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 msgr2=0x55aa07b7a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b7a880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0990a0 tx=0x7f7e1c009850 comp rx=0 tx=0).stop 2024-05-10T04:40:51.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 shutdown_connections 2024-05-10T04:40:51.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07b7eb10 0x55aa07b80f30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b7a880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b696a0 0x55aa07b733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/2729663292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6a570 0x55aa07b73920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e03fff700 0 client.25567 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 >> 172.21.15.150:0/2729663292 conn(0x55aa07acba60 msgr2=0x55aa07b63fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 shutdown_connections 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 1 -- 172.21.15.150:0/2729663292 wait complete. 2024-05-10T04:40:51.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.075+0000 7f7e3b087b80 20 client.25567 trim_cache size 0 max 16384 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 Processor -- start 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 -- start start 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b696a0 0x55aa07b80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b7a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa079e2270 con 0x55aa07b696a0 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07b5e810 con 0x55aa07b6a570 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07b60d30 con 0x55aa07b6aec0 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b696a0 0x55aa07b80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b696a0 0x55aa07b80f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37656/0 (socket says 172.21.15.150:37656) 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 -- 172.21.15.150:0/609472585 learned_addr learned my addr 172.21.15.150:0/609472585 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e292e0700 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e23fff700 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b7a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 -- 172.21.15.150:0/609472585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 msgr2=0x55aa07b733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 -- 172.21.15.150:0/609472585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 msgr2=0x55aa07b7a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b7a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e28adf700 1 -- 172.21.15.150:0/609472585 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa07a4f560 con 0x55aa07b696a0 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.089+0000 7f7e292e0700 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.090+0000 7f7e28adf700 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b696a0 0x55aa07b80f30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55aa07b817d0 tx=0x7f7e18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4799d1fa6c4fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.090+0000 7f7e0f7fe700 1 -- 172.21.15.150:0/609472585 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18024040 con 0x55aa07b696a0 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.090+0000 7f7e0f7fe700 1 -- 172.21.15.150:0/609472585 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e18029920 con 0x55aa07b696a0 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.090+0000 7f7e0f7fe700 1 -- 172.21.15.150:0/609472585 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1801d3b0 con 0x55aa07b696a0 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 -- 172.21.15.150:0/609472585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b696a0 msgr2=0x55aa07b80f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b696a0 0x55aa07b80f30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55aa07b817d0 tx=0x7f7e18014000 comp rx=0 tx=0).stop 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 -- 172.21.15.150:0/609472585 shutdown_connections 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b6aec0 0x55aa07b733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b696a0 0x55aa07b80f30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/609472585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b7a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 -- 172.21.15.150:0/609472585 >> 172.21.15.150:0/609472585 conn(0x55aa07acba60 msgr2=0x55aa07b6d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 -- 172.21.15.150:0/609472585 shutdown_connections 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 -- 172.21.15.150:0/609472585 wait complete. 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 -- start start 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.091+0000 7f7e3b087b80 1 Processor -- start 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b696a0 0x55aa07b77d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b78250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e3b087b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07b796c0 con 0x55aa07b6aec0 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07b79830 con 0x55aa07b6a570 2024-05-10T04:40:51.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e3b087b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa07b799f0 con 0x55aa07b696a0 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e3b087b80 10 client.0 _ll_register_callbacks cb 0x55aa07a2eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b79180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37660/0 (socket says 172.21.15.150:37660) 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 -- 172.21.15.150:0/131492824 learned_addr learned my addr 172.21.15.150:0/131492824 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e28adf700 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b696a0 0x55aa07b77d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e23fff700 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b78250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 -- 172.21.15.150:0/131492824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b696a0 msgr2=0x55aa07b77d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b696a0 0x55aa07b77d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 -- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 msgr2=0x55aa07b78250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b78250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.092+0000 7f7e292e0700 1 -- 172.21.15.150:0/131492824 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e18017070 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e292e0700 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b79180 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c097250 tx=0x7f7e1c0b91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a037e626eaf9e5e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e03fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e03fff700 1 -- 172.21.15.150:0/131492824 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c009b80 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e03fff700 1 -- 172.21.15.150:0/131492824 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7e1c01d930 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e1c09ad50 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e03fff700 1 -- 172.21.15.150:0/131492824 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e1c09a890 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 10 client.16011 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55aa0790ecd0 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e03fff700 1 -- 172.21.15.150:0/131492824 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e1c07d020 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e03fff700 1 -- 172.21.15.150:0/131492824 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e1c09afb0 con 0x55aa07b6aec0 2024-05-10T04:40:51.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 10 client.16011 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 20 client.16011 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07bd5050 0x55aa07bd7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 4 client.16011 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.093+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa07a81ec0 con 0x55aa07bd5050 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e28adf700 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07bd5050 0x55aa07bd7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e03fff700 10 client.16011 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e28adf700 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07bd5050 0x55aa07bd7470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=394219fb6067f33b server_cookie=f1d5f90e665033e4 in_seq=0 out_seq=0 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e03fff700 1 -- 172.21.15.150:0/131492824 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa07a81ec0 con 0x55aa07bd5050 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e03fff700 10 client.16011 handle_command_reply: tid=0 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 client.16011 shutdown 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 msgr2=0x55aa07b79180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b79180 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c097250 tx=0x7f7e1c0b91a0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 shutdown_connections 2024-05-10T04:40:51.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55aa07bd5050 0x55aa07bd7470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa07b6aec0 0x55aa07b79180 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa07b696a0 0x55aa07b77d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 --2- 172.21.15.150:0/131492824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa07b6a570 0x55aa07b78250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e03fff700 0 client.16011 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:40:51.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.094+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 >> 172.21.15.150:0/131492824 conn(0x55aa07acba60 msgr2=0x55aa07b73270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 shutdown_connections 2024-05-10T04:40:51.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e3b087b80 1 -- 172.21.15.150:0/131492824 wait complete. 2024-05-10T04:40:51.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e3b087b80 20 client.16011 trim_cache size 0 max 16384 2024-05-10T04:40:51.109 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7e140a1240 msgr2=0x7f7e140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7e140a1240 0x7f7e140a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7e241012e0 tx=0x7f7e18009260 comp rx=0 tx=0).stop 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 msgr2=0x7f7e2419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e2419f310 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b070 tx=0x7f7e1c0079b0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 shutdown_connections 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e24101d80 0x7f7e2419f310 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e241002c0 0x7f7e240fd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7e140a1240 0x7f7e140a3750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 --2- 172.21.15.150:0/3939417956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e241013e0 0x7f7e2419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.095+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 >> 172.21.15.150:0/3939417956 conn(0x7f7e240fb8f0 msgr2=0x7f7e24105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.096+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 shutdown_connections 2024-05-10T04:40:51.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.096+0000 7f7e2ad43700 1 -- 172.21.15.150:0/3939417956 wait complete. 2024-05-10T04:40:51.112 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:40:51.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 -- 172.21.15.150:0/624846658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc100ae0 msgr2=0x7f08cc100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 --2- 172.21.15.150:0/624846658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc100ae0 0x7f08cc100ef0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009990 tx=0x7f08c8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 -- 172.21.15.150:0/624846658 shutdown_connections 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 --2- 172.21.15.150:0/624846658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08cc1025a0 0x7f08cc108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 --2- 172.21.15.150:0/624846658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc100ae0 0x7f08cc100ef0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 --2- 172.21.15.150:0/624846658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08cc101c00 0x7f08cc102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 -- 172.21.15.150:0/624846658 >> 172.21.15.150:0/624846658 conn(0x7f08cc0fc0f0 msgr2=0x7f08cc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 -- 172.21.15.150:0/624846658 shutdown_connections 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.247+0000 7f08d4711700 1 -- 172.21.15.150:0/624846658 wait complete. 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 Processor -- start 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 -- start start 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08cc100ae0 0x7f08cc06a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc101c00 0x7f08cc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08cc1025a0 0x7f08cc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc19f9c0 con 0x7f08cc101c00 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc19fb30 con 0x7f08cc1025a0 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d4711700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc19fcf0 con 0x7f08cc100ae0 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc101c00 0x7f08cc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc101c00 0x7f08cc19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37684/0 (socket says 172.21.15.150:37684) 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 -- 172.21.15.150:0/3880390862 learned_addr learned my addr 172.21.15.150:0/3880390862 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d2cae700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08cc1025a0 0x7f08cc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d24ad700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08cc100ae0 0x7f08cc06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 -- 172.21.15.150:0/3880390862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08cc100ae0 msgr2=0x7f08cc06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08cc100ae0 0x7f08cc06a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 -- 172.21.15.150:0/3880390862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08cc1025a0 msgr2=0x7f08cc19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08cc1025a0 0x7f08cc19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.248+0000 7f08d1cac700 1 -- 172.21.15.150:0/3880390862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80095f0 con 0x7f08cc101c00 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.249+0000 7f08d1cac700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc101c00 0x7f08cc19ccb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d950 tx=0x7f08bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.249+0000 7f08c37fe700 1 -- 172.21.15.150:0/3880390862 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00f890 con 0x7f08cc101c00 2024-05-10T04:40:51.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.249+0000 7f08c37fe700 1 -- 172.21.15.150:0/3880390862 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08bc010460 con 0x7f08cc101c00 2024-05-10T04:40:51.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.249+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc19ffd0 con 0x7f08cc101c00 2024-05-10T04:40:51.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.249+0000 7f08c37fe700 1 -- 172.21.15.150:0/3880390862 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00e610 con 0x7f08cc101c00 2024-05-10T04:40:51.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.249+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc1a03f0 con 0x7f08cc101c00 2024-05-10T04:40:51.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.251+0000 7f08c37fe700 1 -- 172.21.15.150:0/3880390862 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f08bc0108d0 con 0x7f08cc101c00 2024-05-10T04:40:51.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.251+0000 7f08c37fe700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f08b80a11c0 0x7f08b80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.252+0000 7f08c37fe700 1 -- 172.21.15.150:0/3880390862 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f08bc0cca70 con 0x7f08cc101c00 2024-05-10T04:40:51.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.252+0000 7f08d24ad700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f08b80a11c0 0x7f08b80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.252+0000 7f08d24ad700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f08b80a11c0 0x7f08b80a36d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006010 tx=0x7f08c801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 Processor -- start 2024-05-10T04:40:51.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 -- start start 2024-05-10T04:40:51.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4117af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a4118980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a411f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a403e150 con 0x5654a4118ec0 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a403bb10 con 0x5654a4118570 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a4037400 con 0x5654a41176a0 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08d1cac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a4118980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08d2cae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a411f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08d1cac700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a4118980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37180/0 (socket says 172.21.15.150:37180) 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08d1cac700 1 -- 172.21.15.150:0/2608973572 learned_addr learned my addr 172.21.15.150:0/2608973572 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.253+0000 7f08d24ad700 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4117af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08d1cac700 1 -- 172.21.15.150:0/2608973572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 msgr2=0x5654a4117af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08d1cac700 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4117af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08d1cac700 1 -- 172.21.15.150:0/2608973572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 msgr2=0x5654a411f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08d1cac700 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a411f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08d1cac700 1 -- 172.21.15.150:0/2608973572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654a3eb83b0 con 0x5654a4118570 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08d2cae700 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a411f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08d1cac700 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a4118980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0a1a10 tx=0x7f08bc005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c28e46e70edc26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08c0ff9700 1 -- 172.21.15.150:0/2608973572 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc09d8f0 con 0x5654a4118570 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08c0ff9700 1 -- 172.21.15.150:0/2608973572 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08bc0a1f40 con 0x5654a4118570 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.254+0000 7f08c0ff9700 1 -- 172.21.15.150:0/2608973572 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0a1ab0 con 0x5654a4118570 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2608973572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 msgr2=0x5654a4118980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a4118980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0a1a10 tx=0x7f08bc005910 comp rx=0 tx=0).stop 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2608973572 shutdown_connections 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a411f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4117af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2608973572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a4118980 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2608973572 >> 172.21.15.150:0/2608973572 conn(0x5654a4079a60 msgr2=0x5654a4111fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2608973572 shutdown_connections 2024-05-10T04:40:51.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2608973572 wait complete. 2024-05-10T04:40:51.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.255+0000 7f08e4a55b80 1 -- start start 2024-05-10T04:40:51.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 1 Processor -- start 2024-05-10T04:40:51.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a41176a0 0x5654a4121660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a4118570 0x5654a4121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a4128b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a3fd3560 con 0x5654a41176a0 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a3fd3200 con 0x5654a4118ec0 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a3eba6a0 con 0x5654a4118570 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a41176a0 0x5654a4121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08e4a55b80 10 client.0 _ll_register_callbacks cb 0x5654a3fdcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a41176a0 0x5654a4121660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37708/0 (socket says 172.21.15.150:37708) 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 -- 172.21.15.150:0/2486202335 learned_addr learned my addr 172.21.15.150:0/2486202335 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d1cac700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a4118570 0x5654a4121ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d2cae700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a4128b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 -- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a4118570 msgr2=0x5654a4121ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a4118570 0x5654a4121ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 -- 172.21.15.150:0/2486202335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 msgr2=0x5654a4128b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a4128b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d24ad700 1 -- 172.21.15.150:0/2486202335 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c4011230 con 0x5654a41176a0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.256+0000 7f08d2cae700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a4128b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08d24ad700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a41176a0 0x5654a4121660 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f08cc1007d0 tx=0x7f08c801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276489cad5931fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08b5ffb700 1 -- 172.21.15.150:0/2486202335 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c802d040 con 0x5654a41176a0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08b5ffb700 1 -- 172.21.15.150:0/2486202335 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f08c8024620 con 0x5654a41176a0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654a3f90270 con 0x5654a41176a0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08b5ffb700 1 -- 172.21.15.150:0/2486202335 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c80247e0 con 0x5654a41176a0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 10 client.16023 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f08cc1a03f0 con 0x5654a41176a0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08b5ffb700 1 -- 172.21.15.150:0/2486202335 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5654a3f90270 con 0x5654a41176a0 2024-05-10T04:40:51.264 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08b5ffb700 1 -- 172.21.15.150:0/2486202335 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08c8032af0 con 0x5654a41176a0 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 20 client.16023 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a412f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 4 client.16023 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.257+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654a40e6780 con 0x5654a412d380 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.258+0000 7f08d1cac700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a412f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.258+0000 7f08b5ffb700 10 client.16023 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.258+0000 7f08d1cac700 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a412f7d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8a9a06a6e224d04 server_cookie=7f0eea58dc3d134d in_seq=0 out_seq=0 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08b5ffb700 1 -- 172.21.15.150:0/2486202335 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5654a40e6780 con 0x5654a412d380 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08b5ffb700 10 client.16023 handle_command_reply: tid=0 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 client.16023 shutdown 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a41176a0 msgr2=0x5654a4121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a41176a0 0x5654a4121660 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f08cc1007d0 tx=0x7f08c801f000 comp rx=0 tx=0).stop 2024-05-10T04:40:51.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 shutdown_connections 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a412f7d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a4128b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08b5ffb700 0 client.16023 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a41176a0 0x5654a4121660 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/2486202335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a4118570 0x5654a4121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 >> 172.21.15.150:0/2486202335 conn(0x5654a4079a60 msgr2=0x5654a4111fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 shutdown_connections 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 1 -- 172.21.15.150:0/2486202335 wait complete. 2024-05-10T04:40:51.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.259+0000 7f08e4a55b80 20 client.16023 trim_cache size 0 max 16384 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 Processor -- start 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 -- start start 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a411c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118570 0x5654a4128ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a411c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a411cd50 con 0x5654a4118570 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a411be20 con 0x5654a4118ec0 2024-05-10T04:40:51.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.272+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a4118220 con 0x5654a41176a0 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118570 0x5654a4128ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118570 0x5654a4128ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37712/0 (socket says 172.21.15.150:37712) 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 -- 172.21.15.150:0/146158581 learned_addr learned my addr 172.21.15.150:0/146158581 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d2cae700 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a411c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d24ad700 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a411c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 -- 172.21.15.150:0/146158581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 msgr2=0x5654a411c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a411c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 -- 172.21.15.150:0/146158581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 msgr2=0x5654a411c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a411c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 -- 172.21.15.150:0/146158581 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654a3e64b60 con 0x5654a4118570 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d24ad700 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a411c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d2cae700 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a411c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08d1cac700 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118570 0x5654a4128ac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f08bc095fc0 tx=0x7f08bc010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b31ff78f230281e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08c17fa700 1 -- 172.21.15.150:0/146158581 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc097e60 con 0x5654a4118570 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08c17fa700 1 -- 172.21.15.150:0/146158581 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08bc099d20 con 0x5654a4118570 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.273+0000 7f08c17fa700 1 -- 172.21.15.150:0/146158581 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0a1ab0 con 0x5654a4118570 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 -- 172.21.15.150:0/146158581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118570 msgr2=0x5654a4128ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118570 0x5654a4128ac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f08bc095fc0 tx=0x7f08bc010cd0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 -- 172.21.15.150:0/146158581 shutdown_connections 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118ec0 0x5654a411c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a411c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/146158581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118570 0x5654a4128ac0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 -- 172.21.15.150:0/146158581 >> 172.21.15.150:0/146158581 conn(0x5654a4079a60 msgr2=0x5654a412f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 -- 172.21.15.150:0/146158581 shutdown_connections 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 -- 172.21.15.150:0/146158581 wait complete. 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 -- start start 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.274+0000 7f08e4a55b80 1 Processor -- start 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4125c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a41261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a4127100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a4127640 con 0x5654a4118ec0 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a41277b0 con 0x5654a4118570 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654a4127970 con 0x5654a41176a0 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 10 client.0 _ll_register_callbacks cb 0x5654a3fdcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a4127100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a4127100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37722/0 (socket says 172.21.15.150:37722) 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 -- 172.21.15.150:0/1954311592 learned_addr learned my addr 172.21.15.150:0/1954311592 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d1cac700 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a41261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d24ad700 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4125c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 -- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 msgr2=0x5654a4125c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4125c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 -- 172.21.15.150:0/1954311592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 msgr2=0x5654a41261d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a41261d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 -- 172.21.15.150:0/1954311592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654a3eb83b0 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08d2cae700 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a4127100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f08c40045f0 tx=0x7f08c4004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef6ab7e6d121f23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08b5ffb700 1 -- 172.21.15.150:0/1954311592 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400e040 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08b5ffb700 1 -- 172.21.15.150:0/1954311592 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f08c4008b00 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.275+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08bc0a1f40 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08b5ffb700 1 -- 172.21.15.150:0/1954311592 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c4008cc0 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 10 client.16029 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5654a3ffd560 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08b5ffb700 1 -- 172.21.15.150:0/1954311592 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08bc0a1f40 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08b5ffb700 1 -- 172.21.15.150:0/1954311592 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08c4016400 con 0x5654a4118ec0 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 10 client.16029 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:40:51.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 20 client.16029 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a4185bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 4 client.16029 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5654a402fec0 con 0x5654a412d380 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.276+0000 7f08d24ad700 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a4185bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08b5ffb700 10 client.16029 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08d24ad700 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a4185bf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9480c26cd0df175 server_cookie=f084243424bcdbe0 in_seq=0 out_seq=0 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08b5ffb700 1 -- 172.21.15.150:0/1954311592 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5654a402fec0 con 0x5654a412d380 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08b5ffb700 10 client.16029 handle_command_reply: tid=0 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 client.16029 shutdown 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 msgr2=0x5654a4127100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a4127100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f08c40045f0 tx=0x7f08c4004900 comp rx=0 tx=0).stop 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 shutdown_connections 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5654a412d380 0x5654a4185bf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654a4118ec0 0x5654a4127100 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5654a41176a0 0x5654a4125c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08b5ffb700 0 client.16029 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:40:51.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 --2- 172.21.15.150:0/1954311592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a4118570 0x5654a41261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 >> 172.21.15.150:0/1954311592 conn(0x5654a4079a60 msgr2=0x5654a412f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 shutdown_connections 2024-05-10T04:40:51.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 1 -- 172.21.15.150:0/1954311592 wait complete. 2024-05-10T04:40:51.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.277+0000 7f08e4a55b80 20 client.16029 trim_cache size 0 max 16384 2024-05-10T04:40:51.285 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f08b80a11c0 msgr2=0x7f08b80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f08b80a11c0 0x7f08b80a36d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006010 tx=0x7f08c801a040 comp rx=0 tx=0).stop 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc101c00 msgr2=0x7f08cc19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc101c00 0x7f08cc19ccb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d950 tx=0x7f08bc00dc60 comp rx=0 tx=0).stop 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 shutdown_connections 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08cc1025a0 0x7f08cc19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f08b80a11c0 0x7f08b80a36d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08cc100ae0 0x7f08cc06a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 --2- 172.21.15.150:0/3880390862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08cc101c00 0x7f08cc19ccb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 >> 172.21.15.150:0/3880390862 conn(0x7f08cc0fc0f0 msgr2=0x7f08cc104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 shutdown_connections 2024-05-10T04:40:51.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.278+0000 7f08d4711700 1 -- 172.21.15.150:0/3880390862 wait complete. 2024-05-10T04:40:51.293 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 -- 172.21.15.150:0/2065004679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 msgr2=0x7f4174109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 --2- 172.21.15.150:0/2065004679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f4174109da0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f416c00b0a0 tx=0x7f416c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 -- 172.21.15.150:0/2065004679 shutdown_connections 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 --2- 172.21.15.150:0/2065004679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f4174109da0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 --2- 172.21.15.150:0/2065004679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417406a4d0 0x7f4174104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 --2- 172.21.15.150:0/2065004679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174104d20 0x7f4174107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 -- 172.21.15.150:0/2065004679 >> 172.21.15.150:0/2065004679 conn(0x7f41740fc0c0 msgr2=0x7f41740fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 -- 172.21.15.150:0/2065004679 shutdown_connections 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.429+0000 7f4179b11700 1 -- 172.21.15.150:0/2065004679 wait complete. 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 Processor -- start 2024-05-10T04:40:51.433 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 -- start start 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417406a4d0 0x7f4174198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174104d20 0x7f41741985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f417419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417419fb40 con 0x7f4174107890 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417419fcb0 con 0x7f4174104d20 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4179b11700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417419fe70 con 0x7f417406a4d0 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f417419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f417419f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37734/0 (socket says 172.21.15.150:37734) 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 -- 172.21.15.150:0/4267430071 learned_addr learned my addr 172.21.15.150:0/4267430071 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4172ffd700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174104d20 0x7f41741985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f41737fe700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417406a4d0 0x7f4174198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 -- 172.21.15.150:0/4267430071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417406a4d0 msgr2=0x7f4174198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.434 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417406a4d0 0x7f4174198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 -- 172.21.15.150:0/4267430071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174104d20 msgr2=0x7f41741985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174104d20 0x7f41741985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.430+0000 7f4173fff700 1 -- 172.21.15.150:0/4267430071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009d00 con 0x7f4174107890 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.431+0000 7f4173fff700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f417419f530 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f416c004c80 tx=0x7f416c004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.431+0000 7f4170ff9700 1 -- 172.21.15.150:0/4267430071 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00e040 con 0x7f4174107890 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.431+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41741a00f0 con 0x7f4174107890 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.431+0000 7f4170ff9700 1 -- 172.21.15.150:0/4267430071 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f416c00cec0 con 0x7f4174107890 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.431+0000 7f4170ff9700 1 -- 172.21.15.150:0/4267430071 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c01b930 con 0x7f4174107890 2024-05-10T04:40:51.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.431+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41741a0570 con 0x7f4174107890 2024-05-10T04:40:51.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.434+0000 7f4170ff9700 1 -- 172.21.15.150:0/4267430071 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f416c01bad0 con 0x7f4174107890 2024-05-10T04:40:51.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.434+0000 7f4170ff9700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f415c0a11c0 0x7f415c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4170ff9700 1 -- 172.21.15.150:0/4267430071 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f416c0ce8f0 con 0x7f4174107890 2024-05-10T04:40:51.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f41737fe700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f415c0a11c0 0x7f415c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 Processor -- start 2024-05-10T04:40:51.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 -- start start 2024-05-10T04:40:51.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f41737fe700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f415c0a11c0 0x7f415c0a36d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f417406a8e0 tx=0x7f416400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db5586a0 0x5612db558af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db559980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559ec0 0x5612db5607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db47f150 con 0x5612db559570 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db47cb10 con 0x5612db559ec0 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.435+0000 7f4189e55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db478400 con 0x5612db5586a0 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db559980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db559980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37744/0 (socket says 172.21.15.150:37744) 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 -- 172.21.15.150:0/3816083655 learned_addr learned my addr 172.21.15.150:0/3816083655 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f41737fe700 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db5586a0 0x5612db558af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4173fff700 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559ec0 0x5612db5607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 -- 172.21.15.150:0/3816083655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db5586a0 msgr2=0x5612db558af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db5586a0 0x5612db558af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 -- 172.21.15.150:0/3816083655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559ec0 msgr2=0x5612db5607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559ec0 0x5612db5607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 -- 172.21.15.150:0/3816083655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612db2f93b0 con 0x5612db559570 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4172ffd700 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db559980 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f416800bd30 tx=0x7f41680106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dedb737cb83408 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4159ffb700 1 -- 172.21.15.150:0/3816083655 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168010c10 con 0x5612db559570 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4159ffb700 1 -- 172.21.15.150:0/3816083655 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4168015070 con 0x5612db559570 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.436+0000 7f4159ffb700 1 -- 172.21.15.150:0/3816083655 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416801a5e0 con 0x5612db559570 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 -- 172.21.15.150:0/3816083655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 msgr2=0x5612db559980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db559980 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f416800bd30 tx=0x7f41680106b0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 -- 172.21.15.150:0/3816083655 shutdown_connections 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559ec0 0x5612db5607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db5586a0 0x5612db558af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 --2- 172.21.15.150:0/3816083655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db559980 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 -- 172.21.15.150:0/3816083655 >> 172.21.15.150:0/3816083655 conn(0x5612db4baa60 msgr2=0x5612db552fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 -- 172.21.15.150:0/3816083655 shutdown_connections 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 -- 172.21.15.150:0/3816083655 wait complete. 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.437+0000 7f4189e55b80 1 -- start start 2024-05-10T04:40:51.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 1 Processor -- start 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db5626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db562c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db569b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db414560 con 0x5612db5586a0 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db414200 con 0x5612db559570 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db2fb6a0 con 0x5612db559ec0 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4189e55b80 10 client.0 _ll_register_callbacks cb 0x5612db41db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f41737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db5626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4172ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db562c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f41737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db5626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37746/0 (socket says 172.21.15.150:37746) 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f41737fe700 1 -- 172.21.15.150:0/1368062477 learned_addr learned my addr 172.21.15.150:0/1368062477 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.438+0000 7f4173fff700 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db569b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4172ffd700 1 -- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 msgr2=0x5612db569b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4172ffd700 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db569b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4172ffd700 1 -- 172.21.15.150:0/1368062477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 msgr2=0x5612db5626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4172ffd700 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db5626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4172ffd700 1 -- 172.21.15.150:0/1368062477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416800b960 con 0x5612db559570 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f41737fe700 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db5626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4172ffd700 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db562c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f416800bd00 tx=0x7f4168010ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48026f300f9b0456 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4152ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4152ffd700 1 -- 172.21.15.150:0/1368062477 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680118d0 con 0x5612db559570 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4152ffd700 1 -- 172.21.15.150:0/1368062477 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4168017070 con 0x5612db559570 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612db3d1270 con 0x5612db559570 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4152ffd700 1 -- 172.21.15.150:0/1368062477 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4168011a30 con 0x5612db559570 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4189e55b80 10 client.25682 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.439+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612db2a5b60 con 0x5612db559570 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4152ffd700 1 -- 172.21.15.150:0/1368062477 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612db3d1270 con 0x5612db559570 2024-05-10T04:40:51.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4152ffd700 1 -- 172.21.15.150:0/1368062477 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4168004750 con 0x5612db559570 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4189e55b80 10 client.25682 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4189e55b80 10 client.25682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4189e55b80 20 client.25682 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db56e120 0x5612db570570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4189e55b80 4 client.25682 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612db527780 con 0x5612db56e120 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.440+0000 7f41737fe700 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db56e120 0x5612db570570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.441+0000 7f4152ffd700 10 client.25682 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.441+0000 7f41737fe700 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db56e120 0x5612db570570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=523819bc796440ec server_cookie=496273d297bb8438 in_seq=0 out_seq=0 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.441+0000 7f4152ffd700 1 -- 172.21.15.150:0/1368062477 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5612db527780 con 0x5612db56e120 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.441+0000 7f4152ffd700 10 client.25682 handle_command_reply: tid=0 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.441+0000 7f4189e55b80 1 client.25682 shutdown 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 msgr2=0x5612db562c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db562c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f416800bd00 tx=0x7f4168010ec0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 shutdown_connections 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db56e120 0x5612db570570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db569b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db5626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4152ffd700 0 client.25682 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1368062477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db562c00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 >> 172.21.15.150:0/1368062477 conn(0x5612db4baa60 msgr2=0x5612db552fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 shutdown_connections 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 1 -- 172.21.15.150:0/1368062477 wait complete. 2024-05-10T04:40:51.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.442+0000 7f4189e55b80 20 client.25682 trim_cache size 0 max 16384 2024-05-10T04:40:51.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 Processor -- start 2024-05-10T04:40:51.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 -- start start 2024-05-10T04:40:51.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db55c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db567b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db568050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db56ac60 con 0x5612db5586a0 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db56a5c0 con 0x5612db559570 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4189e55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db563760 con 0x5612db559ec0 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f41737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db55c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f41737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db55c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37748/0 (socket says 172.21.15.150:37748) 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f41737fe700 1 -- 172.21.15.150:0/2540233297 learned_addr learned my addr 172.21.15.150:0/2540233297 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4172ffd700 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db567b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.456+0000 7f4173fff700 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db568050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f41737fe700 1 -- 172.21.15.150:0/2540233297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 msgr2=0x5612db568050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f41737fe700 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db568050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f41737fe700 1 -- 172.21.15.150:0/2540233297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 msgr2=0x5612db567b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f41737fe700 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db567b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f41737fe700 1 -- 172.21.15.150:0/2540233297 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612db43e560 con 0x5612db5586a0 2024-05-10T04:40:51.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f4173fff700 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db568050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:51.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f41737fe700 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db55c7a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x5612db570e30 tx=0x7f4164013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4438029ce188234f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f415a7fc700 1 -- 172.21.15.150:0/2540233297 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416402d070 con 0x5612db5586a0 2024-05-10T04:40:51.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f415a7fc700 1 -- 172.21.15.150:0/2540233297 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4164015070 con 0x5612db5586a0 2024-05-10T04:40:51.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.457+0000 7f415a7fc700 1 -- 172.21.15.150:0/2540233297 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164024040 con 0x5612db5586a0 2024-05-10T04:40:51.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 -- 172.21.15.150:0/2540233297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 msgr2=0x5612db55c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db55c7a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x5612db570e30 tx=0x7f4164013070 comp rx=0 tx=0).stop 2024-05-10T04:40:51.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 -- 172.21.15.150:0/2540233297 shutdown_connections 2024-05-10T04:40:51.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db568050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db5586a0 0x5612db55c7a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 --2- 172.21.15.150:0/2540233297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db559570 0x5612db567b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 -- 172.21.15.150:0/2540233297 >> 172.21.15.150:0/2540233297 conn(0x5612db4baa60 msgr2=0x5612db558d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 -- 172.21.15.150:0/2540233297 shutdown_connections 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 -- 172.21.15.150:0/2540233297 wait complete. 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.458+0000 7f4189e55b80 1 -- start start 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 1 Processor -- start 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db5586a0 0x5612db5be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db5c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db5c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db568380 con 0x5612db559570 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db5c35a0 con 0x5612db5586a0 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612db5c36e0 con 0x5612db559ec0 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4189e55b80 10 client.0 _ll_register_callbacks cb 0x5612db41db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f41737fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db5586a0 0x5612db5be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4172ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db5c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f41737fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db5586a0 0x5612db5be5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37262/0 (socket says 172.21.15.150:37262) 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f41737fe700 1 -- 172.21.15.150:0/1065488404 learned_addr learned my addr 172.21.15.150:0/1065488404 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4173fff700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db5c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4172ffd700 1 -- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 msgr2=0x5612db5c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4172ffd700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db5c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4172ffd700 1 -- 172.21.15.150:0/1065488404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db5586a0 msgr2=0x5612db5be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4172ffd700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db5586a0 0x5612db5be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.459+0000 7f4172ffd700 1 -- 172.21.15.150:0/1065488404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4164010e70 con 0x5612db559570 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4173fff700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db5c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f41737fe700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db5586a0 0x5612db5be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4172ffd700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db5c0b10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f416800bd30 tx=0x7f41680055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a226b39cad1bb441 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4152ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:51.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4152ffd700 1 -- 172.21.15.150:0/1065488404 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168011ae0 con 0x5612db559570 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4152ffd700 1 -- 172.21.15.150:0/1065488404 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4168000e40 con 0x5612db559570 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4168015070 con 0x5612db559570 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4152ffd700 1 -- 172.21.15.150:0/1065488404 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41680056f0 con 0x5612db559570 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 10 client.16047 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612db2fdcd0 con 0x5612db559570 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4152ffd700 1 -- 172.21.15.150:0/1065488404 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4168015070 con 0x5612db559570 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4152ffd700 1 -- 172.21.15.150:0/1065488404 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4168004030 con 0x5612db559570 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 10 client.16047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 20 client.16047 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db5c4d70 0x5612db5c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.460+0000 7f4189e55b80 4 client.16047 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.461+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612db470ec0 con 0x5612db5c4d70 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.461+0000 7f41737fe700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db5c4d70 0x5612db5c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.461+0000 7f4152ffd700 10 client.16047 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.461+0000 7f41737fe700 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db5c4d70 0x5612db5c7190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2a3ec998578bbe5 server_cookie=ea199d9ec7157ad in_seq=0 out_seq=0 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.461+0000 7f4152ffd700 1 -- 172.21.15.150:0/1065488404 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612db470ec0 con 0x5612db5c4d70 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.461+0000 7f4152ffd700 10 client.16047 handle_command_reply: tid=0 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.461+0000 7f4189e55b80 1 client.16047 shutdown 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 msgr2=0x5612db5c0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db5c0b10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f416800bd30 tx=0x7f41680055d0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 shutdown_connections 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5612db5c4d70 0x5612db5c7190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612db559ec0 0x5612db5c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5612db5586a0 0x5612db5be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 --2- 172.21.15.150:0/1065488404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5612db559570 0x5612db5c0b10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4152ffd700 0 client.16047 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 >> 172.21.15.150:0/1065488404 conn(0x5612db4baa60 msgr2=0x5612db558d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 shutdown_connections 2024-05-10T04:40:51.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 1 -- 172.21.15.150:0/1065488404 wait complete. 2024-05-10T04:40:51.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.462+0000 7f4189e55b80 20 client.16047 trim_cache size 0 max 16384 2024-05-10T04:40:51.469 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f415c0a11c0 msgr2=0x7f415c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f415c0a11c0 0x7f415c0a36d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f417406a8e0 tx=0x7f416400b410 comp rx=0 tx=0).stop 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 msgr2=0x7f417419f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f417419f530 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f416c004c80 tx=0x7f416c004cb0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 shutdown_connections 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4174107890 0x7f417419f530 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f415c0a11c0 0x7f415c0a36d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f417406a4d0 0x7f4174198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 --2- 172.21.15.150:0/4267430071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4174104d20 0x7f41741985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 >> 172.21.15.150:0/4267430071 conn(0x7f41740fc0c0 msgr2=0x7f4174102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 shutdown_connections 2024-05-10T04:40:51.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.463+0000 7f4179b11700 1 -- 172.21.15.150:0/4267430071 wait complete. 2024-05-10T04:40:51.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 -- 172.21.15.150:0/1681314520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec101c00 msgr2=0x7f3cec102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/1681314520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec101c00 0x7f3cec102010 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009990 tx=0x7f3ce4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 -- 172.21.15.150:0/1681314520 shutdown_connections 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/1681314520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec1036c0 0x7f3cec109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/1681314520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec101c00 0x7f3cec102010 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/1681314520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cec102d20 0x7f3cec103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 -- 172.21.15.150:0/1681314520 >> 172.21.15.150:0/1681314520 conn(0x7f3cec0fd210 msgr2=0x7f3cec0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 -- 172.21.15.150:0/1681314520 shutdown_connections 2024-05-10T04:40:51.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.612+0000 7f3cf09f5700 1 -- 172.21.15.150:0/1681314520 wait complete. 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.613+0000 7f3cf09f5700 1 Processor -- start 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.613+0000 7f3cf09f5700 1 -- start start 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.613+0000 7f3cf09f5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec101c00 0x7f3cec075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.613+0000 7f3cf09f5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec102d20 0x7f3cec074670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.613+0000 7f3cf09f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cec1036c0 0x7f3cec074bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3cf09f5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec075290 con 0x7f3cec102d20 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3cf09f5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec075400 con 0x7f3cec101c00 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3cf09f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec1a1fb0 con 0x7f3cec1036c0 2024-05-10T04:40:51.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec102d20 0x7f3cec074670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec102d20 0x7f3cec074670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37776/0 (socket says 172.21.15.150:37776) 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/2184663309 learned_addr learned my addr 172.21.15.150:0/2184663309 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3cea59c700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec101c00 0x7f3cec075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3cead9d700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cec1036c0 0x7f3cec074bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/2184663309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cec1036c0 msgr2=0x7f3cec074bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cec1036c0 0x7f3cec074bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/2184663309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec101c00 msgr2=0x7f3cec075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec101c00 0x7f3cec075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/2184663309 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40095f0 con 0x7f3cec102d20 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec102d20 0x7f3cec074670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e9a0 tx=0x7f3cd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce37fe700 1 -- 172.21.15.150:0/2184663309 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4009790 con 0x7f3cec102d20 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce37fe700 1 -- 172.21.15.150:0/2184663309 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd400cbd0 con 0x7f3cec102d20 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3ce37fe700 1 -- 172.21.15.150:0/2184663309 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4018680 con 0x7f3cec102d20 2024-05-10T04:40:51.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec1a2210 con 0x7f3cec102d20 2024-05-10T04:40:51.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.614+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec1a26a0 con 0x7f3cec102d20 2024-05-10T04:40:51.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.617+0000 7f3ce37fe700 1 -- 172.21.15.150:0/2184663309 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3cd40187e0 con 0x7f3cec102d20 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.617+0000 7f3ce37fe700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cd80a11c0 0x7f3cd80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.617+0000 7f3cea59c700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cd80a11c0 0x7f3cd80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.617+0000 7f3ce37fe700 1 -- 172.21.15.150:0/2184663309 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3cd4014070 con 0x7f3cec102d20 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3cea59c700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cd80a11c0 0x7f3cd80a36d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4006010 tx=0x7f3ce401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 Processor -- start 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 -- start start 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbec570 0x562c6fbec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbf37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fb12150 con 0x562c6fbec570 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fb0fb10 con 0x562c6fbecec0 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fb0b400 con 0x562c6fbeb6a0 2024-05-10T04:40:51.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3ce9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbec570 0x562c6fbec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3ce9d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbec570 0x562c6fbec980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37790/0 (socket says 172.21.15.150:37790) 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.618+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/4033074532 learned_addr learned my addr 172.21.15.150:0/4033074532 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3cea59c700 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3cead9d700 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbf37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/4033074532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 msgr2=0x562c6fbebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/4033074532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 msgr2=0x562c6fbf37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbf37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce9d9b700 1 -- 172.21.15.150:0/4033074532 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6f98c3b0 con 0x562c6fbec570 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3cead9d700 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbf37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbec570 0x562c6fbec980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4096440 tx=0x7f3cd4099920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1431ba8d48d6abc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce0ff9700 1 -- 172.21.15.150:0/4033074532 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4099f40 con 0x562c6fbec570 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce0ff9700 1 -- 172.21.15.150:0/4033074532 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd409dd70 con 0x562c6fbec570 2024-05-10T04:40:51.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.619+0000 7f3ce0ff9700 1 -- 172.21.15.150:0/4033074532 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4099690 con 0x562c6fbec570 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 -- 172.21.15.150:0/4033074532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbec570 msgr2=0x562c6fbec980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbec570 0x562c6fbec980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4096440 tx=0x7f3cd4099920 comp rx=0 tx=0).stop 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 -- 172.21.15.150:0/4033074532 shutdown_connections 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbf37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/4033074532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbec570 0x562c6fbec980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 -- 172.21.15.150:0/4033074532 >> 172.21.15.150:0/4033074532 conn(0x562c6fb4da60 msgr2=0x562c6fbe5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 -- 172.21.15.150:0/4033074532 shutdown_connections 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 -- 172.21.15.150:0/4033074532 wait complete. 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.620+0000 7f3d00d39b80 1 -- start start 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 1 Processor -- start 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbeb6a0 0x562c6fbf5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 0x562c6fbf5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6faa7560 con 0x562c6fbecec0 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6faa7200 con 0x562c6fbeb6a0 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6f98e6a0 con 0x562c6fbec570 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3d00d39b80 10 client.0 _ll_register_callbacks cb 0x562c6fab0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37802/0 (socket says 172.21.15.150:37802) 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 -- 172.21.15.150:0/1202479877 learned_addr learned my addr 172.21.15.150:0/1202479877 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cea59c700 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbeb6a0 0x562c6fbf5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 -- 172.21.15.150:0/1202479877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 msgr2=0x562c6fbf5ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 0x562c6fbf5ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 -- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbeb6a0 msgr2=0x562c6fbf5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbeb6a0 0x562c6fbf5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.621+0000 7f3cead9d700 1 -- 172.21.15.150:0/1202479877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec1a26a0 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3cead9d700 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfcc00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc007ae0 tx=0x7f3cdc007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca5cb7cff112b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3cd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/1202479877 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc004bb0 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c6fa64270 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/1202479877 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3cdc004d10 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/1202479877 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cdc003a20 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3d00d39b80 10 client.16059 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562c6f938b60 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/1202479877 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c6fa64270 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/1202479877 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3cdc004090 con 0x562c6fbecec0 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3d00d39b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3d00d39b80 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3d00d39b80 20 client.16059 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.622+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc01480 0x562c6fc038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.623+0000 7f3d00d39b80 4 client.16059 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:51.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.623+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c6fbba780 con 0x562c6fc01480 2024-05-10T04:40:51.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.623+0000 7f3cea59c700 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc01480 0x562c6fc038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.623+0000 7f3cd1ffb700 10 client.16059 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:51.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.623+0000 7f3cea59c700 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc01480 0x562c6fc038d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6144afae4562b2 server_cookie=a11ecd43d5f39278 in_seq=0 out_seq=0 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/1202479877 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c6fbba780 con 0x562c6fc01480 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3cd1ffb700 10 client.16059 handle_command_reply: tid=0 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 client.16059 shutdown 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 msgr2=0x562c6fbfcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfcc00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc007ae0 tx=0x7f3cdc007ea0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 shutdown_connections 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc01480 0x562c6fc038d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfcc00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbeb6a0 0x562c6fbf5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1202479877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 0x562c6fbf5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3cd1ffb700 0 client.16059 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:51.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 >> 172.21.15.150:0/1202479877 conn(0x562c6fb4da60 msgr2=0x562c6fbe5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 shutdown_connections 2024-05-10T04:40:51.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1202479877 wait complete. 2024-05-10T04:40:51.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.624+0000 7f3d00d39b80 20 client.16059 trim_cache size 0 max 16384 2024-05-10T04:40:51.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 Processor -- start 2024-05-10T04:40:51.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 -- start start 2024-05-10T04:40:51.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 0x562c6fbf4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbfcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fbebf50 con 0x562c6fbeb6a0 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fc01100 con 0x562c6fbecec0 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fc038a0 con 0x562c6fbec570 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37818/0 (socket says 172.21.15.150:37818) 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 -- 172.21.15.150:0/1201622539 learned_addr learned my addr 172.21.15.150:0/1201622539 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 0x562c6fbf4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cead9d700 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbfcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 -- 172.21.15.150:0/1201622539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 msgr2=0x562c6fbf4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 0x562c6fbf4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 -- 172.21.15.150:0/1201622539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 msgr2=0x562c6fbfcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbfcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cea59c700 1 -- 172.21.15.150:0/1201622539 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6fad1560 con 0x562c6fbeb6a0 2024-05-10T04:40:51.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.639+0000 7f3cead9d700 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbfcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:51.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.640+0000 7f3cea59c700 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfd1e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3cec1018f0 tx=0x7f3ce401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1547612db052c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.640+0000 7f3ce17fa700 1 -- 172.21.15.150:0/1201622539 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce402d040 con 0x562c6fbeb6a0 2024-05-10T04:40:51.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.640+0000 7f3ce17fa700 1 -- 172.21.15.150:0/1201622539 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce402e040 con 0x562c6fbeb6a0 2024-05-10T04:40:51.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.640+0000 7f3ce17fa700 1 -- 172.21.15.150:0/1201622539 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40348b0 con 0x562c6fbeb6a0 2024-05-10T04:40:51.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1201622539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbeb6a0 msgr2=0x562c6fbfd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfd1e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3cec1018f0 tx=0x7f3ce401f000 comp rx=0 tx=0).stop 2024-05-10T04:40:51.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1201622539 shutdown_connections 2024-05-10T04:40:51.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbecec0 0x562c6fbfcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfd1e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/1201622539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbec570 0x562c6fbf4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1201622539 >> 172.21.15.150:0/1201622539 conn(0x562c6fb4da60 msgr2=0x562c6fbe5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1201622539 shutdown_connections 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- 172.21.15.150:0/1201622539 wait complete. 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- start start 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 Processor -- start 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbec570 0x562c6fbface0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fbfc150 con 0x562c6fbecec0 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fbfc2c0 con 0x562c6fbec570 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6fbfc480 con 0x562c6fbeb6a0 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.641+0000 7f3d00d39b80 10 client.0 _ll_register_callbacks cb 0x562c6fab0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:51.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37834/0 (socket says 172.21.15.150:37834) 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 -- 172.21.15.150:0/342382702 learned_addr learned my addr 172.21.15.150:0/342382702 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cea59c700 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3ce9d9b700 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbec570 0x562c6fbface0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 -- 172.21.15.150:0/342382702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 msgr2=0x562c6fbfa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 -- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbec570 msgr2=0x562c6fbface0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbec570 0x562c6fbface0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 -- 172.21.15.150:0/342382702 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec10aa00 con 0x562c6fbecec0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cead9d700 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfbc10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0043f0 tx=0x7f3cdc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaeeba0f92545a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/342382702 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0107e0 con 0x562c6fbecec0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/342382702 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3cdc029070 con 0x562c6fbecec0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.642+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cd409dd70 con 0x562c6fbecec0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/342382702 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cdc010940 con 0x562c6fbecec0 2024-05-10T04:40:51.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 10 client.16065 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562c6f990cd0 con 0x562c6fbecec0 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/342382702 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd409dd70 con 0x562c6fbecec0 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/342382702 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3cdc005040 con 0x562c6fbecec0 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 20 client.16065 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc58510 0x562c6fc5a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 4 client.16065 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c6fb03ec0 con 0x562c6fc58510 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.643+0000 7f3cea59c700 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc58510 0x562c6fc5a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3cd1ffb700 10 client.16065 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3cea59c700 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc58510 0x562c6fc5a930 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95be2cc219e29b38 server_cookie=4ce2d92b4eb55e8a in_seq=0 out_seq=0 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3cd1ffb700 1 -- 172.21.15.150:0/342382702 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c6fb03ec0 con 0x562c6fc58510 2024-05-10T04:40:51.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3cd1ffb700 10 client.16065 handle_command_reply: tid=0 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 client.16065 shutdown 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 msgr2=0x562c6fbfbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfbc10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0043f0 tx=0x7f3cdc005be0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 shutdown_connections 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x562c6fc58510 0x562c6fc5a930 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c6fbecec0 0x562c6fbfbc10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562c6fbeb6a0 0x562c6fbfa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3cd1ffb700 0 client.16065 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 --2- 172.21.15.150:0/342382702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562c6fbec570 0x562c6fbface0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 >> 172.21.15.150:0/342382702 conn(0x562c6fb4da60 msgr2=0x562c6fbe5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 shutdown_connections 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 1 -- 172.21.15.150:0/342382702 wait complete. 2024-05-10T04:40:51.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.644+0000 7f3d00d39b80 20 client.16065 trim_cache size 0 max 16384 2024-05-10T04:40:51.652 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:40:51.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cd80a11c0 msgr2=0x7f3cd80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cd80a11c0 0x7f3cd80a36d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4006010 tx=0x7f3ce401a040 comp rx=0 tx=0).stop 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec102d20 msgr2=0x7f3cec074670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec102d20 0x7f3cec074670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e9a0 tx=0x7f3cd400ecb0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 shutdown_connections 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3cd80a11c0 0x7f3cd80a36d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cec1036c0 0x7f3cec074bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec101c00 0x7f3cec075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 --2- 172.21.15.150:0/2184663309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cec102d20 0x7f3cec074670 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 >> 172.21.15.150:0/2184663309 conn(0x7f3cec0fd210 msgr2=0x7f3cec105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 shutdown_connections 2024-05-10T04:40:51.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:51.645+0000 7f3cf09f5700 1 -- 172.21.15.150:0/2184663309 wait complete. 2024-05-10T04:40:51.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:51 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1246679733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:40:51.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:51 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1246679733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:40:51.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:51 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1246679733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:40:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.628+0000 7f3baacb4700 1 -- 172.21.15.150:0/1396973559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4101d30 msgr2=0x7f3ba41085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.628+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1396973559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4101d30 0x7f3ba41085e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00c210 tx=0x7f3b9c00c520 comp rx=0 tx=0).stop 2024-05-10T04:40:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.628+0000 7f3baacb4700 1 -- 172.21.15.150:0/1396973559 shutdown_connections 2024-05-10T04:40:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1396973559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4101d30 0x7f3ba41085e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1396973559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba4100270 0x7f3ba4100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1396973559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba4101390 0x7f3ba41017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 -- 172.21.15.150:0/1396973559 >> 172.21.15.150:0/1396973559 conn(0x7f3ba40fb8a0 msgr2=0x7f3ba40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 -- 172.21.15.150:0/1396973559 shutdown_connections 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 -- 172.21.15.150:0/1396973559 wait complete. 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 Processor -- start 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 -- start start 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4100270 0x7f3ba41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba4101390 0x7f3ba4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.629+0000 7f3baacb4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba4101d30 0x7f3ba419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3baacb4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419fb70 con 0x7f3ba4100270 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3baacb4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419fcb0 con 0x7f3ba4101d30 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3baacb4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419fe40 con 0x7f3ba4101390 2024-05-10T04:40:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4100270 0x7f3ba41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4100270 0x7f3ba41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37858/0 (socket says 172.21.15.150:37858) 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1267195834 learned_addr learned my addr 172.21.15.150:0/1267195834 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba9251700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba4101d30 0x7f3ba419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba4101390 0x7f3ba4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1267195834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba4101390 msgr2=0x7f3ba4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba4101390 0x7f3ba4198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1267195834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba4101d30 msgr2=0x7f3ba419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba4101d30 0x7f3ba419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1267195834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c00ae10 con 0x7f3ba4100270 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba4101390 0x7f3ba4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4100270 0x7f3ba41980d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800d980 tx=0x7f3b9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba1ffb700 1 -- 172.21.15.150:0/1267195834 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800b950 con 0x7f3ba4100270 2024-05-10T04:40:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba41a0120 con 0x7f3ba4100270 2024-05-10T04:40:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba1ffb700 1 -- 172.21.15.150:0/1267195834 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b98007620 con 0x7f3ba4100270 2024-05-10T04:40:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.630+0000 7f3ba1ffb700 1 -- 172.21.15.150:0/1267195834 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800f3c0 con 0x7f3ba4100270 2024-05-10T04:40:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.631+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba41a0680 con 0x7f3ba4100270 2024-05-10T04:40:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.633+0000 7f3ba1ffb700 1 -- 172.21.15.150:0/1267195834 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3b980105d0 con 0x7f3ba4100270 2024-05-10T04:40:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.634+0000 7f3ba1ffb700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3b940a1120 0x7f3b940a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.634+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3b940a1120 0x7f3b940a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.634+0000 7f3ba1ffb700 1 -- 172.21.15.150:0/1267195834 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3b980ccf30 con 0x7f3ba4100270 2024-05-10T04:40:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.634+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3b940a1120 0x7f3b940a3630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3b90005e10 tx=0x7f3b90005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 Processor -- start 2024-05-10T04:40:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 -- start start 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac4559b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac4559c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac454c20f0 con 0x55ac4559b180 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac454bfab0 con 0x55ac4559c9a0 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac454bb3a0 con 0x55ac4559c050 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac4559b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac4559b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37874/0 (socket says 172.21.15.150:37874) 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3ba8a50700 1 -- 172.21.15.150:0/3846615251 learned_addr learned my addr 172.21.15.150:0/3846615251 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3ba9251700 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.635+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac4559c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3ba8a50700 1 -- 172.21.15.150:0/3846615251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 msgr2=0x55ac4559c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac4559c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3ba8a50700 1 -- 172.21.15.150:0/3846615251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 msgr2=0x55ac455a32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455a32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3ba8a50700 1 -- 172.21.15.150:0/3846615251 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac4533c350 con 0x55ac4559b180 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac4559b5b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3b98099f10 tx=0x7f3b98099bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600bd0d1cf49491 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3b8effd700 1 -- 172.21.15.150:0/3846615251 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9809ddb0 con 0x55ac4559b180 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3b8effd700 1 -- 172.21.15.150:0/3846615251 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b980a2400 con 0x55ac4559b180 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.636+0000 7f3b8effd700 1 -- 172.21.15.150:0/3846615251 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980a21b0 con 0x55ac4559b180 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3846615251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 msgr2=0x55ac4559b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac4559b5b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3b98099f10 tx=0x7f3b98099bf0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3846615251 shutdown_connections 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455a32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac4559b5b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3846615251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac4559c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3846615251 >> 172.21.15.150:0/3846615251 conn(0x55ac454fda00 msgr2=0x55ac45595a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3846615251 shutdown_connections 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3846615251 wait complete. 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 -- start start 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.637+0000 7f3bbaff8b80 1 Processor -- start 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559b180 0x55ac455a5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac45457500 con 0x55ac4559c9a0 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac454571a0 con 0x55ac4559b180 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4533e640 con 0x55ac4559c050 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3bbaff8b80 10 client.0 _ll_register_callbacks cb 0x55ac45460ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3ba9251700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559b180 0x55ac455a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3ba9251700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37882/0 (socket says 172.21.15.150:37882) 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3ba9251700 1 -- 172.21.15.150:0/3871797816 learned_addr learned my addr 172.21.15.150:0/3871797816 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.638+0000 7f3ba9251700 1 -- 172.21.15.150:0/3871797816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 msgr2=0x55ac455a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3ba9251700 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3ba9251700 1 -- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559b180 msgr2=0x55ac455a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3ba9251700 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559b180 0x55ac455a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3ba9251700 1 -- 172.21.15.150:0/3871797816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba41a0680 con 0x55ac4559c9a0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559b180 0x55ac455a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3ba9251700 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455ac630 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00aa10 tx=0x7f3b9c008270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e5b46f6dc0277 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3b83fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3b83fff700 1 -- 172.21.15.150:0/3871797816 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00f040 con 0x55ac4559c9a0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac45414210 con 0x55ac4559c9a0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3b83fff700 1 -- 172.21.15.150:0/3871797816 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3b9c003e10 con 0x55ac4559c9a0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3b83fff700 1 -- 172.21.15.150:0/3871797816 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b9c00a0d0 con 0x55ac4559c9a0 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3bbaff8b80 10 client.16077 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ac452e8b00 con 0x55ac4559c9a0 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3b83fff700 1 -- 172.21.15.150:0/3871797816 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac45414210 con 0x55ac4559c9a0 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.639+0000 7f3b83fff700 1 -- 172.21.15.150:0/3871797816 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b9c024020 con 0x55ac4559c9a0 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3bbaff8b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3bbaff8b80 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3bbaff8b80 20 client.16077 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac455b0e10 0x55ac455b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3bbaff8b80 4 client.16077 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac4556a720 con 0x55ac455b0e10 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac455b0e10 0x55ac455b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3b83fff700 10 client.16077 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.640+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac455b0e10 0x55ac455b3260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b36247e52550eb2 server_cookie=865ff43771b753ec in_seq=0 out_seq=0 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3b83fff700 1 -- 172.21.15.150:0/3871797816 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac4556a720 con 0x55ac455b0e10 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3b83fff700 10 client.16077 handle_command_reply: tid=0 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 client.16077 shutdown 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 msgr2=0x55ac455ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455ac630 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00aa10 tx=0x7f3b9c008270 comp rx=0 tx=0).stop 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 shutdown_connections 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac455b0e10 0x55ac455b3260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455ac630 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3b83fff700 0 client.16077 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559b180 0x55ac455a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/3871797816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 >> 172.21.15.150:0/3871797816 conn(0x55ac454fda00 msgr2=0x55ac45595a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 shutdown_connections 2024-05-10T04:40:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.641+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/3871797816 wait complete. 2024-05-10T04:40:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.642+0000 7f3bbaff8b80 20 client.16077 trim_cache size 0 max 16384 2024-05-10T04:40:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.656+0000 7f3bbaff8b80 1 Processor -- start 2024-05-10T04:40:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.656+0000 7f3bbaff8b80 1 -- start start 2024-05-10T04:40:52.659 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac455a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455ac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac455967a0 con 0x55ac4559b180 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac455a3260 con 0x55ac4559c9a0 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac455a0b00 con 0x55ac4559c050 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac455a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac455a35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37892/0 (socket says 172.21.15.150:37892) 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1115445139 learned_addr learned my addr 172.21.15.150:0/1115445139 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba9251700 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1115445139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 msgr2=0x55ac455a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1115445139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 msgr2=0x55ac455ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 -- 172.21.15.150:0/1115445139 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac45481500 con 0x55ac4559b180 2024-05-10T04:40:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba9251700 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.657+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac455a35b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3b9809a5e0 tx=0x7f3b9809af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f62cfe542f33cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.658+0000 7f3b8f7fe700 1 -- 172.21.15.150:0/1115445139 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98098320 con 0x55ac4559b180 2024-05-10T04:40:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.658+0000 7f3b8f7fe700 1 -- 172.21.15.150:0/1115445139 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b980a1f70 con 0x55ac4559b180 2024-05-10T04:40:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.658+0000 7f3b8f7fe700 1 -- 172.21.15.150:0/1115445139 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98095d70 con 0x55ac4559b180 2024-05-10T04:40:52.661 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:40:52.662 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:40:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/1115445139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 msgr2=0x55ac455a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac455a35b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3b9809a5e0 tx=0x7f3b9809af90 comp rx=0 tx=0).stop 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/1115445139 shutdown_connections 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c9a0 0x55ac455ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559b180 0x55ac455a35b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/1115445139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559c050 0x55ac455a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/1115445139 >> 172.21.15.150:0/1115445139 conn(0x55ac454fda00 msgr2=0x55ac4559fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/1115445139 shutdown_connections 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/1115445139 wait complete. 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 -- start start 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.659+0000 7f3bbaff8b80 1 Processor -- start 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559b180 0x55ac455a9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c050 0x55ac455a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3bbaff8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455aad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac455ab280 con 0x55ac4559c9a0 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac455ab3f0 con 0x55ac4559c050 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3bbaff8b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac455ab5b0 con 0x55ac4559b180 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3bbaff8b80 10 client.0 _ll_register_callbacks cb 0x55ac45460ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3ba9251700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3ba8a50700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559b180 0x55ac455a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3ba9251700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455aad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37900/0 (socket says 172.21.15.150:37900) 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3ba9251700 1 -- 172.21.15.150:0/2959581783 learned_addr learned my addr 172.21.15.150:0/2959581783 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.660+0000 7f3ba3fff700 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c050 0x55ac455a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3ba9251700 1 -- 172.21.15.150:0/2959581783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559b180 msgr2=0x55ac455a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3ba9251700 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559b180 0x55ac455a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3ba9251700 1 -- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c050 msgr2=0x55ac455a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3ba9251700 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c050 0x55ac455a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3ba9251700 1 -- 172.21.15.150:0/2959581783 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac4533c350 con 0x55ac4559c9a0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559b180 0x55ac455a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3ba9251700 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455aad40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00ade0 tx=0x7f3b9c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449ad2f80eb730a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3b83fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3b83fff700 1 -- 172.21.15.150:0/2959581783 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00f040 con 0x55ac4559c9a0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3b83fff700 1 -- 172.21.15.150:0/2959581783 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3b9c0074a0 con 0x55ac4559c9a0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b980a2400 con 0x55ac4559c9a0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3b83fff700 1 -- 172.21.15.150:0/2959581783 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b9c007ed0 con 0x55ac4559c9a0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3bbaff8b80 10 client.16083 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ac45340c70 con 0x55ac4559c9a0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3b83fff700 1 -- 172.21.15.150:0/2959581783 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b980a2400 con 0x55ac4559c9a0 2024-05-10T04:40:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3b83fff700 1 -- 172.21.15.150:0/2959581783 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b9c01a040 con 0x55ac4559c9a0 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3bbaff8b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3bbaff8b80 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.661+0000 7f3bbaff8b80 20 client.16083 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.662+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac456072c0 0x55ac456096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.662+0000 7f3bbaff8b80 4 client.16083 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.662+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ac454b3e60 con 0x55ac456072c0 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.662+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac456072c0 0x55ac456096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.662+0000 7f3b83fff700 10 client.16083 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.662+0000 7f3ba8a50700 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac456072c0 0x55ac456096e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6cb7c8837c1ca9 server_cookie=fb2373cc3c84f25b in_seq=0 out_seq=0 2024-05-10T04:40:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3b83fff700 1 -- 172.21.15.150:0/2959581783 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ac454b3e60 con 0x55ac456072c0 2024-05-10T04:40:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3b83fff700 10 client.16083 handle_command_reply: tid=0 2024-05-10T04:40:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 client.16083 shutdown 2024-05-10T04:40:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 msgr2=0x55ac455aad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455aad40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00ade0 tx=0x7f3b9c00cb40 comp rx=0 tx=0).stop 2024-05-10T04:40:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 shutdown_connections 2024-05-10T04:40:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ac456072c0 0x55ac456096e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac4559c9a0 0x55ac455aad40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac4559b180 0x55ac455a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 --2- 172.21.15.150:0/2959581783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ac4559c050 0x55ac455a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 >> 172.21.15.150:0/2959581783 conn(0x55ac454fda00 msgr2=0x55ac455a3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.668+0000 7f3b83fff700 0 client.16083 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 shutdown_connections 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3bbaff8b80 1 -- 172.21.15.150:0/2959581783 wait complete. 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3bbaff8b80 20 client.16083 trim_cache size 0 max 16384 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:40:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:40:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3b940a1120 msgr2=0x7f3b940a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3b940a1120 0x7f3b940a3630 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3b90005e10 tx=0x7f3b90005d60 comp rx=0 tx=0).stop 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4100270 msgr2=0x7f3ba41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4100270 0x7f3ba41980d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800d980 tx=0x7f3b9800dc90 comp rx=0 tx=0).stop 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.669+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 shutdown_connections 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.670+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ba4101d30 0x7f3ba419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.670+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ba4100270 0x7f3ba41980d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.670+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3b940a1120 0x7f3b940a3630 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.670+0000 7f3baacb4700 1 --2- 172.21.15.150:0/1267195834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba4101390 0x7f3ba4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.670+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 >> 172.21.15.150:0/1267195834 conn(0x7f3ba40fb8a0 msgr2=0x7f3ba41045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.670+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 shutdown_connections 2024-05-10T04:40:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:40:52.670+0000 7f3baacb4700 1 -- 172.21.15.150:0/1267195834 wait complete. 2024-05-10T04:40:52.689 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:52 smithi150 ceph-mon[42404]: pgmap v310: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T04:40:52.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.821+0000 7fc592844700 1 -- 172.21.15.150:0/3356427345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c100ac0 msgr2=0x7fc58c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.821+0000 7fc592844700 1 --2- 172.21.15.150:0/3356427345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c100ac0 0x7fc58c100ed0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009990 tx=0x7fc57c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 -- 172.21.15.150:0/3356427345 shutdown_connections 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 --2- 172.21.15.150:0/3356427345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c102580 0x7fc58c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 --2- 172.21.15.150:0/3356427345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c100ac0 0x7fc58c100ed0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 --2- 172.21.15.150:0/3356427345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc58c101be0 0x7fc58c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 -- 172.21.15.150:0/3356427345 >> 172.21.15.150:0/3356427345 conn(0x7fc58c0fc0d0 msgr2=0x7fc58c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 -- 172.21.15.150:0/3356427345 shutdown_connections 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 -- 172.21.15.150:0/3356427345 wait complete. 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 Processor -- start 2024-05-10T04:40:52.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.822+0000 7fc592844700 1 -- start start 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc592844700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc58c100ac0 0x7fc58c068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc592844700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c101be0 0x7fc58c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc592844700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c102580 0x7fc58c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc592844700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c1a1b90 con 0x7fc58c102580 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc592844700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c1a1d00 con 0x7fc58c101be0 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc592844700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c1a1ec0 con 0x7fc58c100ac0 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c102580 0x7fc58c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc58b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c101be0 0x7fc58c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c102580 0x7fc58c1a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37924/0 (socket says 172.21.15.150:37924) 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 -- 172.21.15.150:0/2525717766 learned_addr learned my addr 172.21.15.150:0/2525717766 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc58bfff700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc58c100ac0 0x7fc58c068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 -- 172.21.15.150:0/2525717766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc58c100ac0 msgr2=0x7fc58c068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc58c100ac0 0x7fc58c068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 -- 172.21.15.150:0/2525717766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c101be0 msgr2=0x7fc58c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c101be0 0x7fc58c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 -- 172.21.15.150:0/2525717766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c0095f0 con 0x7fc58c102580 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c101be0 0x7fc58c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc58bfff700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc58c100ac0 0x7fc58c068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:52.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.823+0000 7fc590de1700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c102580 0x7fc58c1a15c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ef30 tx=0x7fc58400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.824+0000 7fc5897fa700 1 -- 172.21.15.150:0/2525717766 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584018070 con 0x7fc58c102580 2024-05-10T04:40:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.824+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c1a2140 con 0x7fc58c102580 2024-05-10T04:40:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.824+0000 7fc5897fa700 1 -- 172.21.15.150:0/2525717766 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc58400f040 con 0x7fc58c102580 2024-05-10T04:40:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.824+0000 7fc5897fa700 1 -- 172.21.15.150:0/2525717766 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400cdb0 con 0x7fc58c102580 2024-05-10T04:40:52.828 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.824+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c1a2650 con 0x7fc58c102580 2024-05-10T04:40:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.826+0000 7fc5897fa700 1 -- 172.21.15.150:0/2525717766 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc584003a00 con 0x7fc58c102580 2024-05-10T04:40:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.827+0000 7fc5897fa700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc5740a11c0 0x7fc5740a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.827+0000 7fc58bfff700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc5740a11c0 0x7fc5740a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.827+0000 7fc5897fa700 1 -- 172.21.15.150:0/2525717766 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc5840cc840 con 0x7fc58c102580 2024-05-10T04:40:52.829 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.827+0000 7fc58bfff700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc5740a11c0 0x7fc5740a36d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00b7b0 tx=0x7fc57c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.828+0000 7fc5a2b88b80 1 Processor -- start 2024-05-10T04:40:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.828+0000 7fc5a2b88b80 1 -- start start 2024-05-10T04:40:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.828+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a54720 0x563628a54b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a55a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a55f40 0x563628a5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362897b1d0 con 0x563628a55f40 2024-05-10T04:40:52.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628978b90 con 0x563628a555f0 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628974480 con 0x563628a54720 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a55f40 0x563628a5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a55f40 0x563628a5c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37940/0 (socket says 172.21.15.150:37940) 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 -- 172.21.15.150:0/4030701692 learned_addr learned my addr 172.21.15.150:0/4030701692 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a55a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc58bfff700 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a54720 0x563628a54b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 -- 172.21.15.150:0/4030701692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a54720 msgr2=0x563628a54b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a54720 0x563628a54b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 -- 172.21.15.150:0/4030701692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 msgr2=0x563628a55a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a55a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 -- 172.21.15.150:0/4030701692 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636287f5430 con 0x563628a55f40 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a55a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc590de1700 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a55f40 0x563628a5c840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc5840963f0 tx=0x7fc584099750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddd6d3350e64f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.829+0000 7fc5727fc700 1 -- 172.21.15.150:0/4030701692 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584004ad0 con 0x563628a55f40 2024-05-10T04:40:52.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5727fc700 1 -- 172.21.15.150:0/4030701692 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc58401b320 con 0x563628a55f40 2024-05-10T04:40:52.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5727fc700 1 -- 172.21.15.150:0/4030701692 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584097b50 con 0x563628a55f40 2024-05-10T04:40:52.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/4030701692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a55f40 msgr2=0x563628a5c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a55f40 0x563628a5c840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc5840963f0 tx=0x7fc584099750 comp rx=0 tx=0).stop 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/4030701692 shutdown_connections 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a55f40 0x563628a5c840 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a54720 0x563628a54b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/4030701692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a55a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/4030701692 >> 172.21.15.150:0/4030701692 conn(0x5636289b6ae0 msgr2=0x563628a4f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/4030701692 shutdown_connections 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.830+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/4030701692 wait complete. 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 -- start start 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 Processor -- start 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a5e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628a5ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a55f40 0x563628a65c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636289105e0 con 0x563628a54720 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628910280 con 0x563628a55f40 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636287f7720 con 0x563628a555f0 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.831+0000 7fc5a2b88b80 10 client.0 _ll_register_callbacks cb 0x563628919bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a5e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a5e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37948/0 (socket says 172.21.15.150:37948) 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 -- 172.21.15.150:0/2457827128 learned_addr learned my addr 172.21.15.150:0/2457827128 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc590de1700 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a55f40 0x563628a65c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628a5ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 -- 172.21.15.150:0/2457827128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 msgr2=0x563628a5ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628a5ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 -- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a55f40 msgr2=0x563628a65c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a55f40 0x563628a65c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 -- 172.21.15.150:0/2457827128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58000bcd0 con 0x563628a54720 2024-05-10T04:40:52.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc58bfff700 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a5e7f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc57c015000 tx=0x7fc57c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ed27367d9f8587 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc56b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc56b7fe700 1 -- 172.21.15.150:0/2457827128 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c02a050 con 0x563628a54720 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc56b7fe700 1 -- 172.21.15.150:0/2457827128 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc57c025830 con 0x563628a54720 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.832+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636288cd2f0 con 0x563628a54720 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc56b7fe700 1 -- 172.21.15.150:0/2457827128 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc57c01d040 con 0x563628a54720 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 10 client.16095 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fc58c1a2650 con 0x563628a54720 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc56b7fe700 1 -- 172.21.15.150:0/2457827128 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5636288cd2f0 con 0x563628a54720 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc56b7fe700 1 -- 172.21.15.150:0/2457827128 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc57c02f940 con 0x563628a54720 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 20 client.16095 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a6a4c0 0x563628a6c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 4 client.16095 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:52.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563628a23800 con 0x563628a6a4c0 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.833+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a6a4c0 0x563628a6c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.834+0000 7fc56b7fe700 10 client.16095 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.834+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a6a4c0 0x563628a6c910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c3f869f2c7a9b server_cookie=3bbf3bb020d3531b in_seq=0 out_seq=0 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.834+0000 7fc56b7fe700 1 -- 172.21.15.150:0/2457827128 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563628a23800 con 0x563628a6a4c0 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.834+0000 7fc56b7fe700 10 client.16095 handle_command_reply: tid=0 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.834+0000 7fc5a2b88b80 1 client.16095 shutdown 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 msgr2=0x563628a5e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a5e7f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc57c015000 tx=0x7fc57c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 shutdown_connections 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a6a4c0 0x563628a6c910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a55f40 0x563628a65c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a5e7f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc56b7fe700 0 client.16095 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/2457827128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628a5ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 >> 172.21.15.150:0/2457827128 conn(0x5636289b6ae0 msgr2=0x563628a4f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 shutdown_connections 2024-05-10T04:40:52.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/2457827128 wait complete. 2024-05-10T04:40:52.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.835+0000 7fc5a2b88b80 20 client.16095 trim_cache size 0 max 16384 2024-05-10T04:40:52.841 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:52 smithi161 ceph-mon[46912]: pgmap v310: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 Processor -- start 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 -- start start 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636289b6ae0 0x563628a59a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a65c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a53a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628a66dc0 con 0x563628a54720 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628a50820 con 0x563628a555f0 2024-05-10T04:40:52.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628a5d7e0 con 0x5636289b6ae0 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a65c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a65c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37958/0 (socket says 172.21.15.150:37958) 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 -- 172.21.15.150:0/802425868 learned_addr learned my addr 172.21.15.150:0/802425868 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58bfff700 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636289b6ae0 0x563628a59a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc590de1700 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a53a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 -- 172.21.15.150:0/802425868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636289b6ae0 msgr2=0x563628a59a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636289b6ae0 0x563628a59a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 -- 172.21.15.150:0/802425868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 msgr2=0x563628a53a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a53a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc58b7fe700 1 -- 172.21.15.150:0/802425868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636287a1be0 con 0x563628a54720 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.858+0000 7fc590de1700 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a53a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.859+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a65c50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0687d0 tx=0x7fc58000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd32b74ba477af3b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.859+0000 7fc572ffd700 1 -- 172.21.15.150:0/802425868 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580016040 con 0x563628a54720 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.859+0000 7fc572ffd700 1 -- 172.21.15.150:0/802425868 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc580017040 con 0x563628a54720 2024-05-10T04:40:52.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.859+0000 7fc572ffd700 1 -- 172.21.15.150:0/802425868 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800052a0 con 0x563628a54720 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/802425868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 msgr2=0x563628a65c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a65c50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0687d0 tx=0x7fc58000b410 comp rx=0 tx=0).stop 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/802425868 shutdown_connections 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a555f0 0x563628a53a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5636289b6ae0 0x563628a59a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/802425868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563628a54720 0x563628a65c50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/802425868 >> 172.21.15.150:0/802425868 conn(0x563628a4b4d0 msgr2=0x563628a53330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/802425868 shutdown_connections 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/802425868 wait complete. 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.860+0000 7fc5a2b88b80 1 -- start start 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 1 Processor -- start 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5636289b6ae0 0x563628a652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a54720 0x563628a65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628ac0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628ac14e0 con 0x5636289b6ae0 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628ac1620 con 0x563628a54720 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563628ac17b0 con 0x563628a555f0 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc5a2b88b80 10 client.0 _ll_register_callbacks cb 0x563628919bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc58bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5636289b6ae0 0x563628a652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc58bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5636289b6ae0 0x563628a652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37968/0 (socket says 172.21.15.150:37968) 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc58bfff700 1 -- 172.21.15.150:0/3987635513 learned_addr learned my addr 172.21.15.150:0/3987635513 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a54720 0x563628a65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.861+0000 7fc590de1700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628ac0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc58bfff700 1 -- 172.21.15.150:0/3987635513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 msgr2=0x563628ac0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc58bfff700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628ac0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc58bfff700 1 -- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a54720 msgr2=0x563628a65830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc58bfff700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a54720 0x563628a65830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc58bfff700 1 -- 172.21.15.150:0/3987635513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc580011940 con 0x5636289b6ae0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a54720 0x563628a65830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc58bfff700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5636289b6ae0 0x563628a652f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02fd70 tx=0x7fc57c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d809c4b7728824c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc56b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc56b7fe700 1 -- 172.21.15.150:0/3987635513 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c02a040 con 0x5636289b6ae0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc56b7fe700 1 -- 172.21.15.150:0/3987635513 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc57c02f610 con 0x5636289b6ae0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc58401b320 con 0x5636289b6ae0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc56b7fe700 1 -- 172.21.15.150:0/3987635513 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc57c01d040 con 0x5636289b6ae0 2024-05-10T04:40:52.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc5a2b88b80 10 client.16101 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56362893a5e0 con 0x5636289b6ae0 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.862+0000 7fc56b7fe700 1 -- 172.21.15.150:0/3987635513 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc58401b320 con 0x5636289b6ae0 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc56b7fe700 1 -- 172.21.15.150:0/3987635513 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc57c016600 con 0x5636289b6ae0 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc5a2b88b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc5a2b88b80 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc5a2b88b80 20 client.16101 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a55f40 0x563628ac4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc5a2b88b80 4 client.16101 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56362896cf40 con 0x563628a55f40 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a55f40 0x563628ac4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc56b7fe700 10 client.16101 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.863+0000 7fc58b7fe700 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a55f40 0x563628ac4d70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75069058afc50503 server_cookie=4a3c844ded337b4b in_seq=0 out_seq=0 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc56b7fe700 1 -- 172.21.15.150:0/3987635513 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56362896cf40 con 0x563628a55f40 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc56b7fe700 10 client.16101 handle_command_reply: tid=0 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 client.16101 shutdown 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5636289b6ae0 msgr2=0x563628a652f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5636289b6ae0 0x563628a652f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02fd70 tx=0x7fc57c005d50 comp rx=0 tx=0).stop 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 shutdown_connections 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563628a55f40 0x563628ac4d70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563628a555f0 0x563628ac0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5636289b6ae0 0x563628a652f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 --2- 172.21.15.150:0/3987635513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563628a54720 0x563628a65830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc56b7fe700 0 client.16101 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 >> 172.21.15.150:0/3987635513 conn(0x563628a4b4d0 msgr2=0x563628a53d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.870 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 shutdown_connections 2024-05-10T04:40:52.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 1 -- 172.21.15.150:0/3987635513 wait complete. 2024-05-10T04:40:52.871 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.865+0000 7fc5a2b88b80 20 client.16101 trim_cache size 0 max 16384 2024-05-10T04:40:52.871 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:40:52.871 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:40:52.871 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "f97b146f-d943-459a-a190-454116bf5fe5", 2024-05-10T04:40:52.871 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:40:52.871 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc5740a11c0 msgr2=0x7fc5740a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc5740a11c0 0x7fc5740a36d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00b7b0 tx=0x7fc57c01a040 comp rx=0 tx=0).stop 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c102580 msgr2=0x7fc58c1a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c102580 0x7fc58c1a15c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ef30 tx=0x7fc58400c3a0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 shutdown_connections 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc58c102580 0x7fc58c1a15c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc5740a11c0 0x7fc5740a36d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc58c100ac0 0x7fc58c068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 --2- 172.21.15.150:0/2525717766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc58c101be0 0x7fc58c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 >> 172.21.15.150:0/2525717766 conn(0x7fc58c0fc0d0 msgr2=0x7fc58c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:52.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 shutdown_connections 2024-05-10T04:40:52.873 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:52.866+0000 7fc592844700 1 -- 172.21.15.150:0/2525717766 wait complete. 2024-05-10T04:40:52.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:40:52.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:52 smithi181 ceph-mon[47314]: pgmap v310: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-10T04:40:53.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 -- 172.21.15.150:0/3372984876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 msgr2=0x7f653c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 --2- 172.21.15.150:0/3372984876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c109d90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f653400b0a0 tx=0x7f653400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 -- 172.21.15.150:0/3372984876 shutdown_connections 2024-05-10T04:40:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 --2- 172.21.15.150:0/3372984876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c109d90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 --2- 172.21.15.150:0/3372984876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f653c069090 0x7f653c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 --2- 172.21.15.150:0/3372984876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c104d10 0x7f653c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 -- 172.21.15.150:0/3372984876 >> 172.21.15.150:0/3372984876 conn(0x7f653c0fc0d0 msgr2=0x7f653c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 -- 172.21.15.150:0/3372984876 shutdown_connections 2024-05-10T04:40:53.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.066+0000 7f65446fe700 1 -- 172.21.15.150:0/3372984876 wait complete. 2024-05-10T04:40:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 Processor -- start 2024-05-10T04:40:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 -- start start 2024-05-10T04:40:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c069090 0x7f653c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f653c104d10 0x7f653c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c19fac0 con 0x7f653c107880 2024-05-10T04:40:53.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c19fc30 con 0x7f653c069090 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f65446fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c19fdf0 con 0x7f653c104d10 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c19f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37978/0 (socket says 172.21.15.150:37978) 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 -- 172.21.15.150:0/1406331999 learned_addr learned my addr 172.21.15.150:0/1406331999 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6541c99700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f653c104d10 0x7f653c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f654249a700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c069090 0x7f653c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 -- 172.21.15.150:0/1406331999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f653c104d10 msgr2=0x7f653c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f653c104d10 0x7f653c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 -- 172.21.15.150:0/1406331999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c069090 msgr2=0x7f653c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c069090 0x7f653c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.067+0000 7f6542c9b700 1 -- 172.21.15.150:0/1406331999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534009d00 con 0x7f653c107880 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.068+0000 7f6542c9b700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c19f4b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f653400b070 tx=0x7f65340095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.068+0000 7f65337fe700 1 -- 172.21.15.150:0/1406331999 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653400e040 con 0x7f653c107880 2024-05-10T04:40:53.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.068+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c1a0070 con 0x7f653c107880 2024-05-10T04:40:53.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.068+0000 7f65337fe700 1 -- 172.21.15.150:0/1406331999 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6534003c00 con 0x7f653c107880 2024-05-10T04:40:53.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.068+0000 7f65337fe700 1 -- 172.21.15.150:0/1406331999 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653401b6c0 con 0x7f653c107880 2024-05-10T04:40:53.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.068+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c1a04f0 con 0x7f653c107880 2024-05-10T04:40:53.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.070+0000 7f65337fe700 1 -- 172.21.15.150:0/1406331999 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f653401d470 con 0x7f653c107880 2024-05-10T04:40:53.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.071+0000 7f65337fe700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f65280a11c0 0x7f65280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.071+0000 7f654249a700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f65280a11c0 0x7f65280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.071+0000 7f65337fe700 1 -- 172.21.15.150:0/1406331999 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f65340cdc90 con 0x7f653c107880 2024-05-10T04:40:53.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.071+0000 7f654249a700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f65280a11c0 0x7f65280a36d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f653c0694a0 tx=0x7f6538009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 Processor -- start 2024-05-10T04:40:53.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 -- start start 2024-05-10T04:40:53.080 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d636a0 0x5646e8d63af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d64980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d64ec0 0x5646e8d6b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8c8a150 con 0x5646e8d64570 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8c87b10 con 0x5646e8d64ec0 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6554a42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8c83400 con 0x5646e8d636a0 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6541c99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d64980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.073+0000 7f6541c99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d64980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37988/0 (socket says 172.21.15.150:37988) 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6541c99700 1 -- 172.21.15.150:0/3722987646 learned_addr learned my addr 172.21.15.150:0/3722987646 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f654249a700 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d636a0 0x5646e8d63af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6542c9b700 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d64ec0 0x5646e8d6b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6541c99700 1 -- 172.21.15.150:0/3722987646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d636a0 msgr2=0x5646e8d63af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6541c99700 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d636a0 0x5646e8d63af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6541c99700 1 -- 172.21.15.150:0/3722987646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d64ec0 msgr2=0x5646e8d6b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6541c99700 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d64ec0 0x5646e8d6b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6541c99700 1 -- 172.21.15.150:0/3722987646 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e8b043b0 con 0x5646e8d64570 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f654249a700 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d636a0 0x5646e8d63af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6541c99700 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d64980 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f652c00bd30 tx=0x7f652c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6b174477df66a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6530ff9700 1 -- 172.21.15.150:0/3722987646 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c010e10 con 0x5646e8d64570 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6530ff9700 1 -- 172.21.15.150:0/3722987646 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f652c015070 con 0x5646e8d64570 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.074+0000 7f6530ff9700 1 -- 172.21.15.150:0/3722987646 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c011680 con 0x5646e8d64570 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 -- 172.21.15.150:0/3722987646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 msgr2=0x5646e8d64980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d64980 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f652c00bd30 tx=0x7f652c0106b0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 -- 172.21.15.150:0/3722987646 shutdown_connections 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d64ec0 0x5646e8d6b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d636a0 0x5646e8d63af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3722987646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d64980 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 -- 172.21.15.150:0/3722987646 >> 172.21.15.150:0/3722987646 conn(0x5646e8cc5a60 msgr2=0x5646e8d5dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 -- 172.21.15.150:0/3722987646 shutdown_connections 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.075+0000 7f6554a42b80 1 -- 172.21.15.150:0/3722987646 wait complete. 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 -- start start 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 Processor -- start 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d6d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d6dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d74bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8c1f560 con 0x5646e8d64ec0 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8c1f200 con 0x5646e8d636a0 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8b066a0 con 0x5646e8d64570 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6554a42b80 10 client.0 _ll_register_callbacks cb 0x5646e8c28b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6542c9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d74bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6542c9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d74bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37992/0 (socket says 172.21.15.150:37992) 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6542c9b700 1 -- 172.21.15.150:0/2685055445 learned_addr learned my addr 172.21.15.150:0/2685055445 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:53.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f654249a700 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d6d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6541c99700 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d6dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.076+0000 7f6542c9b700 1 -- 172.21.15.150:0/2685055445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 msgr2=0x5646e8d6dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6542c9b700 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d6dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6542c9b700 1 -- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 msgr2=0x5646e8d6d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6542c9b700 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d6d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6542c9b700 1 -- 172.21.15.150:0/2685055445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6538011a10 con 0x5646e8d64ec0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6542c9b700 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d74bd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f65340cb3f0 tx=0x7f653409a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dff1d7ff7d8d07c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6525ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6525ffb700 1 -- 172.21.15.150:0/2685055445 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653409a4c0 con 0x5646e8d64ec0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646e8bdc270 con 0x5646e8d64ec0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6525ffb700 1 -- 172.21.15.150:0/2685055445 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f653401aea0 con 0x5646e8d64ec0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6525ffb700 1 -- 172.21.15.150:0/2685055445 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f653409b4d0 con 0x5646e8d64ec0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6554a42b80 10 client.16113 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f652c00b960 con 0x5646e8d64ec0 2024-05-10T04:40:53.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6525ffb700 1 -- 172.21.15.150:0/2685055445 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5646e8bdc270 con 0x5646e8d64ec0 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6525ffb700 1 -- 172.21.15.150:0/2685055445 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f65340a2dc0 con 0x5646e8d64ec0 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.077+0000 7f6554a42b80 10 client.16113 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f6554a42b80 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f6554a42b80 20 client.16113 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8d79190 0x5646e8d7b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f6554a42b80 4 client.16113 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646e8d32780 con 0x5646e8d79190 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f654249a700 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8d79190 0x5646e8d7b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f6525ffb700 10 client.16113 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.078+0000 7f654249a700 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8d79190 0x5646e8d7b5b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c448130f9e99cace server_cookie=702d5c802373f84f in_seq=0 out_seq=0 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6525ffb700 1 -- 172.21.15.150:0/2685055445 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646e8d32780 con 0x5646e8d79190 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6525ffb700 10 client.16113 handle_command_reply: tid=0 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 client.16113 shutdown 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 msgr2=0x5646e8d74bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d74bd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f65340cb3f0 tx=0x7f653409a730 comp rx=0 tx=0).stop 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 shutdown_connections 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8d79190 0x5646e8d7b5b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d74bd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d6d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2685055445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d6dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6525ffb700 0 client.16113 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 >> 172.21.15.150:0/2685055445 conn(0x5646e8cc5a60 msgr2=0x5646e8d5dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 shutdown_connections 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 1 -- 172.21.15.150:0/2685055445 wait complete. 2024-05-10T04:40:53.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.079+0000 7f6554a42b80 20 client.16113 trim_cache size 0 max 16384 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 Processor -- start 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 -- start start 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d74b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d688f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8d5ed20 con 0x5646e8d64ec0 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8d78f90 con 0x5646e8d636a0 2024-05-10T04:40:53.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6554a42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8d6ba80 con 0x5646e8d64570 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.096+0000 7f6542c9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d688f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38004/0 (socket says 172.21.15.150:38004) 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 -- 172.21.15.150:0/2432778928 learned_addr learned my addr 172.21.15.150:0/2432778928 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f654249a700 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6541c99700 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d74b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 -- 172.21.15.150:0/2432778928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 msgr2=0x5646e8d74b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d74b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 -- 172.21.15.150:0/2432778928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 msgr2=0x5646e8d683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 -- 172.21.15.150:0/2432778928 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e8ab0b60 con 0x5646e8d64ec0 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f6542c9b700 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d688f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f65340972d0 tx=0x7f653400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c115bcb36fdb7b49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f65317fa700 1 -- 172.21.15.150:0/2432778928 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340a2f30 con 0x5646e8d64ec0 2024-05-10T04:40:53.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f65317fa700 1 -- 172.21.15.150:0/2432778928 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f653401d9b0 con 0x5646e8d64ec0 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.097+0000 7f65317fa700 1 -- 172.21.15.150:0/2432778928 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653409a520 con 0x5646e8d64ec0 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 -- 172.21.15.150:0/2432778928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 msgr2=0x5646e8d688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d688f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f65340972d0 tx=0x7f653400bef0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 -- 172.21.15.150:0/2432778928 shutdown_connections 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64ec0 0x5646e8d688f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 --2- 172.21.15.150:0/2432778928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64570 0x5646e8d74b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 -- 172.21.15.150:0/2432778928 >> 172.21.15.150:0/2432778928 conn(0x5646e8cc5a60 msgr2=0x5646e8d5dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 -- 172.21.15.150:0/2432778928 shutdown_connections 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 -- 172.21.15.150:0/2432778928 wait complete. 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.098+0000 7f6554a42b80 1 -- start start 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 1 Processor -- start 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d72690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d72bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64ec0 0x5646e8d73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8d74040 con 0x5646e8d64570 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8d741b0 con 0x5646e8d636a0 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e8d74370 con 0x5646e8d64ec0 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6554a42b80 10 client.0 _ll_register_callbacks cb 0x5646e8c28b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d72bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d72bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38010/0 (socket says 172.21.15.150:38010) 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 -- 172.21.15.150:0/3025952633 learned_addr learned my addr 172.21.15.150:0/3025952633 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f654249a700 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d72690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6542c9b700 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64ec0 0x5646e8d73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 -- 172.21.15.150:0/3025952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64ec0 msgr2=0x5646e8d73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64ec0 0x5646e8d73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 -- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 msgr2=0x5646e8d72690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d72690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.099+0000 7f6541c99700 1 -- 172.21.15.150:0/3025952633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c00b960 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6541c99700 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d72bd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f652c00bd30 tx=0x7f652c010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584dd6ed5b7d0a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6525ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6525ffb700 1 -- 172.21.15.150:0/3025952633 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c022040 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f652c015070 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6525ffb700 1 -- 172.21.15.150:0/3025952633 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f652c017070 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6525ffb700 1 -- 172.21.15.150:0/3025952633 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f652c0056c0 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6554a42b80 10 client.16119 fetch_fsmap learned FSMap version 29 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5646e8c49560 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6525ffb700 1 -- 172.21.15.150:0/3025952633 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f652c015070 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6525ffb700 1 -- 172.21.15.150:0/3025952633 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f652c025070 con 0x5646e8d64570 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6554a42b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:40:53.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.100+0000 7f6554a42b80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6554a42b80 20 client.16119 populate_metadata read hostname 'smithi150' 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8dd01e0 0x5646e8dd2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6554a42b80 4 client.16119 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5646e8c7bec0 con 0x5646e8dd01e0 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f654249a700 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8dd01e0 0x5646e8dd2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6525ffb700 10 client.16119 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f654249a700 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8dd01e0 0x5646e8dd2600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e829710a5292f4 server_cookie=f6a7964f0e5faefa in_seq=0 out_seq=0 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6525ffb700 1 -- 172.21.15.150:0/3025952633 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5646e8c7bec0 con 0x5646e8dd01e0 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6525ffb700 10 client.16119 handle_command_reply: tid=0 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.101+0000 7f6554a42b80 1 client.16119 shutdown 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 msgr2=0x5646e8d72bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d72bd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f652c00bd30 tx=0x7f652c010e40 comp rx=0 tx=0).stop 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 shutdown_connections 2024-05-10T04:40:53.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5646e8dd01e0 0x5646e8dd2600 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5646e8d64ec0 0x5646e8d73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646e8d636a0 0x5646e8d72690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 --2- 172.21.15.150:0/3025952633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5646e8d64570 0x5646e8d72bd0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 >> 172.21.15.150:0/3025952633 conn(0x5646e8cc5a60 msgr2=0x5646e8d6bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6525ffb700 0 client.16119 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 shutdown_connections 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 1 -- 172.21.15.150:0/3025952633 wait complete. 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f6554a42b80 20 client.16119 trim_cache size 0 max 16384 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:40:53.110 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout: "f97b146f-d943-459a-a190-454116bf5fe5": { 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "f97b146f-d943-459a-a190-454116bf5fe5", 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:40:53.111 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f65280a11c0 msgr2=0x7f65280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f65280a11c0 0x7f65280a36d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f653c0694a0 tx=0x7f6538009260 comp rx=0 tx=0).stop 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 msgr2=0x7f653c19f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c19f4b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f653400b070 tx=0x7f65340095a0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 shutdown_connections 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f653c107880 0x7f653c19f4b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c069090 0x7f653c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f65280a11c0 0x7f65280a36d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 --2- 172.21.15.150:0/1406331999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f653c104d10 0x7f653c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 >> 172.21.15.150:0/1406331999 conn(0x7f653c0fc0d0 msgr2=0x7f653c103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:40:53.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.102+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 shutdown_connections 2024-05-10T04:40:53.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:40:53.103+0000 7f65446fe700 1 -- 172.21.15.150:0/1406331999 wait complete. 2024-05-10T04:40:53.116 INFO:tasks.cephfs.filesystem:scrub status for tag:f97b146f-d943-459a-a190-454116bf5fe5 - {'path': '/', 'tag': 'f97b146f-d943-459a-a190-454116bf5fe5', 'options': 'recursive,force'} 2024-05-10T04:40:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:54 smithi161 ceph-mon[46912]: scrub queued for path: / 2024-05-10T04:40:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:54 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:54 smithi161 ceph-mon[46912]: scrub summary: active paths [/] 2024-05-10T04:40:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:54 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:54 smithi161 ceph-mon[46912]: pgmap v311: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:54.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:54 smithi181 ceph-mon[47314]: scrub queued for path: / 2024-05-10T04:40:54.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:54 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:54.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:54 smithi181 ceph-mon[47314]: scrub summary: active paths [/] 2024-05-10T04:40:54.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:54 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:54.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:54 smithi181 ceph-mon[47314]: pgmap v311: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:54.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:54 smithi150 ceph-mon[42404]: scrub queued for path: / 2024-05-10T04:40:54.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:54 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:54.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:54 smithi150 ceph-mon[42404]: scrub summary: active paths [/] 2024-05-10T04:40:54.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:54 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:40:54.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:54 smithi150 ceph-mon[42404]: pgmap v311: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-10T04:40:55.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:55 smithi161 ceph-mon[46912]: pgmap v312: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T04:40:55.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:55 smithi181 ceph-mon[47314]: pgmap v312: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T04:40:55.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:55 smithi150 ceph-mon[42404]: pgmap v312: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T04:40:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:58 smithi150 ceph-mon[42404]: pgmap v313: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-10T04:40:58.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:40:58 smithi150 ceph-mon[42404]: scrub summary: idle 2024-05-10T04:40:58.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:58 smithi161 ceph-mon[46912]: pgmap v313: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-10T04:40:58.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:40:58 smithi161 ceph-mon[46912]: scrub summary: idle 2024-05-10T04:40:58.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:58 smithi181 ceph-mon[47314]: pgmap v313: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-10T04:40:58.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:40:58 smithi181 ceph-mon[47314]: scrub summary: idle 2024-05-10T04:41:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:00 smithi161 ceph-mon[46912]: pgmap v314: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:41:00.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:00 smithi181 ceph-mon[47314]: pgmap v314: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:41:00.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:00 smithi150 ceph-mon[42404]: pgmap v314: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-10T04:41:02.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:02 smithi161 ceph-mon[46912]: pgmap v315: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T04:41:02.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:02 smithi181 ceph-mon[47314]: pgmap v315: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T04:41:02.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:02 smithi150 ceph-mon[42404]: pgmap v315: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T04:41:04.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:04 smithi161 ceph-mon[46912]: pgmap v316: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-10T04:41:04.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:04 smithi181 ceph-mon[47314]: pgmap v316: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-10T04:41:04.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:04 smithi150 ceph-mon[42404]: pgmap v316: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-05-10T04:41:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 -- 172.21.15.150:0/1987281858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 msgr2=0x7f763c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 --2- 172.21.15.150:0/1987281858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c100ed0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f762c009990 tx=0x7f762c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 -- 172.21.15.150:0/1987281858 shutdown_connections 2024-05-10T04:41:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 --2- 172.21.15.150:0/1987281858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102580 0x7f763c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 --2- 172.21.15.150:0/1987281858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c100ed0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 --2- 172.21.15.150:0/1987281858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f763c101be0 0x7f763c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 -- 172.21.15.150:0/1987281858 >> 172.21.15.150:0/1987281858 conn(0x7f763c0fc0f0 msgr2=0x7f763c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 -- 172.21.15.150:0/1987281858 shutdown_connections 2024-05-10T04:41:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.806+0000 7f7640f72700 1 -- 172.21.15.150:0/1987281858 wait complete. 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.807+0000 7f7640f72700 1 Processor -- start 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.807+0000 7f7640f72700 1 -- start start 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f7640f72700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f7640f72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f763c101be0 0x7f763c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f7640f72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102580 0x7f763c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f7640f72700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c19fa80 con 0x7f763c100ac0 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f7640f72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c19fbc0 con 0x7f763c102580 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f7640f72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c19fd50 con 0x7f763c101be0 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49192/0 (socket says 172.21.15.150:49192) 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 -- 172.21.15.150:0/2828659064 learned_addr learned my addr 172.21.15.150:0/2828659064 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f7639d9b700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f763c101be0 0x7f763c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763ad9d700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102580 0x7f763c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 -- 172.21.15.150:0/2828659064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f763c101be0 msgr2=0x7f763c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f763c101be0 0x7f763c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 -- 172.21.15.150:0/2828659064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102580 msgr2=0x7f763c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102580 0x7f763c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 -- 172.21.15.150:0/2828659064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c0095f0 con 0x7f763c100ac0 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763ad9d700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102580 0x7f763c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.808+0000 7f763a59c700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c197fe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f762c000c00 tx=0x7f762c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.809+0000 7f762b7fe700 1 -- 172.21.15.150:0/2828659064 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c01d070 con 0x7f763c100ac0 2024-05-10T04:41:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.809+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c19ffd0 con 0x7f763c100ac0 2024-05-10T04:41:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.809+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c1a0390 con 0x7f763c100ac0 2024-05-10T04:41:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.809+0000 7f762b7fe700 1 -- 172.21.15.150:0/2828659064 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f762c021410 con 0x7f763c100ac0 2024-05-10T04:41:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.809+0000 7f762b7fe700 1 -- 172.21.15.150:0/2828659064 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c017650 con 0x7f763c100ac0 2024-05-10T04:41:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.812+0000 7f76512b6b80 1 Processor -- start 2024-05-10T04:41:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.813+0000 7f762b7fe700 1 -- 172.21.15.150:0/2828659064 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f762c0177b0 con 0x7f763c100ac0 2024-05-10T04:41:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.813+0000 7f76512b6b80 1 -- start start 2024-05-10T04:41:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97056990 0x560a97056dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a97057860 0x560a97057c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a970581b0 0x560a9705eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f76512b6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a96f750f0 con 0x560a970581b0 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f76512b6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a96f72ab0 con 0x560a97056990 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f76512b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a96f6e3a0 con 0x560a97057860 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97056990 0x560a97056dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97056990 0x560a97056dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37590/0 (socket says 172.21.15.150:37590) 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 -- 172.21.15.150:0/1678091942 learned_addr learned my addr 172.21.15.150:0/1678091942 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763a59c700 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a97057860 0x560a97057c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f762b7fe700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f76240a11b0 0x7f76240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 -- 172.21.15.150:0/1678091942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a97057860 msgr2=0x560a97057c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a97057860 0x560a97057c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 -- 172.21.15.150:0/1678091942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a970581b0 msgr2=0x560a9705eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a970581b0 0x560a9705eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f763ad9d700 1 -- 172.21.15.150:0/1678091942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a96def350 con 0x560a97056990 2024-05-10T04:41:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.814+0000 7f762b7fe700 1 -- 172.21.15.150:0/2828659064 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f762c0cdf90 con 0x7f763c100ac0 2024-05-10T04:41:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.815+0000 7f763ad9d700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f76240a11b0 0x7f76240a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.815+0000 7f763ad9d700 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97056990 0x560a97056dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7634007530 tx=0x7f7634007840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5df01dbf06d8976 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.815+0000 7f7628ff9700 1 -- 172.21.15.150:0/1678091942 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763401b070 con 0x560a97056990 2024-05-10T04:41:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.815+0000 7f7628ff9700 1 -- 172.21.15.150:0/1678091942 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f763401c030 con 0x560a97056990 2024-05-10T04:41:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.815+0000 7f7628ff9700 1 -- 172.21.15.150:0/1678091942 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400e070 con 0x560a97056990 2024-05-10T04:41:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 -- 172.21.15.150:0/1678091942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97056990 msgr2=0x560a97056dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97056990 0x560a97056dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7634007530 tx=0x7f7634007840 comp rx=0 tx=0).stop 2024-05-10T04:41:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f763ad9d700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f76240a11b0 0x7f76240a36c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7634020000 tx=0x7f763401d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 -- 172.21.15.150:0/1678091942 shutdown_connections 2024-05-10T04:41:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a970581b0 0x560a9705eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97056990 0x560a97056dc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 --2- 172.21.15.150:0/1678091942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a97057860 0x560a97057c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 -- 172.21.15.150:0/1678091942 >> 172.21.15.150:0/1678091942 conn(0x560a96fb0a00 msgr2=0x560a97048a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 -- 172.21.15.150:0/1678091942 shutdown_connections 2024-05-10T04:41:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 -- 172.21.15.150:0/1678091942 wait complete. 2024-05-10T04:41:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.817+0000 7f76512b6b80 1 -- start start 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 1 Processor -- start 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a970609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97060f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a96f0a500 con 0x560a97056990 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a96f0a1a0 con 0x560a97057860 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a96df1640 con 0x560a970581b0 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f76512b6b80 10 client.0 _ll_register_callbacks cb 0x560a96f13ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97060f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97060f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37600/0 (socket says 172.21.15.150:37600) 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f7639d9b700 1 -- 172.21.15.150:0/4143192818 learned_addr learned my addr 172.21.15.150:0/4143192818 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.818+0000 7f763ad9d700 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7639d9b700 1 -- 172.21.15.150:0/4143192818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 msgr2=0x560a97067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7639d9b700 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7639d9b700 1 -- 172.21.15.150:0/4143192818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 msgr2=0x560a970609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7639d9b700 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a970609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7639d9b700 1 -- 172.21.15.150:0/4143192818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7634009140 con 0x560a97057860 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7639d9b700 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97060f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f763000be70 tx=0x7f76300106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3287bf3212485ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7621ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7621ffb700 1 -- 172.21.15.150:0/4143192818 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763001c680 con 0x560a97057860 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7621ffb700 1 -- 172.21.15.150:0/4143192818 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7630017070 con 0x560a97057860 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a96ec7210 con 0x560a97057860 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f7621ffb700 1 -- 172.21.15.150:0/4143192818 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7630010e50 con 0x560a97057860 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f76512b6b80 10 client.25769 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.819+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560a96d9bb00 con 0x560a97057860 2024-05-10T04:41:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f7621ffb700 1 -- 172.21.15.150:0/4143192818 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a96ec7210 con 0x560a97057860 2024-05-10T04:41:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f7621ffb700 1 -- 172.21.15.150:0/4143192818 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7630011990 con 0x560a97057860 2024-05-10T04:41:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f76512b6b80 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f76512b6b80 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f76512b6b80 20 client.25769 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f76512b6b80 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a97052280 0x560a970546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f76512b6b80 4 client.25769 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a9701d720 con 0x560a97052280 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.820+0000 7f763a59c700 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a97052280 0x560a970546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.821+0000 7f7621ffb700 10 client.25769 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.821+0000 7f763a59c700 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a97052280 0x560a970546d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490d7e1f05753aa server_cookie=c926140621443a8d in_seq=0 out_seq=0 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.821+0000 7f7621ffb700 1 -- 172.21.15.150:0/4143192818 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a9701d720 con 0x560a97052280 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.821+0000 7f7621ffb700 10 client.25769 handle_command_reply: tid=0 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.821+0000 7f76512b6b80 1 client.25769 shutdown 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 msgr2=0x560a97060f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97060f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f763000be70 tx=0x7f76300106b0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 shutdown_connections 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a97052280 0x560a970546d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f7621ffb700 0 client.25769 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a970609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 --2- 172.21.15.150:0/4143192818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97060f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 >> 172.21.15.150:0/4143192818 conn(0x560a96fb0a00 msgr2=0x560a97048a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 shutdown_connections 2024-05-10T04:41:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 1 -- 172.21.15.150:0/4143192818 wait complete. 2024-05-10T04:41:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.822+0000 7f76512b6b80 20 client.25769 trim_cache size 0 max 16384 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 Processor -- start 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 -- start start 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97060f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a970546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9707dca0 con 0x560a97056990 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9705bff0 con 0x560a97057860 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.837+0000 7f76512b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a970409b0 con 0x560a970581b0 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f763ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a970546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a970546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37614/0 (socket says 172.21.15.150:37614) 2024-05-10T04:41:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 -- 172.21.15.150:0/297676397 learned_addr learned my addr 172.21.15.150:0/297676397 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f763a59c700 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97060f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 -- 172.21.15.150:0/297676397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 msgr2=0x560a97067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 -- 172.21.15.150:0/297676397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 msgr2=0x560a97060f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97060f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 -- 172.21.15.150:0/297676397 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a96f34500 con 0x560a97057860 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.838+0000 7f7639d9b700 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a970546a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7630005cb0 tx=0x7f7630005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63bd585338efaf8b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.839+0000 7f76297fa700 1 -- 172.21.15.150:0/297676397 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76300050b0 con 0x560a97057860 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.839+0000 7f76297fa700 1 -- 172.21.15.150:0/297676397 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7630015070 con 0x560a97057860 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.839+0000 7f76297fa700 1 -- 172.21.15.150:0/297676397 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630003680 con 0x560a97057860 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.839+0000 7f76512b6b80 1 -- 172.21.15.150:0/297676397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 msgr2=0x560a970546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.839+0000 7f76512b6b80 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a970546a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7630005cb0 tx=0x7f7630005c40 comp rx=0 tx=0).stop 2024-05-10T04:41:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- 172.21.15.150:0/297676397 shutdown_connections 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97067d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97060f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 --2- 172.21.15.150:0/297676397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a970546a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- 172.21.15.150:0/297676397 >> 172.21.15.150:0/297676397 conn(0x560a96fb0a00 msgr2=0x560a9705aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- 172.21.15.150:0/297676397 shutdown_connections 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- 172.21.15.150:0/297676397 wait complete. 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- start start 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 Processor -- start 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97063a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97063fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97066eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a970673f0 con 0x560a97056990 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97067560 con 0x560a97057860 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a97067720 con 0x560a970581b0 2024-05-10T04:41:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.840+0000 7f76512b6b80 10 client.0 _ll_register_callbacks cb 0x560a96f13ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97063fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f763a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97063a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97063fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:37630/0 (socket says 172.21.15.150:37630) 2024-05-10T04:41:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 -- 172.21.15.150:0/759846249 learned_addr learned my addr 172.21.15.150:0/759846249 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f763ad9d700 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97066eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 -- 172.21.15.150:0/759846249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 msgr2=0x560a97066eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97066eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 -- 172.21.15.150:0/759846249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 msgr2=0x560a97063a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97063a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 -- 172.21.15.150:0/759846249 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a96d9bb00 con 0x560a97057860 2024-05-10T04:41:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7639d9b700 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97063fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7630000c00 tx=0x7f763000b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6afd947630470219 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7621ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7621ffb700 1 -- 172.21.15.150:0/759846249 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630005700 con 0x560a97057860 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f763401c030 con 0x560a97057860 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.841+0000 7f7621ffb700 1 -- 172.21.15.150:0/759846249 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7630017050 con 0x560a97057860 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f7621ffb700 1 -- 172.21.15.150:0/759846249 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7630003680 con 0x560a97057860 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 10 client.25775 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560a96df3c70 con 0x560a97057860 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f7621ffb700 1 -- 172.21.15.150:0/759846249 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f763401c030 con 0x560a97057860 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f7621ffb700 1 -- 172.21.15.150:0/759846249 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f763001ca20 con 0x560a97057860 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 20 client.25775 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a970cb470 0x560a970cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 4 client.25775 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:41:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.842+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a96f66e60 con 0x560a970cb470 2024-05-10T04:41:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.843+0000 7f763a59c700 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a970cb470 0x560a970cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.844+0000 7f7621ffb700 10 client.25775 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.844+0000 7f763a59c700 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a970cb470 0x560a970cd890 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e877c1847aa87e64 server_cookie=2298946c140ed201 in_seq=0 out_seq=0 2024-05-10T04:41:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f7621ffb700 1 -- 172.21.15.150:0/759846249 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a96f66e60 con 0x560a970cb470 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f7621ffb700 10 client.25775 handle_command_reply: tid=0 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 client.25775 shutdown 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 msgr2=0x560a97063fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97063fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7630000c00 tx=0x7f763000b630 comp rx=0 tx=0).stop 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 shutdown_connections 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560a970cb470 0x560a970cd890 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a970581b0 0x560a97066eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560a97056990 0x560a97063a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 --2- 172.21.15.150:0/759846249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560a97057860 0x560a97063fc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 >> 172.21.15.150:0/759846249 conn(0x560a96fb0a00 msgr2=0x560a9705ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.848+0000 7f7621ffb700 0 client.25775 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.851+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 shutdown_connections 2024-05-10T04:41:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.851+0000 7f76512b6b80 1 -- 172.21.15.150:0/759846249 wait complete. 2024-05-10T04:41:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.851+0000 7f76512b6b80 20 client.25775 trim_cache size 0 max 16384 2024-05-10T04:41:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:41:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:41:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:41:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:41:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f76240a11b0 msgr2=0x7f76240a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f76240a11b0 0x7f76240a36c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7634020000 tx=0x7f763401d000 comp rx=0 tx=0).stop 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 msgr2=0x7f763c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c197fe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f762c000c00 tx=0x7f762c009580 comp rx=0 tx=0).stop 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 shutdown_connections 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f763c102580 0x7f763c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c100ac0 0x7f763c197fe0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f76240a11b0 0x7f76240a36c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 --2- 172.21.15.150:0/2828659064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f763c101be0 0x7f763c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 >> 172.21.15.150:0/2828659064 conn(0x7f763c0fc0f0 msgr2=0x7f763c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 shutdown_connections 2024-05-10T04:41:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:05.852+0000 7f7640f72700 1 -- 172.21.15.150:0/2828659064 wait complete. 2024-05-10T04:41:06.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:06 smithi161 ceph-mon[46912]: pgmap v317: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T04:41:06.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:06 smithi181 ceph-mon[47314]: pgmap v317: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T04:41:06.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:06 smithi150 ceph-mon[42404]: pgmap v317: 129 pgs: 129 active+clean; 672 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 11 op/s 2024-05-10T04:41:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:08 smithi161 ceph-mon[46912]: pgmap v318: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T04:41:08.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:08 smithi181 ceph-mon[47314]: pgmap v318: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T04:41:08.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:08 smithi150 ceph-mon[42404]: pgmap v318: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-10T04:41:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:10 smithi161 ceph-mon[46912]: pgmap v319: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:41:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:41:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:41:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:41:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:10 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:41:10.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:10 smithi181 ceph-mon[47314]: pgmap v319: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:41:10.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:41:10.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:41:10.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:41:10.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:10 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:41:10.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:10 smithi150 ceph-mon[42404]: pgmap v319: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:41:10.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:41:10.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:41:10.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:41:10.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:10 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:41:12.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:12 smithi161 ceph-mon[46912]: pgmap v320: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:41:12.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:12 smithi181 ceph-mon[47314]: pgmap v320: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:41:12.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:12 smithi150 ceph-mon[42404]: pgmap v320: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:41:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:13 smithi161 ceph-mon[46912]: pgmap v321: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:13.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:13 smithi181 ceph-mon[47314]: pgmap v321: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:13.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:13 smithi150 ceph-mon[42404]: pgmap v321: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:16.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:16 smithi161 ceph-mon[46912]: pgmap v322: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:16.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:16 smithi181 ceph-mon[47314]: pgmap v322: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:16.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:16 smithi150 ceph-mon[42404]: pgmap v322: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:18.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:18 smithi161 ceph-mon[46912]: pgmap v323: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:18.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:18 smithi181 ceph-mon[47314]: pgmap v323: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:18.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:18 smithi150 ceph-mon[42404]: pgmap v323: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 -- 172.21.15.150:0/3782751368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94100250 msgr2=0x7fca94100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/3782751368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94100250 0x7fca94100660 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fca88009990 tx=0x7fca88009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 -- 172.21.15.150:0/3782751368 shutdown_connections 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/3782751368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca94101d10 0x7fca941085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/3782751368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94100250 0x7fca94100660 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/3782751368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca94101370 0x7fca94101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 -- 172.21.15.150:0/3782751368 >> 172.21.15.150:0/3782751368 conn(0x7fca940fb840 msgr2=0x7fca940fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 -- 172.21.15.150:0/3782751368 shutdown_connections 2024-05-10T04:41:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.033+0000 7fca9b1d7700 1 -- 172.21.15.150:0/3782751368 wait complete. 2024-05-10T04:41:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 Processor -- start 2024-05-10T04:41:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 -- start start 2024-05-10T04:41:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca94100250 0x7fca9410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca94101370 0x7fca94109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94101d10 0x7fca9410a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9419bd80 con 0x7fca94101d10 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9419bef0 con 0x7fca94100250 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca9b1d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9419c0b0 con 0x7fca94101370 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca99774700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94101d10 0x7fca9410a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca99774700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94101d10 0x7fca9410a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44832/0 (socket says 172.21.15.150:44832) 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.034+0000 7fca99774700 1 -- 172.21.15.150:0/239782768 learned_addr learned my addr 172.21.15.150:0/239782768 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca93fff700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca94101370 0x7fca94109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca98f73700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca94100250 0x7fca9410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca99774700 1 -- 172.21.15.150:0/239782768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca94101370 msgr2=0x7fca94109dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca99774700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca94101370 0x7fca94109dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca99774700 1 -- 172.21.15.150:0/239782768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca94100250 msgr2=0x7fca9410b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca99774700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca94100250 0x7fca9410b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca99774700 1 -- 172.21.15.150:0/239782768 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880095f0 con 0x7fca94101d10 2024-05-10T04:41:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca99774700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94101d10 0x7fca9410a300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00ada0 tx=0x7fca8c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca91ffb700 1 -- 172.21.15.150:0/239782768 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00f070 con 0x7fca94101d10 2024-05-10T04:41:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca91ffb700 1 -- 172.21.15.150:0/239782768 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8c007b30 con 0x7fca94101d10 2024-05-10T04:41:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca91ffb700 1 -- 172.21.15.150:0/239782768 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0053e0 con 0x7fca94101d10 2024-05-10T04:41:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.035+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9419c390 con 0x7fca94101d10 2024-05-10T04:41:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.036+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9419c850 con 0x7fca94101d10 2024-05-10T04:41:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.041+0000 7fcaab51bb80 1 Processor -- start 2024-05-10T04:41:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fca91ffb700 1 -- 172.21.15.150:0/239782768 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fca8c00c980 con 0x7fca94101d10 2024-05-10T04:41:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fcaab51bb80 1 -- start start 2024-05-10T04:41:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e459460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e4602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e37f0f0 con 0x55f84e459050 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e37cab0 con 0x55f84e458180 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.044+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e3783a0 con 0x55f84e4599a0 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca98f73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e459460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:40790/0 (socket says 172.21.15.150:40790) 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 -- 172.21.15.150:0/204465132 learned_addr learned my addr 172.21.15.150:0/204465132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e4602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca91ffb700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca840a11a0 0x7fca840a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 -- 172.21.15.150:0/204465132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 msgr2=0x55f84e4602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e4602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 -- 172.21.15.150:0/204465132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 msgr2=0x55f84e459460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e459460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 -- 172.21.15.150:0/204465132 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84e1f9350 con 0x55f84e458180 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca91ffb700 1 -- 172.21.15.150:0/239782768 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fca8c0cc230 con 0x7fca94101d10 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca99774700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca840a11a0 0x7fca840a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.045+0000 7fca93fff700 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4585b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca80011880 tx=0x7fca800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ae2fe95adf6479 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.046+0000 7fca7effd700 1 -- 172.21.15.150:0/204465132 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80011ea0 con 0x55f84e458180 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.046+0000 7fca7effd700 1 -- 172.21.15.150:0/204465132 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca800044f0 con 0x55f84e458180 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.046+0000 7fca7effd700 1 -- 172.21.15.150:0/204465132 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80005280 con 0x55f84e458180 2024-05-10T04:41:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.046+0000 7fca99774700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca840a11a0 0x7fca840a36b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fca8c095de0 tx=0x7fca8c004cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 -- 172.21.15.150:0/204465132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 msgr2=0x55f84e4585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4585b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca80011880 tx=0x7fca800047c0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 -- 172.21.15.150:0/204465132 shutdown_connections 2024-05-10T04:41:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e4602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4585b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/204465132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e459460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 -- 172.21.15.150:0/204465132 >> 172.21.15.150:0/204465132 conn(0x55f84e3baa00 msgr2=0x55f84e452a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 -- 172.21.15.150:0/204465132 shutdown_connections 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.047+0000 7fcaab51bb80 1 -- 172.21.15.150:0/204465132 wait complete. 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.048+0000 7fcaab51bb80 1 -- start start 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.048+0000 7fcaab51bb80 1 Processor -- start 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.048+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.048+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e459050 0x55f84e462830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.048+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e4599a0 0x55f84e469790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e314500 con 0x55f84e4599a0 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e3141a0 con 0x55f84e458180 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e1fb640 con 0x55f84e459050 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fcaab51bb80 10 client.0 _ll_register_callbacks cb 0x55f84e31dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:40806/0 (socket says 172.21.15.150:40806) 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 -- 172.21.15.150:0/3025542614 learned_addr learned my addr 172.21.15.150:0/3025542614 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e4599a0 0x55f84e469790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca98f73700 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e459050 0x55f84e462830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 -- 172.21.15.150:0/3025542614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e459050 msgr2=0x55f84e462830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e459050 0x55f84e462830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 -- 172.21.15.150:0/3025542614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e4599a0 msgr2=0x55f84e469790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e4599a0 0x55f84e469790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.049+0000 7fca93fff700 1 -- 172.21.15.150:0/3025542614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca80019040 con 0x55f84e458180 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fca93fff700 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4622f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca800098b0 tx=0x7fca80009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64901921ce851794 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fca73fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fca73fff700 1 -- 172.21.15.150:0/3025542614 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800050e0 con 0x55f84e458180 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fca73fff700 1 -- 172.21.15.150:0/3025542614 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fca80005240 con 0x55f84e458180 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f84e2d1210 con 0x55f84e458180 2024-05-10T04:41:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fca73fff700 1 -- 172.21.15.150:0/3025542614 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca80005400 con 0x55f84e458180 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fcaab51bb80 10 client.25787 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f84e1a5b00 con 0x55f84e458180 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.050+0000 7fca73fff700 1 -- 172.21.15.150:0/3025542614 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f84e2d1210 con 0x55f84e458180 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.051+0000 7fca73fff700 1 -- 172.21.15.150:0/3025542614 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca8001d070 con 0x55f84e458180 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.051+0000 7fcaab51bb80 10 client.25787 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.051+0000 7fcaab51bb80 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.051+0000 7fcaab51bb80 20 client.25787 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.051+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e46dd50 0x55f84e4701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.051+0000 7fcaab51bb80 4 client.25787 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.051+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f84e427720 con 0x55f84e46dd50 2024-05-10T04:41:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.052+0000 7fca98f73700 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e46dd50 0x55f84e4701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.052+0000 7fca73fff700 10 client.25787 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.052+0000 7fca98f73700 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e46dd50 0x55f84e4701a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58aeb9f80bad0b40 server_cookie=2b1e209161608e16 in_seq=0 out_seq=0 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fca73fff700 1 -- 172.21.15.150:0/3025542614 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f84e427720 con 0x55f84e46dd50 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fca73fff700 10 client.25787 handle_command_reply: tid=0 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 client.25787 shutdown 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 msgr2=0x55f84e4622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4622f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca800098b0 tx=0x7fca80009750 comp rx=0 tx=0).stop 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 shutdown_connections 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e46dd50 0x55f84e4701a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e4599a0 0x55f84e469790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fca73fff700 0 client.25787 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e4622f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/3025542614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e459050 0x55f84e462830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.053+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 >> 172.21.15.150:0/3025542614 conn(0x55f84e3baa00 msgr2=0x55f84e452a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.054+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 shutdown_connections 2024-05-10T04:41:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.054+0000 7fcaab51bb80 1 -- 172.21.15.150:0/3025542614 wait complete. 2024-05-10T04:41:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.054+0000 7fcaab51bb80 20 client.25787 trim_cache size 0 max 16384 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.082+0000 7fcaab51bb80 1 Processor -- start 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.082+0000 7fcaab51bb80 1 -- start start 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e470170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e461860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e467710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e46a8d0 con 0x55f84e459050 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e44d390 con 0x55f84e458180 2024-05-10T04:41:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e46a1f0 con 0x55f84e4599a0 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e467710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca98f73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e461860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca93fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e467710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:37592/0 (socket says 172.21.15.150:37592) 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca93fff700 1 -- 172.21.15.150:0/1334511775 learned_addr learned my addr 172.21.15.150:0/1334511775 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca98f73700 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e461860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44864/0 (socket says 172.21.15.150:44864) 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca93fff700 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e470170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca98f73700 1 -- 172.21.15.150:0/1334511775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 msgr2=0x55f84e467710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca98f73700 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e467710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca98f73700 1 -- 172.21.15.150:0/1334511775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 msgr2=0x55f84e470170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca98f73700 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e470170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca98f73700 1 -- 172.21.15.150:0/1334511775 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84e33e500 con 0x55f84e459050 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.083+0000 7fca93fff700 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e467710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.084+0000 7fca98f73700 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e461860 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fca88000c00 tx=0x7fca88009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bc6483b9e5559a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.084+0000 7fca7f7fe700 1 -- 172.21.15.150:0/1334511775 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8801d040 con 0x55f84e459050 2024-05-10T04:41:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.084+0000 7fca7f7fe700 1 -- 172.21.15.150:0/1334511775 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca8801e040 con 0x55f84e459050 2024-05-10T04:41:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.084+0000 7fca7f7fe700 1 -- 172.21.15.150:0/1334511775 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88026760 con 0x55f84e459050 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.085+0000 7fcaab51bb80 1 -- 172.21.15.150:0/1334511775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 msgr2=0x55f84e461860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.085+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e461860 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fca88000c00 tx=0x7fca88009290 comp rx=0 tx=0).stop 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.085+0000 7fcaab51bb80 1 -- 172.21.15.150:0/1334511775 shutdown_connections 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.085+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e467710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.085+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e470170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.085+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/1334511775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e461860 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.085+0000 7fcaab51bb80 1 -- 172.21.15.150:0/1334511775 >> 172.21.15.150:0/1334511775 conn(0x55f84e3baa00 msgr2=0x55f84e45cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.086+0000 7fcaab51bb80 1 -- 172.21.15.150:0/1334511775 shutdown_connections 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.086+0000 7fcaab51bb80 1 -- 172.21.15.150:0/1334511775 wait complete. 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.086+0000 7fcaab51bb80 1 -- start start 2024-05-10T04:41:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.086+0000 7fcaab51bb80 1 Processor -- start 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e467060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e4675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fcaab51bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e4c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e4c2a90 con 0x55f84e459050 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e4c2bd0 con 0x55f84e458180 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fcaab51bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e4c2d90 con 0x55f84e4599a0 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fcaab51bb80 10 client.0 _ll_register_callbacks cb 0x55f84e31dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e4675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e4675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44878/0 (socket says 172.21.15.150:44878) 2024-05-10T04:41:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 -- 172.21.15.150:0/2600022972 learned_addr learned my addr 172.21.15.150:0/2600022972 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 -- 172.21.15.150:0/2600022972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 msgr2=0x55f84e4c2550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e4c2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 -- 172.21.15.150:0/2600022972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 msgr2=0x55f84e467060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e467060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.087+0000 7fca98f73700 1 -- 172.21.15.150:0/2600022972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f84e1a5b00 con 0x55f84e459050 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fca98f73700 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e4675a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b970 tx=0x7fca880264e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e1ee42fc3ddda9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fca73fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fca73fff700 1 -- 172.21.15.150:0/2600022972 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8801d070 con 0x55f84e459050 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca800044f0 con 0x55f84e459050 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fca73fff700 1 -- 172.21.15.150:0/2600022972 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fca88005240 con 0x55f84e459050 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fca73fff700 1 -- 172.21.15.150:0/2600022972 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca88005400 con 0x55f84e459050 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fcaab51bb80 10 client.16149 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fca940682b0 con 0x55f84e459050 2024-05-10T04:41:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.088+0000 7fca73fff700 1 -- 172.21.15.150:0/2600022972 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca800044f0 con 0x55f84e459050 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fca73fff700 1 -- 172.21.15.150:0/2600022972 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca8800f620 con 0x55f84e459050 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fcaab51bb80 10 client.16149 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fcaab51bb80 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fcaab51bb80 20 client.16149 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e4c4460 0x55f84e4c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fcaab51bb80 4 client.16149 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f84e370e60 con 0x55f84e4c4460 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.089+0000 7fca93fff700 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e4c4460 0x55f84e4c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.090+0000 7fca73fff700 10 client.16149 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.090+0000 7fca93fff700 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e4c4460 0x55f84e4c6880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd00ffa6a55a672 server_cookie=63ba01e582beab61 in_seq=0 out_seq=0 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fca73fff700 1 -- 172.21.15.150:0/2600022972 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f84e370e60 con 0x55f84e4c4460 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fca73fff700 10 client.16149 handle_command_reply: tid=0 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 client.16149 shutdown 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 msgr2=0x55f84e4675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e4675a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b970 tx=0x7fca880264e0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 shutdown_connections 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55f84e4c4460 0x55f84e4c6880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f84e4599a0 0x55f84e4c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f84e458180 0x55f84e467060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 --2- 172.21.15.150:0/2600022972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f84e459050 0x55f84e4675a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 >> 172.21.15.150:0/2600022972 conn(0x55f84e3baa00 msgr2=0x55f84e460600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fca73fff700 0 client.16149 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.094+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 shutdown_connections 2024-05-10T04:41:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.096+0000 7fcaab51bb80 1 -- 172.21.15.150:0/2600022972 wait complete. 2024-05-10T04:41:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.096+0000 7fcaab51bb80 20 client.16149 trim_cache size 0 max 16384 2024-05-10T04:41:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:41:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:41:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:41:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:41:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.096+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca840a11a0 msgr2=0x7fca840a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.096+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca840a11a0 0x7fca840a36b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fca8c095de0 tx=0x7fca8c004cc0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.096+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94101d10 msgr2=0x7fca9410a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.096+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94101d10 0x7fca9410a300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00ada0 tx=0x7fca8c00c330 comp rx=0 tx=0).stop 2024-05-10T04:41:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 shutdown_connections 2024-05-10T04:41:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca94101d10 0x7fca9410a300 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca94100250 0x7fca9410b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fca840a11a0 0x7fca840a36b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 --2- 172.21.15.150:0/239782768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca94101370 0x7fca94109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 >> 172.21.15.150:0/239782768 conn(0x7fca940fb840 msgr2=0x7fca94104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 shutdown_connections 2024-05-10T04:41:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:19.098+0000 7fca9b1d7700 1 -- 172.21.15.150:0/239782768 wait complete. 2024-05-10T04:41:20.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:20 smithi161 ceph-mon[46912]: pgmap v324: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T04:41:20.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:20 smithi181 ceph-mon[47314]: pgmap v324: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T04:41:20.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:20 smithi150 ceph-mon[42404]: pgmap v324: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T04:41:22.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:22 smithi161 ceph-mon[46912]: pgmap v325: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T04:41:22.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:22 smithi181 ceph-mon[47314]: pgmap v325: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T04:41:22.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:22 smithi150 ceph-mon[42404]: pgmap v325: 129 pgs: 129 active+clean; 673 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-10T04:41:23.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:41:23.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.326+0000 7f7f942e2700 1 -- 172.21.15.150:0/3801999494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1002c0 msgr2=0x7f7f8c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.326+0000 7f7f942e2700 1 --2- 172.21.15.150:0/3801999494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c1006d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7f880099c0 tx=0x7f7f88009cd0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.326+0000 7f7f942e2700 1 -- 172.21.15.150:0/3801999494 shutdown_connections 2024-05-10T04:41:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.326+0000 7f7f942e2700 1 --2- 172.21.15.150:0/3801999494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c101d80 0x7f7f8c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.326+0000 7f7f942e2700 1 --2- 172.21.15.150:0/3801999494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c1006d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.326+0000 7f7f942e2700 1 --2- 172.21.15.150:0/3801999494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1013e0 0x7f7f8c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.326+0000 7f7f942e2700 1 -- 172.21.15.150:0/3801999494 >> 172.21.15.150:0/3801999494 conn(0x7f7f8c0fb8f0 msgr2=0x7f7f8c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.327+0000 7f7f942e2700 1 -- 172.21.15.150:0/3801999494 shutdown_connections 2024-05-10T04:41:23.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.327+0000 7f7f942e2700 1 -- 172.21.15.150:0/3801999494 wait complete. 2024-05-10T04:41:23.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.327+0000 7f7f942e2700 1 Processor -- start 2024-05-10T04:41:23.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.327+0000 7f7f942e2700 1 -- start start 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f942e2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c194230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f942e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1013e0 0x7f7f8c194770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f942e2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c101d80 0x7f7f8c190e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f942e2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c191540 con 0x7f7f8c101d80 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f942e2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c1916b0 con 0x7f7f8c1002c0 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f942e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c191870 con 0x7f7f8c1013e0 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c194230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c194230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:40832/0 (socket says 172.21.15.150:40832) 2024-05-10T04:41:23.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 -- 172.21.15.150:0/2824249094 learned_addr learned my addr 172.21.15.150:0/2824249094 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9187d700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1013e0 0x7f7f8c194770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9287f700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c101d80 0x7f7f8c190e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 -- 172.21.15.150:0/2824249094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1013e0 msgr2=0x7f7f8c194770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1013e0 0x7f7f8c194770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 -- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c101d80 msgr2=0x7f7f8c190e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c101d80 0x7f7f8c190e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9207e700 1 -- 172.21.15.150:0/2824249094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880095f0 con 0x7f7f8c1002c0 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.328+0000 7f7f9187d700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1013e0 0x7f7f8c194770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.329+0000 7f7f9207e700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c194230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009fd0 tx=0x7f7f880036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.333 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.329+0000 7f7f837fe700 1 -- 172.21.15.150:0/2824249094 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8801d070 con 0x7f7f8c1002c0 2024-05-10T04:41:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.329+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c199d50 con 0x7f7f8c1002c0 2024-05-10T04:41:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.329+0000 7f7f837fe700 1 -- 172.21.15.150:0/2824249094 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f88005030 con 0x7f7f8c1002c0 2024-05-10T04:41:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.329+0000 7f7f837fe700 1 -- 172.21.15.150:0/2824249094 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88004990 con 0x7f7f8c1002c0 2024-05-10T04:41:23.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.329+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c19a250 con 0x7f7f8c1002c0 2024-05-10T04:41:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.332+0000 7f7f837fe700 1 -- 172.21.15.150:0/2824249094 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7f88004b90 con 0x7f7f8c1002c0 2024-05-10T04:41:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.333+0000 7f7f837fe700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7f780a1240 0x7f7f780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.335 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.333+0000 7f7f837fe700 1 -- 172.21.15.150:0/2824249094 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7f880cd960 con 0x7f7f8c1002c0 2024-05-10T04:41:23.337 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 Processor -- start 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 -- start start 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57eadaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eae570 0x559e57eae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57dd4150 con 0x559e57ead6a0 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57dd1b10 con 0x559e57eae570 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.334+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57dcd400 con 0x559e57eaeec0 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57eadaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.338 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57eadaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44898/0 (socket says 172.21.15.150:44898) 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 -- 172.21.15.150:0/2852120876 learned_addr learned my addr 172.21.15.150:0/2852120876 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9187d700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7f780a1240 0x7f7f780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9187d700 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eae570 0x559e57eae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9287f700 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 -- 172.21.15.150:0/2852120876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 msgr2=0x559e57eb57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 -- 172.21.15.150:0/2852120876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eae570 msgr2=0x559e57eae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eae570 0x559e57eae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 -- 172.21.15.150:0/2852120876 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e57c4e3b0 con 0x559e57ead6a0 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9187d700 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eae570 0x559e57eae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f9207e700 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57eadaf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7f880a2960 tx=0x7f7f8809a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39af080b3de4db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.335+0000 7f7f80d7f700 1 -- 172.21.15.150:0/2852120876 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88098f20 con 0x559e57ead6a0 2024-05-10T04:41:23.339 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.336+0000 7f7f80d7f700 1 -- 172.21.15.150:0/2852120876 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f880051a0 con 0x559e57ead6a0 2024-05-10T04:41:23.340 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.336+0000 7f7f80d7f700 1 -- 172.21.15.150:0/2852120876 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88096970 con 0x559e57ead6a0 2024-05-10T04:41:23.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7fa4626b80 1 -- 172.21.15.150:0/2852120876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 msgr2=0x559e57eadaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57eadaf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7f880a2960 tx=0x7f7f8809a7d0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7f9187d700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7f780a1240 0x7f7f780a3750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c191e20 tx=0x7f7f7c009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7fa4626b80 1 -- 172.21.15.150:0/2852120876 shutdown_connections 2024-05-10T04:41:23.346 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57eadaf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/2852120876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eae570 0x559e57eae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.337+0000 7f7fa4626b80 1 -- 172.21.15.150:0/2852120876 >> 172.21.15.150:0/2852120876 conn(0x559e57e0fa60 msgr2=0x559e57ea7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.338+0000 7f7fa4626b80 1 -- 172.21.15.150:0/2852120876 shutdown_connections 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.338+0000 7f7fa4626b80 1 -- 172.21.15.150:0/2852120876 wait complete. 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.338+0000 7f7fa4626b80 1 -- start start 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.338+0000 7f7fa4626b80 1 Processor -- start 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57eb7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eaeec0 0x559e57ebe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57d69560 con 0x559e57eaeec0 2024-05-10T04:41:23.347 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57d69200 con 0x559e57ead6a0 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57c506a0 con 0x559e57eae570 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7fa4626b80 10 client.0 _ll_register_callbacks cb 0x559e57d72b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57eb7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9287f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eaeec0 0x559e57ebe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57eb7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:37614/0 (socket says 172.21.15.150:37614) 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 -- 172.21.15.150:0/343432371 learned_addr learned my addr 172.21.15.150:0/343432371 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9207e700 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 -- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 msgr2=0x559e57eb73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 -- 172.21.15.150:0/343432371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eaeec0 msgr2=0x559e57ebe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eaeec0 0x559e57ebe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.339+0000 7f7f9187d700 1 -- 172.21.15.150:0/343432371 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c19a250 con 0x559e57eae570 2024-05-10T04:41:23.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7f9187d700 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57eb7930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c010be0 tx=0x7f7f7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca99f7bd5a4af596 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:41:23.349 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7f75ffb700 1 -- 172.21.15.150:0/343432371 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c018070 con 0x559e57eae570 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7f75ffb700 1 -- 172.21.15.150:0/343432371 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7f7c00fa60 con 0x559e57eae570 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e57d26270 con 0x559e57eae570 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7f75ffb700 1 -- 172.21.15.150:0/343432371 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f7c00fc20 con 0x559e57eae570 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7fa4626b80 10 client.25711 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.340+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559e57bfab60 con 0x559e57eae570 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7f75ffb700 1 -- 172.21.15.150:0/343432371 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e57d26270 con 0x559e57eae570 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7f75ffb700 1 -- 172.21.15.150:0/343432371 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f7c025020 con 0x559e57eae570 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7fa4626b80 10 client.25711 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:23.350 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7fa4626b80 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7fa4626b80 20 client.25711 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57ec3100 0x559e57ec5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7fa4626b80 4 client.25711 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e57e7c780 con 0x559e57ec3100 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.341+0000 7f7f9207e700 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57ec3100 0x559e57ec5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.342+0000 7f7f75ffb700 10 client.25711 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.342+0000 7f7f9207e700 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57ec3100 0x559e57ec5550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7515cbc47f9434a5 server_cookie=ea2619a92e28a82a in_seq=0 out_seq=0 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.342+0000 7f7f75ffb700 1 -- 172.21.15.150:0/343432371 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e57e7c780 con 0x559e57ec3100 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.342+0000 7f7f75ffb700 10 client.25711 handle_command_reply: tid=0 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 client.25711 shutdown 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 msgr2=0x559e57eb7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57eb7930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c010be0 tx=0x7f7f7c004750 comp rx=0 tx=0).stop 2024-05-10T04:41:23.351 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 shutdown_connections 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57ec3100 0x559e57ec5550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eaeec0 0x559e57ebe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/343432371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57eb7930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 >> 172.21.15.150:0/343432371 conn(0x559e57e0fa60 msgr2=0x559e57ea7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7f75ffb700 0 client.25711 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 shutdown_connections 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 1 -- 172.21.15.150:0/343432371 wait complete. 2024-05-10T04:41:23.352 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.343+0000 7f7fa4626b80 20 client.25711 trim_cache size 0 max 16384 2024-05-10T04:41:23.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 Processor -- start 2024-05-10T04:41:23.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 -- start start 2024-05-10T04:41:23.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eae570 0x559e57ebe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57eae3c0 con 0x559e57eae570 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57ea8ad0 con 0x559e57ead6a0 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57eb5620 con 0x559e57eaeec0 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.376+0000 7f7f9187d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eae570 0x559e57ebe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9187d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eae570 0x559e57ebe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44924/0 (socket says 172.21.15.150:44924) 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9187d700 1 -- 172.21.15.150:0/3834559615 learned_addr learned my addr 172.21.15.150:0/3834559615 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9207e700 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9287f700 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9207e700 1 -- 172.21.15.150:0/3834559615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 msgr2=0x559e57eb59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9207e700 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9207e700 1 -- 172.21.15.150:0/3834559615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eae570 msgr2=0x559e57ebe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9207e700 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eae570 0x559e57ebe850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9207e700 1 -- 172.21.15.150:0/3834559615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e57d93560 con 0x559e57ead6a0 2024-05-10T04:41:23.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f9207e700 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb2290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f8809ab10 tx=0x7f7f88009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdbb32be4340d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f81580700 1 -- 172.21.15.150:0/3834559615 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8809a070 con 0x559e57ead6a0 2024-05-10T04:41:23.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f81580700 1 -- 172.21.15.150:0/3834559615 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f8809a1d0 con 0x559e57ead6a0 2024-05-10T04:41:23.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.377+0000 7f7f81580700 1 -- 172.21.15.150:0/3834559615 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880b3360 con 0x559e57ead6a0 2024-05-10T04:41:23.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 -- 172.21.15.150:0/3834559615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 msgr2=0x559e57eb2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb2290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f8809ab10 tx=0x7f7f88009310 comp rx=0 tx=0).stop 2024-05-10T04:41:23.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 -- 172.21.15.150:0/3834559615 shutdown_connections 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eaeec0 0x559e57eb59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57ead6a0 0x559e57eb2290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/3834559615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57eae570 0x559e57ebe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 -- 172.21.15.150:0/3834559615 >> 172.21.15.150:0/3834559615 conn(0x559e57e0fa60 msgr2=0x559e57eb17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 -- 172.21.15.150:0/3834559615 shutdown_connections 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 -- 172.21.15.150:0/3834559615 wait complete. 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 -- start start 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.379+0000 7f7fa4626b80 1 Processor -- start 2024-05-10T04:41:23.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57ebbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57ebc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7fa4626b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eaeec0 0x559e57ebd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57ebd560 con 0x559e57ead6a0 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57ebd6d0 con 0x559e57eaeec0 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7fa4626b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e57ebd890 con 0x559e57eae570 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7fa4626b80 10 client.0 _ll_register_callbacks cb 0x559e57d72b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57ebbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57ebbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44930/0 (socket says 172.21.15.150:44930) 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 -- 172.21.15.150:0/694270711 learned_addr learned my addr 172.21.15.150:0/694270711 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9287f700 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eaeec0 0x559e57ebd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 -- 172.21.15.150:0/694270711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 msgr2=0x559e57ebc0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57ebc0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 -- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eaeec0 msgr2=0x559e57ebd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eaeec0 0x559e57ebd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.380+0000 7f7f9207e700 1 -- 172.21.15.150:0/694270711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e57c4e3b0 con 0x559e57ead6a0 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7f9207e700 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57ebbbb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7f88097080 tx=0x7f7f88009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a91648d947de37e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7f75ffb700 1 -- 172.21.15.150:0/694270711 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8809a340 con 0x559e57ead6a0 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7f75ffb700 1 -- 172.21.15.150:0/694270711 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7f880b3120 con 0x559e57ead6a0 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f880051a0 con 0x559e57ead6a0 2024-05-10T04:41:23.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7f75ffb700 1 -- 172.21.15.150:0/694270711 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f880b32e0 con 0x559e57ead6a0 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7fa4626b80 10 client.16167 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559e57c52cd0 con 0x559e57ead6a0 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7f75ffb700 1 -- 172.21.15.150:0/694270711 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f880051a0 con 0x559e57ead6a0 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.381+0000 7f7f75ffb700 1 -- 172.21.15.150:0/694270711 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7f8809bd10 con 0x559e57ead6a0 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7fa4626b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7fa4626b80 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7fa4626b80 20 client.16167 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57f195d0 0x559e57f1b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7fa4626b80 4 client.16167 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559e57dc5ec0 con 0x559e57f195d0 2024-05-10T04:41:23.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7f9187d700 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57f195d0 0x559e57f1b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7f75ffb700 10 client.16167 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.382+0000 7f7f9187d700 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57f195d0 0x559e57f1b9f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3000233772104914 server_cookie=503714d4df965d73 in_seq=0 out_seq=0 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7f75ffb700 1 -- 172.21.15.150:0/694270711 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559e57dc5ec0 con 0x559e57f195d0 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7f75ffb700 10 client.16167 handle_command_reply: tid=0 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 client.16167 shutdown 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 msgr2=0x559e57ebbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57ebbbb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7f88097080 tx=0x7f7f88009910 comp rx=0 tx=0).stop 2024-05-10T04:41:23.391 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 shutdown_connections 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x559e57f195d0 0x559e57f1b9f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559e57eaeec0 0x559e57ebd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559e57ead6a0 0x559e57ebbbb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 --2- 172.21.15.150:0/694270711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x559e57eae570 0x559e57ebc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 >> 172.21.15.150:0/694270711 conn(0x559e57e0fa60 msgr2=0x559e57eb2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7f75ffb700 0 client.16167 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 shutdown_connections 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 1 -- 172.21.15.150:0/694270711 wait complete. 2024-05-10T04:41:23.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.383+0000 7f7fa4626b80 20 client.16167 trim_cache size 0 max 16384 2024-05-10T04:41:23.393 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:41:23.393 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:41:23.393 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:41:23.393 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:41:23.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7f780a1240 msgr2=0x7f7f780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7f780a1240 0x7f7f780a3750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c191e20 tx=0x7f7f7c009650 comp rx=0 tx=0).stop 2024-05-10T04:41:23.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1002c0 msgr2=0x7f7f8c194230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c194230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009fd0 tx=0x7f7f880036f0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 shutdown_connections 2024-05-10T04:41:23.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f8c101d80 0x7f7f8c190e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f8c1002c0 0x7f7f8c194230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f7f780a1240 0x7f7f780a3750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 --2- 172.21.15.150:0/2824249094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f8c1013e0 0x7f7f8c194770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 >> 172.21.15.150:0/2824249094 conn(0x7f7f8c0fb8f0 msgr2=0x7f7f8c104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 shutdown_connections 2024-05-10T04:41:23.395 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.384+0000 7f7f942e2700 1 -- 172.21.15.150:0/2824249094 wait complete. 2024-05-10T04:41:23.405 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:41:23.405 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 -- 172.21.15.150:0/1343164417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414104d10 msgr2=0x7ff4141071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 --2- 172.21.15.150:0/1343164417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414104d10 0x7ff4141071f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff408009990 tx=0x7ff408009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 -- 172.21.15.150:0/1343164417 shutdown_connections 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 --2- 172.21.15.150:0/1343164417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff414107880 0x7ff414109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 --2- 172.21.15.150:0/1343164417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff414069090 0x7ff414104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 --2- 172.21.15.150:0/1343164417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414104d10 0x7ff4141071f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 -- 172.21.15.150:0/1343164417 >> 172.21.15.150:0/1343164417 conn(0x7ff4140fc0d0 msgr2=0x7ff4140fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 -- 172.21.15.150:0/1343164417 shutdown_connections 2024-05-10T04:41:23.550 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.548+0000 7ff41941d700 1 -- 172.21.15.150:0/1343164417 wait complete. 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.549+0000 7ff41941d700 1 Processor -- start 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.549+0000 7ff41941d700 1 -- start start 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff41941d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414069090 0x7ff414197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff41941d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff414104d10 0x7ff4141984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff41941d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff414107880 0x7ff41419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff41941d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41419fa70 con 0x7ff414069090 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff41941d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41419fbe0 con 0x7ff414104d10 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff41941d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41419fda0 con 0x7ff414107880 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff412ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414069090 0x7ff414197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff412ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414069090 0x7ff414197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44938/0 (socket says 172.21.15.150:44938) 2024-05-10T04:41:23.553 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff412ffd700 1 -- 172.21.15.150:0/1953013491 learned_addr learned my addr 172.21.15.150:0/1953013491 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4127fc700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff414104d10 0x7ff4141984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4137fe700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff414107880 0x7ff41419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4127fc700 1 -- 172.21.15.150:0/1953013491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff414107880 msgr2=0x7ff41419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4127fc700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff414107880 0x7ff41419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4127fc700 1 -- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414069090 msgr2=0x7ff414197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4127fc700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414069090 0x7ff414197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4127fc700 1 -- 172.21.15.150:0/1953013491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080095f0 con 0x7ff414104d10 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff412ffd700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414069090 0x7ff414197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.550+0000 7ff4127fc700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff414104d10 0x7ff4141984d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff408000c00 tx=0x7ff408005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.551+0000 7ff3fbfff700 1 -- 172.21.15.150:0/1953013491 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40801d070 con 0x7ff414104d10 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.551+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4141a0020 con 0x7ff414104d10 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.551+0000 7ff3fbfff700 1 -- 172.21.15.150:0/1953013491 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff408021410 con 0x7ff414104d10 2024-05-10T04:41:23.554 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.551+0000 7ff3fbfff700 1 -- 172.21.15.150:0/1953013491 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408017650 con 0x7ff414104d10 2024-05-10T04:41:23.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.551+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4141a0450 con 0x7ff414104d10 2024-05-10T04:41:23.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.552+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4140fdc80 con 0x7ff414104d10 2024-05-10T04:41:23.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.553+0000 7ff3fbfff700 1 -- 172.21.15.150:0/1953013491 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7ff408021580 con 0x7ff414104d10 2024-05-10T04:41:23.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.553+0000 7ff3fbfff700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3fc0a1120 0x7ff3fc0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.554+0000 7ff3fbfff700 1 -- 172.21.15.150:0/1953013491 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7ff4080ce150 con 0x7ff414104d10 2024-05-10T04:41:23.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.555+0000 7ff412ffd700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3fc0a1120 0x7ff3fc0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.555+0000 7ff412ffd700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3fc0a1120 0x7ff3fc0a3630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff404011550 tx=0x7ff404009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.557+0000 7ff3fbfff700 1 -- 172.21.15.150:0/1953013491 <== mon.2 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4080975d0 con 0x7ff414104d10 2024-05-10T04:41:23.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.811+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4140694d0 con 0x7ff414104d10 2024-05-10T04:41:23.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.812+0000 7ff3fbfff700 1 -- 172.21.15.150:0/1953013491 <== mon.2 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7ff40809abf0 con 0x7ff414104d10 2024-05-10T04:41:23.814 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:41:23.814 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:41:23.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3fc0a1120 msgr2=0x7ff3fc0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3fc0a1120 0x7ff3fc0a3630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff404011550 tx=0x7ff404009260 comp rx=0 tx=0).stop 2024-05-10T04:41:23.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff414104d10 msgr2=0x7ff4141984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff414104d10 0x7ff4141984d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff408000c00 tx=0x7ff408005740 comp rx=0 tx=0).stop 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 shutdown_connections 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7ff3fc0a1120 0x7ff3fc0a3630 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff414107880 0x7ff41419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff414069090 0x7ff414197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 --2- 172.21.15.150:0/1953013491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff414104d10 0x7ff4141984d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 >> 172.21.15.150:0/1953013491 conn(0x7ff4140fc0d0 msgr2=0x7ff414104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 shutdown_connections 2024-05-10T04:41:23.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.815+0000 7ff41941d700 1 -- 172.21.15.150:0/1953013491 wait complete. 2024-05-10T04:41:23.818 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:41:23.830 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 -- 172.21.15.150:0/2330086070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 msgr2=0x7f884c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2330086070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c104750 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8848009990 tx=0x7f8848009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 -- 172.21.15.150:0/2330086070 shutdown_connections 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2330086070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f884c1078b0 0x7f884c109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2330086070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c104750 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2330086070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c104d40 0x7f884c107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 -- 172.21.15.150:0/2330086070 >> 172.21.15.150:0/2330086070 conn(0x7f884c0fc0e0 msgr2=0x7f884c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.983 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 -- 172.21.15.150:0/2330086070 shutdown_connections 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.979+0000 7f8853ceb700 1 -- 172.21.15.150:0/2330086070 wait complete. 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 Processor -- start 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 -- start start 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f884c104d40 0x7f884c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c1078b0 0x7f884c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c19f9e0 con 0x7f884c06a7f0 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c19fb50 con 0x7f884c1078b0 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8853ceb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c19fd10 con 0x7f884c104d40 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8851a87700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8851a87700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44948/0 (socket says 172.21.15.150:44948) 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8851a87700 1 -- 172.21.15.150:0/2476688819 learned_addr learned my addr 172.21.15.150:0/2476688819 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.984 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.980+0000 7f8852288700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c1078b0 0x7f884c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851286700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f884c104d40 0x7f884c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851a87700 1 -- 172.21.15.150:0/2476688819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f884c104d40 msgr2=0x7f884c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851a87700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f884c104d40 0x7f884c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851a87700 1 -- 172.21.15.150:0/2476688819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c1078b0 msgr2=0x7f884c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851a87700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c1078b0 0x7f884c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851a87700 1 -- 172.21.15.150:0/2476688819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88480095f0 con 0x7f884c06a7f0 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851286700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f884c104d40 0x7f884c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8851a87700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c197f00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f88480098e0 tx=0x7f8848009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8842ffd700 1 -- 172.21.15.150:0/2476688819 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884801d070 con 0x7f884c06a7f0 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c19ff90 con 0x7f884c06a7f0 2024-05-10T04:41:23.985 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.981+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c1a0350 con 0x7f884c06a7f0 2024-05-10T04:41:23.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.984+0000 7f8842ffd700 1 -- 172.21.15.150:0/2476688819 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8848021410 con 0x7f884c06a7f0 2024-05-10T04:41:23.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.984+0000 7f8842ffd700 1 -- 172.21.15.150:0/2476688819 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848017650 con 0x7f884c06a7f0 2024-05-10T04:41:23.986 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.985+0000 7f886402fb80 1 Processor -- start 2024-05-10T04:41:23.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.985+0000 7f886402fb80 1 -- start start 2024-05-10T04:41:23.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f886402fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa63af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f886402fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa64980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f886402fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa6b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f886402fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46f98a150 con 0x55a46fa64570 2024-05-10T04:41:23.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f886402fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46f987b10 con 0x55a46fa636a0 2024-05-10T04:41:23.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f886402fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46f983400 con 0x55a46fa64ec0 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa63af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa63af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:40914/0 (socket says 172.21.15.150:40914) 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 -- 172.21.15.150:0/915501488 learned_addr learned my addr 172.21.15.150:0/915501488 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8851286700 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa6b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 -- 172.21.15.150:0/915501488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 msgr2=0x55a46fa6b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa6b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 -- 172.21.15.150:0/915501488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 msgr2=0x55a46fa64980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa64980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 -- 172.21.15.150:0/915501488 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a46f8043b0 con 0x55a46fa636a0 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.986+0000 7f8852288700 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa63af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88440115d0 tx=0x7f8844009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7d350498534259 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.987+0000 7f8837fff700 1 -- 172.21.15.150:0/915501488 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884400e040 con 0x55a46fa636a0 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.987+0000 7f8837fff700 1 -- 172.21.15.150:0/915501488 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f884400f040 con 0x55a46fa636a0 2024-05-10T04:41:23.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.987+0000 7f8837fff700 1 -- 172.21.15.150:0/915501488 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844004770 con 0x55a46fa636a0 2024-05-10T04:41:23.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.987+0000 7f8842ffd700 1 -- 172.21.15.150:0/2476688819 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f884800fb10 con 0x7f884c06a7f0 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f886402fb80 1 -- 172.21.15.150:0/915501488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 msgr2=0x55a46fa63af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f886402fb80 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa63af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88440115d0 tx=0x7f8844009920 comp rx=0 tx=0).stop 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f8842ffd700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88380a11c0 0x7f88380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f886402fb80 1 -- 172.21.15.150:0/915501488 shutdown_connections 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f886402fb80 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa6b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f886402fb80 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa63af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f886402fb80 1 --2- 172.21.15.150:0/915501488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa64980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f886402fb80 1 -- 172.21.15.150:0/915501488 >> 172.21.15.150:0/915501488 conn(0x55a46f9c5a60 msgr2=0x55a46fa5dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f8842ffd700 1 -- 172.21.15.150:0/2476688819 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f88480cde30 con 0x7f884c06a7f0 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.988+0000 7f8851a87700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88380a11c0 0x7f88380a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.989+0000 7f886402fb80 1 -- 172.21.15.150:0/915501488 shutdown_connections 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.989+0000 7f886402fb80 1 -- 172.21.15.150:0/915501488 wait complete. 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.989+0000 7f8851a87700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88380a11c0 0x7f88380a36d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8848097470 tx=0x7f8848005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.989+0000 7f886402fb80 1 -- start start 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.989+0000 7f886402fb80 1 Processor -- start 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f886402fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa7e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f886402fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64570 0x55a46fa7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f886402fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64ec0 0x55a46fa85b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f886402fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46f91f560 con 0x55a46fa64ec0 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f886402fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46f91f200 con 0x55a46fa636a0 2024-05-10T04:41:23.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f886402fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46f8066a0 con 0x55a46fa64570 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f886402fb80 10 client.0 _ll_register_callbacks cb 0x55a46f928b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64ec0 0x55a46fa85b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64ec0 0x55a46fa85b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44960/0 (socket says 172.21.15.150:44960) 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 -- 172.21.15.150:0/1167063701 learned_addr learned my addr 172.21.15.150:0/1167063701 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa7e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8851286700 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64570 0x55a46fa7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 -- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64570 msgr2=0x55a46fa7ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64570 0x55a46fa7ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 -- 172.21.15.150:0/1167063701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 msgr2=0x55a46fa7e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa7e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.990+0000 7f8852288700 1 -- 172.21.15.150:0/1167063701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88440112b0 con 0x55a46fa64ec0 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f8852288700 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64ec0 0x55a46fa85b30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f88440041c0 tx=0x7f8844005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c1892513204e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f88357fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:23.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f88357fa700 1 -- 172.21.15.150:0/1167063701 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884400e040 con 0x55a46fa64ec0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f88357fa700 1 -- 172.21.15.150:0/1167063701 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f8844007500 con 0x55a46fa64ec0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a46f8dc270 con 0x55a46fa64ec0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f88357fa700 1 -- 172.21.15.150:0/1167063701 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88440078d0 con 0x55a46fa64ec0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f886402fb80 10 client.16182 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.991+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a46f7b0b60 con 0x55a46fa64ec0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f88357fa700 1 -- 172.21.15.150:0/1167063701 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a46f8dc270 con 0x55a46fa64ec0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f88357fa700 1 -- 172.21.15.150:0/1167063701 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8844015aa0 con 0x55a46fa64ec0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f886402fb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f886402fb80 10 client.16182 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f886402fb80 20 client.16182 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f886402fb80 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fa8e6e0 0x55a46fa90b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f886402fb80 4 client.16182 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a46fa32780 con 0x55a46fa8e6e0 2024-05-10T04:41:23.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f8851286700 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fa8e6e0 0x55a46fa90b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.992+0000 7f88357fa700 10 client.16182 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.993+0000 7f8851286700 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fa8e6e0 0x55a46fa90b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b35350b8f866c59c server_cookie=e0ab935b3e03e1e5 in_seq=0 out_seq=0 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f88357fa700 1 -- 172.21.15.150:0/1167063701 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a46fa32780 con 0x55a46fa8e6e0 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f88357fa700 10 client.16182 handle_command_reply: tid=0 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 client.16182 shutdown 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64ec0 msgr2=0x55a46fa85b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64ec0 0x55a46fa85b30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f88440041c0 tx=0x7f8844005ae0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 shutdown_connections 2024-05-10T04:41:23.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fa8e6e0 0x55a46fa90b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64ec0 0x55a46fa85b30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f88357fa700 0 client.16182 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa7e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 --2- 172.21.15.150:0/1167063701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64570 0x55a46fa7ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 >> 172.21.15.150:0/1167063701 conn(0x55a46f9c5a60 msgr2=0x55a46fa5dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 shutdown_connections 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.994+0000 7f886402fb80 1 -- 172.21.15.150:0/1167063701 wait complete. 2024-05-10T04:41:23.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:23.995+0000 7f886402fb80 20 client.16182 trim_cache size 0 max 16384 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.014+0000 7f886402fb80 1 Processor -- start 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.014+0000 7f886402fb80 1 -- start start 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.014+0000 7f886402fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa636a0 0x55a46fa8a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.014+0000 7f886402fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa64570 0x55a46fa85af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f886402fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa5a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f886402fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa63f50 con 0x55a46fa636a0 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f886402fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa652d0 con 0x55a46fa64570 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f886402fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa95c90 con 0x55a46fa64ec0 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa636a0 0x55a46fa8a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa636a0 0x55a46fa8a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44972/0 (socket says 172.21.15.150:44972) 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 -- 172.21.15.150:0/3021618060 learned_addr learned my addr 172.21.15.150:0/3021618060 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa5a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8851286700 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa64570 0x55a46fa85af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 -- 172.21.15.150:0/3021618060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 msgr2=0x55a46fa5a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa5a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 -- 172.21.15.150:0/3021618060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa64570 msgr2=0x55a46fa85af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa64570 0x55a46fa85af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 -- 172.21.15.150:0/3021618060 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a46f949560 con 0x55a46fa636a0 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8851286700 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa64570 0x55a46fa85af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.015+0000 7f8852288700 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa636a0 0x55a46fa8a520 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8844002230 tx=0x7f8844009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8399262369088ba server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.016+0000 7f8840ff9700 1 -- 172.21.15.150:0/3021618060 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884400e070 con 0x55a46fa636a0 2024-05-10T04:41:24.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.016+0000 7f8840ff9700 1 -- 172.21.15.150:0/3021618060 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8844007500 con 0x55a46fa636a0 2024-05-10T04:41:24.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.016+0000 7f8840ff9700 1 -- 172.21.15.150:0/3021618060 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844022ea0 con 0x55a46fa636a0 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 -- 172.21.15.150:0/3021618060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa636a0 msgr2=0x55a46fa8a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa636a0 0x55a46fa8a520 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8844002230 tx=0x7f8844009920 comp rx=0 tx=0).stop 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 -- 172.21.15.150:0/3021618060 shutdown_connections 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa5a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa636a0 0x55a46fa8a520 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 --2- 172.21.15.150:0/3021618060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa64570 0x55a46fa85af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 -- 172.21.15.150:0/3021618060 >> 172.21.15.150:0/3021618060 conn(0x55a46f9c5a60 msgr2=0x55a46fa677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 -- 172.21.15.150:0/3021618060 shutdown_connections 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 -- 172.21.15.150:0/3021618060 wait complete. 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.017+0000 7f886402fb80 1 -- start start 2024-05-10T04:41:24.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 1 Processor -- start 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa82df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa83330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa84210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa84750 con 0x55a46fa64570 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa848c0 con 0x55a46fa636a0 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa84a80 con 0x55a46fa64ec0 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f886402fb80 10 client.0 _ll_register_callbacks cb 0x55a46f928b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f8851286700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa83330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f8851286700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa83330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44986/0 (socket says 172.21.15.150:44986) 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f8851286700 1 -- 172.21.15.150:0/950371927 learned_addr learned my addr 172.21.15.150:0/950371927 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.018+0000 7f8852288700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa84210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8852288700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa82df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8851286700 1 -- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 msgr2=0x55a46fa84210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8851286700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa84210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8851286700 1 -- 172.21.15.150:0/950371927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 msgr2=0x55a46fa82df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8851286700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa82df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8851286700 1 -- 172.21.15.150:0/950371927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a46f7b0b60 con 0x55a46fa64570 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8852288700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa82df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f8851286700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa83330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x55a46fa91390 tx=0x7f883c004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c389d9b51043e662 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f88357fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f88357fa700 1 -- 172.21.15.150:0/950371927 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c017040 con 0x55a46fa64570 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f88357fa700 1 -- 172.21.15.150:0/950371927 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f883c01c3a0 con 0x55a46fa64570 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f884400f040 con 0x55a46fa64570 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.019+0000 7f88357fa700 1 -- 172.21.15.150:0/950371927 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f883c003e70 con 0x55a46fa64570 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 10 client.16188 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a46f808cd0 con 0x55a46fa64570 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f88357fa700 1 -- 172.21.15.150:0/950371927 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f884400f040 con 0x55a46fa64570 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f88357fa700 1 -- 172.21.15.150:0/950371927 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f883c011cb0 con 0x55a46fa64570 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 20 client.16188 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fae4ef0 0x55a46fae7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 4 client.16188 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.020+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a46f97bec0 con 0x55a46fae4ef0 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.021+0000 7f8852288700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fae4ef0 0x55a46fae7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f88357fa700 10 client.16188 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f8852288700 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fae4ef0 0x55a46fae7310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a3e417c71840d3f server_cookie=8ca5bc907768cf15 in_seq=0 out_seq=0 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f88357fa700 1 -- 172.21.15.150:0/950371927 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a46f97bec0 con 0x55a46fae4ef0 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f88357fa700 10 client.16188 handle_command_reply: tid=0 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 client.16188 shutdown 2024-05-10T04:41:24.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 msgr2=0x55a46fa83330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa83330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x55a46fa91390 tx=0x7f883c004480 comp rx=0 tx=0).stop 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 shutdown_connections 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a46fae4ef0 0x55a46fae7310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a46fa64ec0 0x55a46fa84210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f88357fa700 0 client.16188 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a46fa636a0 0x55a46fa82df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 --2- 172.21.15.150:0/950371927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a46fa64570 0x55a46fa83330 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 >> 172.21.15.150:0/950371927 conn(0x55a46f9c5a60 msgr2=0x55a46fa677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 shutdown_connections 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 1 -- 172.21.15.150:0/950371927 wait complete. 2024-05-10T04:41:24.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.022+0000 7f886402fb80 20 client.16188 trim_cache size 0 max 16384 2024-05-10T04:41:24.029 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:24.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88380a11c0 msgr2=0x7f88380a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88380a11c0 0x7f88380a36d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8848097470 tx=0x7f8848005fd0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 msgr2=0x7f884c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c197f00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f88480098e0 tx=0x7f8848009580 comp rx=0 tx=0).stop 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 shutdown_connections 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c1078b0 0x7f884c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f884c06a7f0 0x7f884c197f00 secure :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f88480098e0 tx=0x7f8848009580 comp rx=0 tx=0).stop 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88380a11c0 0x7f88380a36d0 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8848097470 tx=0x7f8848005fd0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 --2- 172.21.15.150:0/2476688819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f884c104d40 0x7f884c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.023+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 >> 172.21.15.150:0/2476688819 conn(0x7f884c0fc0e0 msgr2=0x7f884c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.024+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 shutdown_connections 2024-05-10T04:41:24.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.024+0000 7f8853ceb700 1 -- 172.21.15.150:0/2476688819 wait complete. 2024-05-10T04:41:24.040 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 -- 172.21.15.150:0/4273422770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e01002c0 msgr2=0x7f92e01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 --2- 172.21.15.150:0/4273422770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e01002c0 0x7f92e01006d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f92d400d990 tx=0x7f92d400dca0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 -- 172.21.15.150:0/4273422770 shutdown_connections 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 --2- 172.21.15.150:0/4273422770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e0101d80 0x7f92e0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 --2- 172.21.15.150:0/4273422770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e01002c0 0x7f92e01006d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 --2- 172.21.15.150:0/4273422770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92e01013e0 0x7f92e01017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 -- 172.21.15.150:0/4273422770 >> 172.21.15.150:0/4273422770 conn(0x7f92e00fb8f0 msgr2=0x7f92e00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 -- 172.21.15.150:0/4273422770 shutdown_connections 2024-05-10T04:41:24.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.249+0000 7f92e71be700 1 -- 172.21.15.150:0/4273422770 wait complete. 2024-05-10T04:41:24.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.250+0000 7f92e71be700 1 Processor -- start 2024-05-10T04:41:24.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.250+0000 7f92e71be700 1 -- start start 2024-05-10T04:41:24.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.250+0000 7f92e71be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92e01002c0 0x7f92e0195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e71be700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e01013e0 0x7f92e01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e71be700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e0101d80 0x7f92e019d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e71be700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e019d860 con 0x7f92e01013e0 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e71be700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e019d9d0 con 0x7f92e0101d80 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e71be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e01a1fa0 con 0x7f92e01002c0 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e01013e0 0x7f92e01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e4f5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92e01002c0 0x7f92e0195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e01013e0 0x7f92e01962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44996/0 (socket says 172.21.15.150:44996) 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92dffff700 1 -- 172.21.15.150:0/3628781001 learned_addr learned my addr 172.21.15.150:0/3628781001 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e575b700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e0101d80 0x7f92e019d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e4f5a700 1 -- 172.21.15.150:0/3628781001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e0101d80 msgr2=0x7f92e019d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e0101d80 0x7f92e019d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e4f5a700 1 -- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e01013e0 msgr2=0x7f92e01962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e01013e0 0x7f92e01962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e4f5a700 1 -- 172.21.15.150:0/3628781001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d400d590 con 0x7f92e01002c0 2024-05-10T04:41:24.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.251+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92e01002c0 0x7f92e0195d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92d4010040 tx=0x7f92d4005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.252+0000 7f92ddffb700 1 -- 172.21.15.150:0/3628781001 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4022070 con 0x7f92e01002c0 2024-05-10T04:41:24.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.252+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e01a2220 con 0x7f92e01002c0 2024-05-10T04:41:24.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.252+0000 7f92ddffb700 1 -- 172.21.15.150:0/3628781001 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92d400c540 con 0x7f92e01002c0 2024-05-10T04:41:24.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.252+0000 7f92ddffb700 1 -- 172.21.15.150:0/3628781001 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4003b10 con 0x7f92e01002c0 2024-05-10T04:41:24.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.252+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e01a2640 con 0x7f92e01002c0 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 Processor -- start 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 -- start start 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e46af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e47980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e4e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822d6d150 con 0x55a822e47570 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822d6ab10 con 0x55a822e466a0 2024-05-10T04:41:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92f7502b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822d66400 con 0x55a822e47ec0 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92dffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e4e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92dffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e4e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:37730/0 (socket says 172.21.15.150:37730) 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92dffff700 1 -- 172.21.15.150:0/3370565772 learned_addr learned my addr 172.21.15.150:0/3370565772 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.257+0000 7f92e575b700 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e46af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92dffff700 1 -- 172.21.15.150:0/3370565772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 msgr2=0x55a822e46af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92dffff700 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e46af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92dffff700 1 -- 172.21.15.150:0/3370565772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 msgr2=0x55a822e47980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92dffff700 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e47980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92dffff700 1 -- 172.21.15.150:0/3370565772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a822be73b0 con 0x55a822e47ec0 2024-05-10T04:41:24.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92dffff700 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e4e7c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009e50 tx=0x7f92cc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17492a9dabf58536 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92caffd700 1 -- 172.21.15.150:0/3370565772 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004a30 con 0x55a822e47ec0 2024-05-10T04:41:24.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92caffd700 1 -- 172.21.15.150:0/3370565772 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92cc0054e0 con 0x55a822e47ec0 2024-05-10T04:41:24.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.258+0000 7f92caffd700 1 -- 172.21.15.150:0/3370565772 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0197e0 con 0x55a822e47ec0 2024-05-10T04:41:24.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.259+0000 7f92f7502b80 1 -- 172.21.15.150:0/3370565772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 msgr2=0x55a822e4e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.259+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e4e7c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009e50 tx=0x7f92cc011720 comp rx=0 tx=0).stop 2024-05-10T04:41:24.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.259+0000 7f92ddffb700 1 -- 172.21.15.150:0/3628781001 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f92d4003c70 con 0x7f92e01002c0 2024-05-10T04:41:24.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.260+0000 7f92ddffb700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92d00a12c0 0x7f92d00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.260+0000 7f92e575b700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92d00a12c0 0x7f92d00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.260+0000 7f92ddffb700 1 -- 172.21.15.150:0/3628781001 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f92d40cf4f0 con 0x7f92e01002c0 2024-05-10T04:41:24.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 -- 172.21.15.150:0/3370565772 shutdown_connections 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e4e7c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e46af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3370565772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e47980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 -- 172.21.15.150:0/3370565772 >> 172.21.15.150:0/3370565772 conn(0x55a822da8a60 msgr2=0x55a822e40fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92e575b700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92d00a12c0 0x7f92d00a37d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55a822e46380 tx=0x7f92d800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 -- 172.21.15.150:0/3370565772 shutdown_connections 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 -- 172.21.15.150:0/3370565772 wait complete. 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.261+0000 7f92f7502b80 1 -- start start 2024-05-10T04:41:24.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.262+0000 7f92f7502b80 1 Processor -- start 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e466a0 0x55a822e58e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e47570 0x55a822e593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47ec0 0x55a822e60300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92f7502b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822d02560 con 0x55a822e47ec0 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92f7502b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822d02200 con 0x55a822e47570 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92f7502b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822be96a0 con 0x55a822e466a0 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92f7502b80 10 client.0 _ll_register_callbacks cb 0x55a822d0bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.265 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92e4f5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e466a0 0x55a822e58e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92e575b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47ec0 0x55a822e60300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92e4f5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e466a0 0x55a822e58e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:37734/0 (socket says 172.21.15.150:37734) 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92e4f5a700 1 -- 172.21.15.150:0/90034865 learned_addr learned my addr 172.21.15.150:0/90034865 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.263+0000 7f92dffff700 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e47570 0x55a822e593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92e4f5a700 1 -- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e47570 msgr2=0x55a822e593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e47570 0x55a822e593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92e4f5a700 1 -- 172.21.15.150:0/90034865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47ec0 msgr2=0x55a822e60300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47ec0 0x55a822e60300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92e4f5a700 1 -- 172.21.15.150:0/90034865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc009a80 con 0x55a822e466a0 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e466a0 0x55a822e58e60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92d4099040 tx=0x7f92d4009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb32098a175c2347 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.266 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92bffff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:41:24.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92bffff700 1 -- 172.21.15.150:0/90034865 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d409c110 con 0x55a822e466a0 2024-05-10T04:41:24.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92bffff700 1 -- 172.21.15.150:0/90034865 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f92d409ccd0 con 0x55a822e466a0 2024-05-10T04:41:24.267 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a822cbf270 con 0x55a822e466a0 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92bffff700 1 -- 172.21.15.150:0/90034865 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92d409bf30 con 0x55a822e466a0 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.264+0000 7f92f7502b80 10 client.25750 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a822b93b60 con 0x55a822e466a0 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92bffff700 1 -- 172.21.15.150:0/90034865 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a822cbf270 con 0x55a822e466a0 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92bffff700 1 -- 172.21.15.150:0/90034865 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92d40ba030 con 0x55a822e466a0 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92f7502b80 10 client.25750 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92f7502b80 10 client.25750 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92f7502b80 20 client.25750 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92f7502b80 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e54340 0x55a822e56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92f7502b80 4 client.25750 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a822e15780 con 0x55a822e54340 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.265+0000 7f92dffff700 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e54340 0x55a822e56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.266+0000 7f92bffff700 10 client.25750 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:24.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.266+0000 7f92dffff700 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e54340 0x55a822e56760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e8d685d31486485 server_cookie=161631a3bdd3695 in_seq=0 out_seq=0 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92bffff700 1 -- 172.21.15.150:0/90034865 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a822e15780 con 0x55a822e54340 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92bffff700 10 client.25750 handle_command_reply: tid=0 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 client.25750 shutdown 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e466a0 msgr2=0x55a822e58e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e466a0 0x55a822e58e60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92d4099040 tx=0x7f92d4009f60 comp rx=0 tx=0).stop 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 shutdown_connections 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e54340 0x55a822e56760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47ec0 0x55a822e60300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e466a0 0x55a822e58e60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 --2- 172.21.15.150:0/90034865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e47570 0x55a822e593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92bffff700 0 client.25750 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 >> 172.21.15.150:0/90034865 conn(0x55a822da8a60 msgr2=0x55a822e40fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.267+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 shutdown_connections 2024-05-10T04:41:24.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.268+0000 7f92f7502b80 1 -- 172.21.15.150:0/90034865 wait complete. 2024-05-10T04:41:24.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.268+0000 7f92f7502b80 20 client.25750 trim_cache size 0 max 16384 2024-05-10T04:41:24.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 Processor -- start 2024-05-10T04:41:24.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 -- start start 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e59040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822e471f0 con 0x55a822e47570 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822e46f50 con 0x55a822e466a0 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92f7502b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822e4e5b0 con 0x55a822e47ec0 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45022/0 (socket says 172.21.15.150:45022) 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.281+0000 7f92dffff700 1 -- 172.21.15.150:0/857027889 learned_addr learned my addr 172.21.15.150:0/857027889 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92e575b700 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e59040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92dffff700 1 -- 172.21.15.150:0/857027889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 msgr2=0x55a822e59040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92dffff700 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e59040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92dffff700 1 -- 172.21.15.150:0/857027889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 msgr2=0x55a822e4a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92dffff700 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e4a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92dffff700 1 -- 172.21.15.150:0/857027889 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a822d2c560 con 0x55a822e47570 2024-05-10T04:41:24.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92dffff700 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e602c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92cc01b040 tx=0x7f92cc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3e2719c3fbb337 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92cb7fe700 1 -- 172.21.15.150:0/857027889 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc005080 con 0x55a822e47570 2024-05-10T04:41:24.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92cb7fe700 1 -- 172.21.15.150:0/857027889 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92cc0051e0 con 0x55a822e47570 2024-05-10T04:41:24.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.282+0000 7f92cb7fe700 1 -- 172.21.15.150:0/857027889 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc003680 con 0x55a822e47570 2024-05-10T04:41:24.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 -- 172.21.15.150:0/857027889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 msgr2=0x55a822e602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e602c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92cc01b040 tx=0x7f92cc011720 comp rx=0 tx=0).stop 2024-05-10T04:41:24.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 -- 172.21.15.150:0/857027889 shutdown_connections 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e59040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e4a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 --2- 172.21.15.150:0/857027889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e602c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 -- 172.21.15.150:0/857027889 >> 172.21.15.150:0/857027889 conn(0x55a822da8a60 msgr2=0x55a822e40f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 -- 172.21.15.150:0/857027889 shutdown_connections 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.283+0000 7f92f7502b80 1 -- 172.21.15.150:0/857027889 wait complete. 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 -- start start 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 Processor -- start 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e5e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e5e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e5f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822e5fd00 con 0x55a822e47570 2024-05-10T04:41:24.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822e5fe70 con 0x55a822e466a0 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a822e60030 con 0x55a822e47ec0 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.284+0000 7f92f7502b80 10 client.0 _ll_register_callbacks cb 0x55a822d0bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e5e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e5e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45038/0 (socket says 172.21.15.150:45038) 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 -- 172.21.15.150:0/3622297106 learned_addr learned my addr 172.21.15.150:0/3622297106 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92e575b700 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e5f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 -- 172.21.15.150:0/3622297106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 msgr2=0x55a822e5f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e5f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 -- 172.21.15.150:0/3622297106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 msgr2=0x55a822e5e3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e5e3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 -- 172.21.15.150:0/3622297106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a822be73b0 con 0x55a822e47570 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92dffff700 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e5e930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f92cc000c00 tx=0x7f92cc009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d0406521290064 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.289 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.285+0000 7f92bffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:24.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.286+0000 7f92bffff700 1 -- 172.21.15.150:0/3622297106 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc02ba70 con 0x55a822e47570 2024-05-10T04:41:24.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.286+0000 7f92bffff700 1 -- 172.21.15.150:0/3622297106 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f92cc02a070 con 0x55a822e47570 2024-05-10T04:41:24.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.286+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92cc0054e0 con 0x55a822e47570 2024-05-10T04:41:24.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.286+0000 7f92bffff700 1 -- 172.21.15.150:0/3622297106 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92cc02b4c0 con 0x55a822e47570 2024-05-10T04:41:24.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.286+0000 7f92f7502b80 10 client.16203 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.286+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a822bebcd0 con 0x55a822e47570 2024-05-10T04:41:24.290 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.286+0000 7f92bffff700 1 -- 172.21.15.150:0/3622297106 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92cc0054e0 con 0x55a822e47570 2024-05-10T04:41:24.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92bffff700 1 -- 172.21.15.150:0/3622297106 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92cc0614e0 con 0x55a822e47570 2024-05-10T04:41:24.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92f7502b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92f7502b80 10 client.16203 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:41:24.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92f7502b80 20 client.16203 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e60be0 0x55a822e63000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92f7502b80 4 client.16203 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a822d5eec0 con 0x55a822e60be0 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.287+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e60be0 0x55a822e63000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92bffff700 10 client.16203 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92e4f5a700 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e60be0 0x55a822e63000 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba1835872e669bbf server_cookie=a31f54de2f7ed6af in_seq=0 out_seq=0 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92bffff700 1 -- 172.21.15.150:0/3622297106 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a822d5eec0 con 0x55a822e60be0 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92bffff700 10 client.16203 handle_command_reply: tid=0 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 client.16203 shutdown 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 msgr2=0x55a822e5e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e5e930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f92cc000c00 tx=0x7f92cc009dd0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 shutdown_connections 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55a822e60be0 0x55a822e63000 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a822e47ec0 0x55a822e5f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a822e466a0 0x55a822e5e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 --2- 172.21.15.150:0/3622297106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a822e47570 0x55a822e5e930 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 >> 172.21.15.150:0/3622297106 conn(0x55a822da8a60 msgr2=0x55a822e40f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92bffff700 0 client.16203 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:24.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 shutdown_connections 2024-05-10T04:41:24.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.288+0000 7f92f7502b80 1 -- 172.21.15.150:0/3622297106 wait complete. 2024-05-10T04:41:24.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92f7502b80 20 client.16203 trim_cache size 0 max 16384 2024-05-10T04:41:24.294 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92d00a12c0 msgr2=0x7f92d00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92d00a12c0 0x7f92d00a37d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55a822e46380 tx=0x7f92d800c040 comp rx=0 tx=0).stop 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92e01002c0 msgr2=0x7f92e0195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92e01002c0 0x7f92e0195d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92d4010040 tx=0x7f92d4005300 comp rx=0 tx=0).stop 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 shutdown_connections 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92e0101d80 0x7f92e019d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92d00a12c0 0x7f92d00a37d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92e01002c0 0x7f92e0195d90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 --2- 172.21.15.150:0/3628781001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e01013e0 0x7f92e01962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.289+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 >> 172.21.15.150:0/3628781001 conn(0x7f92e00fb8f0 msgr2=0x7f92e0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.291+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 shutdown_connections 2024-05-10T04:41:24.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.291+0000 7f92e71be700 1 -- 172.21.15.150:0/3628781001 wait complete. 2024-05-10T04:41:24.304 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:41:24.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 -- 172.21.15.150:0/3696609063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28107a20 msgr2=0x7fde28109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 --2- 172.21.15.150:0/3696609063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28107a20 0x7fde28109e40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fde2000b0a0 tx=0x7fde2000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 -- 172.21.15.150:0/3696609063 shutdown_connections 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 --2- 172.21.15.150:0/3696609063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28107a20 0x7fde28109e40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 --2- 172.21.15.150:0/3696609063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde28102760 0x7fde28104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 --2- 172.21.15.150:0/3696609063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde281050c0 0x7fde281074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 -- 172.21.15.150:0/3696609063 >> 172.21.15.150:0/3696609063 conn(0x7fde280fc180 msgr2=0x7fde280fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.441+0000 7fde2f196700 1 -- 172.21.15.150:0/3696609063 shutdown_connections 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 -- 172.21.15.150:0/3696609063 wait complete. 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 Processor -- start 2024-05-10T04:41:24.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 -- start start 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28102760 0x7fde28101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde281050c0 0x7fde28100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28107a20 0x7fde28100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde28101290 con 0x7fde28102760 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde28101400 con 0x7fde28107a20 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.442+0000 7fde2f196700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde281a20c0 con 0x7fde281050c0 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28102760 0x7fde28101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28102760 0x7fde28101fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45052/0 (socket says 172.21.15.150:45052) 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 -- 172.21.15.150:0/2339684675 learned_addr learned my addr 172.21.15.150:0/2339684675 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde27fff700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde281050c0 0x7fde28100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2d733700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28107a20 0x7fde28100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 -- 172.21.15.150:0/2339684675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde281050c0 msgr2=0x7fde28100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde281050c0 0x7fde28100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 -- 172.21.15.150:0/2339684675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28107a20 msgr2=0x7fde28100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28107a20 0x7fde28100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 -- 172.21.15.150:0/2339684675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde20009d00 con 0x7fde28102760 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28102760 0x7fde28101fd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00d980 tx=0x7fde1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde25ffb700 1 -- 172.21.15.150:0/2339684675 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00f890 con 0x7fde28102760 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde25ffb700 1 -- 172.21.15.150:0/2339684675 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1c010460 con 0x7fde28102760 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde25ffb700 1 -- 172.21.15.150:0/2339684675 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00e610 con 0x7fde28102760 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.443+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde281a2320 con 0x7fde28102760 2024-05-10T04:41:24.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.444+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde281a27b0 con 0x7fde28102760 2024-05-10T04:41:24.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.446+0000 7fde25ffb700 1 -- 172.21.15.150:0/2339684675 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fde1c0105d0 con 0x7fde28102760 2024-05-10T04:41:24.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.447+0000 7fde25ffb700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fde180a12c0 0x7fde180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.447+0000 7fde25ffb700 1 -- 172.21.15.150:0/2339684675 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fde1c0cc640 con 0x7fde28102760 2024-05-10T04:41:24.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 Processor -- start 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 -- start start 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596447c9150 con 0x5596448abd80 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596447c6b10 con 0x5596448aaeb0 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596447c2400 con 0x5596448ac6d0 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde27fff700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fde180a12c0 0x7fde180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde2cf32700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde2cf32700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41034/0 (socket says 172.21.15.150:41034) 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.448+0000 7fde2cf32700 1 -- 172.21.15.150:0/2012424647 learned_addr learned my addr 172.21.15.150:0/2012424647 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde27fff700 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde2d733700 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde2cf32700 1 -- 172.21.15.150:0/2012424647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 msgr2=0x5596448b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde2cf32700 1 -- 172.21.15.150:0/2012424647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 msgr2=0x5596448ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde2cf32700 1 -- 172.21.15.150:0/2012424647 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596446433b0 con 0x5596448aaeb0 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde27fff700 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde27fff700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fde180a12c0 0x7fde180a37d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde28101970 tx=0x7fde1400bb90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448ab300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde1c095b90 tx=0x7fde1c099070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b45af72faecbca11 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde12ffd700 1 -- 172.21.15.150:0/2012424647 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c099690 con 0x5596448aaeb0 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde12ffd700 1 -- 172.21.15.150:0/2012424647 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1c09d4c0 con 0x5596448aaeb0 2024-05-10T04:41:24.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.449+0000 7fde12ffd700 1 -- 172.21.15.150:0/2012424647 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0108b0 con 0x5596448aaeb0 2024-05-10T04:41:24.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.450+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2012424647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 msgr2=0x5596448ab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448ab300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde1c095b90 tx=0x7fde1c099070 comp rx=0 tx=0).stop 2024-05-10T04:41:24.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2012424647 shutdown_connections 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448ab300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2012424647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2012424647 >> 172.21.15.150:0/2012424647 conn(0x559644804a60 msgr2=0x55964489cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2012424647 shutdown_connections 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2012424647 wait complete. 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 -- start start 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.451+0000 7fde3f4dab80 1 Processor -- start 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448abd80 0x5596448c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448ac6d0 0x5596448cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964475e560 con 0x5596448aaeb0 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964475e200 con 0x5596448ac6d0 2024-05-10T04:41:24.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596446456a0 con 0x5596448abd80 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde3f4dab80 10 client.0 _ll_register_callbacks cb 0x559644767b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448ac6d0 0x5596448cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448ac6d0 0x5596448cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41042/0 (socket says 172.21.15.150:41042) 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 -- 172.21.15.150:0/597967611 learned_addr learned my addr 172.21.15.150:0/597967611 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde27fff700 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448abd80 0x5596448c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 -- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448abd80 msgr2=0x5596448c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448abd80 0x5596448c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 -- 172.21.15.150:0/597967611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 msgr2=0x5596448c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.452+0000 7fde2d733700 1 -- 172.21.15.150:0/597967611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1401a040 con 0x5596448ac6d0 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.453+0000 7fde2d733700 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448ac6d0 0x5596448cd300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde2806a860 tx=0x7fde20009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80bfc6eb5a90f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.456 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.453+0000 7fde07fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:24.458 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.454+0000 7fde07fff700 1 -- 172.21.15.150:0/597967611 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2000e040 con 0x5596448ac6d0 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.454+0000 7fde07fff700 1 -- 172.21.15.150:0/597967611 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fde20003c10 con 0x5596448ac6d0 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.454+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55964471b270 con 0x5596448ac6d0 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.454+0000 7fde07fff700 1 -- 172.21.15.150:0/597967611 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde20009780 con 0x5596448ac6d0 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.454+0000 7fde3f4dab80 10 client.25856 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.454+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fde281a27b0 con 0x5596448ac6d0 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde07fff700 1 -- 172.21.15.150:0/597967611 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55964471b270 con 0x5596448ac6d0 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde07fff700 1 -- 172.21.15.150:0/597967611 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fde20019070 con 0x5596448ac6d0 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde3f4dab80 10 client.25856 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde3f4dab80 10 client.25856 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde3f4dab80 20 client.25856 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5596448d1a80 0x5596448d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde3f4dab80 4 client.25856 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559644871780 con 0x5596448d1a80 2024-05-10T04:41:24.459 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.455+0000 7fde2cf32700 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5596448d1a80 0x5596448d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.456+0000 7fde2cf32700 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5596448d1a80 0x5596448d3ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8048f737eb9b2bac server_cookie=9330139aa5f01cae in_seq=0 out_seq=0 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.456+0000 7fde07fff700 10 client.25856 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.456+0000 7fde07fff700 1 -- 172.21.15.150:0/597967611 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559644871780 con 0x5596448d1a80 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.456+0000 7fde07fff700 10 client.25856 handle_command_reply: tid=0 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.456+0000 7fde3f4dab80 1 client.25856 shutdown 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448ac6d0 msgr2=0x5596448cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448ac6d0 0x5596448cd300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fde2806a860 tx=0x7fde20009300 comp rx=0 tx=0).stop 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 shutdown_connections 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x5596448d1a80 0x5596448d3ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448ac6d0 0x5596448cd300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/597967611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448abd80 0x5596448c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 >> 172.21.15.150:0/597967611 conn(0x559644804a60 msgr2=0x55964489cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde07fff700 0 client.25856 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:24.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 shutdown_connections 2024-05-10T04:41:24.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 1 -- 172.21.15.150:0/597967611 wait complete. 2024-05-10T04:41:24.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.457+0000 7fde3f4dab80 20 client.25856 trim_cache size 0 max 16384 2024-05-10T04:41:24.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 Processor -- start 2024-05-10T04:41:24.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 -- start start 2024-05-10T04:41:24.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448abd80 0x5596448cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596448977c0 con 0x5596448aaeb0 2024-05-10T04:41:24.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596448ab9b0 con 0x5596448abd80 2024-05-10T04:41:24.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55964489e770 con 0x5596448ac6d0 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde2cf32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45090/0 (socket says 172.21.15.150:45090) 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.472+0000 7fde2cf32700 1 -- 172.21.15.150:0/3965870331 learned_addr learned my addr 172.21.15.150:0/3965870331 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde2d733700 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde27fff700 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448abd80 0x5596448cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde2cf32700 1 -- 172.21.15.150:0/3965870331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 msgr2=0x5596448cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde2cf32700 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde2cf32700 1 -- 172.21.15.150:0/3965870331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448abd80 msgr2=0x5596448cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde2cf32700 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448abd80 0x5596448cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde2cf32700 1 -- 172.21.15.150:0/3965870331 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596445efb60 con 0x5596448aaeb0 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde27fff700 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448abd80 0x5596448cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde2cf32700 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c50f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fde1c099160 tx=0x7fde1c084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f3f22cf574ca1f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde137fe700 1 -- 172.21.15.150:0/3965870331 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c099690 con 0x5596448aaeb0 2024-05-10T04:41:24.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde137fe700 1 -- 172.21.15.150:0/3965870331 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1c097a30 con 0x5596448aaeb0 2024-05-10T04:41:24.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.473+0000 7fde137fe700 1 -- 172.21.15.150:0/3965870331 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c09d170 con 0x5596448aaeb0 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 -- 172.21.15.150:0/3965870331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 msgr2=0x5596448c50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c50f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fde1c099160 tx=0x7fde1c084040 comp rx=0 tx=0).stop 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 -- 172.21.15.150:0/3965870331 shutdown_connections 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x5596448cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448aaeb0 0x5596448c50f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/3965870331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448abd80 0x5596448cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 -- 172.21.15.150:0/3965870331 >> 172.21.15.150:0/3965870331 conn(0x559644804a60 msgr2=0x55964489dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 -- 172.21.15.150:0/3965870331 shutdown_connections 2024-05-10T04:41:24.478 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.474+0000 7fde3f4dab80 1 -- 172.21.15.150:0/3965870331 wait complete. 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 -- start start 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 Processor -- start 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x559644926270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596449267b0 con 0x5596448abd80 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596449268f0 con 0x5596448aaeb0 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x559644926ab0 con 0x5596448ac6d0 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.475+0000 7fde3f4dab80 10 client.0 _ll_register_callbacks cb 0x559644767b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde27fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde27fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45092/0 (socket says 172.21.15.150:45092) 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde27fff700 1 -- 172.21.15.150:0/2468087758 learned_addr learned my addr 172.21.15.150:0/2468087758 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.479 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2d733700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x559644926270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2cf32700 1 -- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 msgr2=0x559644926270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x559644926270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2cf32700 1 -- 172.21.15.150:0/2468087758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 msgr2=0x5596448cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2cf32700 1 -- 172.21.15.150:0/2468087758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596446433b0 con 0x5596448aaeb0 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde27fff700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde2cf32700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448cad10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5596448c5ca0 tx=0x7fde1c095a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93d1cb8b1607836 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.476+0000 7fde07fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.477+0000 7fde07fff700 1 -- 172.21.15.150:0/2468087758 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c099690 con 0x5596448aaeb0 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.477+0000 7fde07fff700 1 -- 172.21.15.150:0/2468087758 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fde1c0108b0 con 0x5596448aaeb0 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.477+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde1c09d4c0 con 0x5596448aaeb0 2024-05-10T04:41:24.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.477+0000 7fde07fff700 1 -- 172.21.15.150:0/2468087758 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde1c09d170 con 0x5596448aaeb0 2024-05-10T04:41:24.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 10 client.25859 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559644788560 con 0x5596448aaeb0 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde07fff700 1 -- 172.21.15.150:0/2468087758 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1c09d4c0 con 0x5596448aaeb0 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde07fff700 1 -- 172.21.15.150:0/2468087758 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fde1c07f030 con 0x5596448aaeb0 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 10 client.25859 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 10 client.25859 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 20 client.25859 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x559644927ec0 0x55964492a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 4 client.25859 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596447baec0 con 0x559644927ec0 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.478+0000 7fde27fff700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x559644927ec0 0x55964492a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde27fff700 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x559644927ec0 0x55964492a2e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f44ff58d89cbc74 server_cookie=46180d2111acfd09 in_seq=0 out_seq=0 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde07fff700 10 client.25859 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:24.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde07fff700 1 -- 172.21.15.150:0/2468087758 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596447baec0 con 0x559644927ec0 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde07fff700 10 client.25859 handle_command_reply: tid=0 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 client.25859 shutdown 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 msgr2=0x5596448cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448cad10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5596448c5ca0 tx=0x7fde1c095a90 comp rx=0 tx=0).stop 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 shutdown_connections 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x559644927ec0 0x55964492a2e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5596448ac6d0 0x559644926270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5596448aaeb0 0x5596448cad10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 --2- 172.21.15.150:0/2468087758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5596448abd80 0x5596448cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 >> 172.21.15.150:0/2468087758 conn(0x559644804a60 msgr2=0x5596448c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde07fff700 0 client.25859 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 shutdown_connections 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.479+0000 7fde3f4dab80 1 -- 172.21.15.150:0/2468087758 wait complete. 2024-05-10T04:41:24.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.480+0000 7fde3f4dab80 20 client.25859 trim_cache size 0 max 16384 2024-05-10T04:41:24.485 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:24.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.480+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fde180a12c0 msgr2=0x7fde180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.480+0000 7fde2f196700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fde180a12c0 0x7fde180a37d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde28101970 tx=0x7fde1400bb90 comp rx=0 tx=0).stop 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.481+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28102760 msgr2=0x7fde28101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.481+0000 7fde2f196700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28102760 0x7fde28101fd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00d980 tx=0x7fde1c00dc90 comp rx=0 tx=0).stop 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 shutdown_connections 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde28107a20 0x7fde28100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde28102760 0x7fde28101fd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fde180a12c0 0x7fde180a37d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 --2- 172.21.15.150:0/2339684675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde281050c0 0x7fde28100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 >> 172.21.15.150:0/2339684675 conn(0x7fde280fc180 msgr2=0x7fde281035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 shutdown_connections 2024-05-10T04:41:24.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.483+0000 7fde2f196700 1 -- 172.21.15.150:0/2339684675 wait complete. 2024-05-10T04:41:24.501 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/1941544081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 msgr2=0x7fc0f4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/1941544081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f4100ed0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009990 tx=0x7fc0f0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/1941544081 shutdown_connections 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/1941544081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0f4102580 0x7fc0f4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/1941544081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f4100ed0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/1941544081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f4101be0 0x7fc0f4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/1941544081 >> 172.21.15.150:0/1941544081 conn(0x7fc0f40fc0d0 msgr2=0x7fc0f40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/1941544081 shutdown_connections 2024-05-10T04:41:24.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.641+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/1941544081 wait complete. 2024-05-10T04:41:24.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 Processor -- start 2024-05-10T04:41:24.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 -- start start 2024-05-10T04:41:24.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f410b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0f4101be0 0x7fc0f4109820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f4102580 0x7fc0f4109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410a440 con 0x7fc0f4100ac0 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f410a5b0 con 0x7fc0f4101be0 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0fb9d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f41a1f70 con 0x7fc0f4102580 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.642+0000 7fc0f9775700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f410b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f410b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45116/0 (socket says 172.21.15.150:45116) 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 -- 172.21.15.150:0/3415846414 learned_addr learned my addr 172.21.15.150:0/3415846414 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f4102580 0x7fc0f4109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 -- 172.21.15.150:0/3415846414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f4102580 msgr2=0x7fc0f4109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f4102580 0x7fc0f4109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 -- 172.21.15.150:0/3415846414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0f4101be0 msgr2=0x7fc0f4109820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0f4101be0 0x7fc0f4109820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 -- 172.21.15.150:0/3415846414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f00095f0 con 0x7fc0f4100ac0 2024-05-10T04:41:24.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0f9775700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f410b180 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00098e0 tx=0x7fc0f0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0ea7fc700 1 -- 172.21.15.150:0/3415846414 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f001d070 con 0x7fc0f4100ac0 2024-05-10T04:41:24.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f41a2170 con 0x7fc0f4100ac0 2024-05-10T04:41:24.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f41a25f0 con 0x7fc0f4100ac0 2024-05-10T04:41:24.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0ea7fc700 1 -- 172.21.15.150:0/3415846414 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f0021410 con 0x7fc0f4100ac0 2024-05-10T04:41:24.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.643+0000 7fc0ea7fc700 1 -- 172.21.15.150:0/3415846414 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0017650 con 0x7fc0f4100ac0 2024-05-10T04:41:24.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.646+0000 7fc0ea7fc700 1 -- 172.21.15.150:0/3415846414 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fc0f00177b0 con 0x7fc0f4100ac0 2024-05-10T04:41:24.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.646+0000 7fc0ea7fc700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc0e00a1240 0x7fc0e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.647+0000 7fc0ea7fc700 1 -- 172.21.15.150:0/3415846414 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc0f00ce020 con 0x7fc0f4100ac0 2024-05-10T04:41:24.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.647+0000 7fc0f8f74700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc0e00a1240 0x7fc0e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.647+0000 7fc0f8f74700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc0e00a1240 0x7fc0e00a3750 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005950 tx=0x7fc0e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 Processor -- start 2024-05-10T04:41:24.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 -- start start 2024-05-10T04:41:24.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c305300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c306190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c30cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c223150 con 0x55840c3066d0 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c220b10 con 0x55840c305d80 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.648+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c21c400 con 0x55840c304eb0 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c30cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c30cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45124/0 (socket says 172.21.15.150:45124) 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 -- 172.21.15.150:0/1512627839 learned_addr learned my addr 172.21.15.150:0/1512627839 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f8f74700 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c306190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9775700 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c305300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 -- 172.21.15.150:0/1512627839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 msgr2=0x55840c305300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c305300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 -- 172.21.15.150:0/1512627839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 msgr2=0x55840c306190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c306190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 -- 172.21.15.150:0/1512627839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840c09d3b0 con 0x55840c3066d0 2024-05-10T04:41:24.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c30cfd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009370 tx=0x7fc0ec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b65f089358e526a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.649+0000 7fc0df7fe700 1 -- 172.21.15.150:0/1512627839 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec019070 con 0x55840c3066d0 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc0df7fe700 1 -- 172.21.15.150:0/1512627839 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0ec011040 con 0x55840c3066d0 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc0df7fe700 1 -- 172.21.15.150:0/1512627839 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec015cd0 con 0x55840c3066d0 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc10bd1db80 1 -- 172.21.15.150:0/1512627839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 msgr2=0x55840c30cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c30cfd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009370 tx=0x7fc0ec009920 comp rx=0 tx=0).stop 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc10bd1db80 1 -- 172.21.15.150:0/1512627839 shutdown_connections 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c30cfd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c305300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.650+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/1512627839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c306190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.651+0000 7fc10bd1db80 1 -- 172.21.15.150:0/1512627839 >> 172.21.15.150:0/1512627839 conn(0x55840c25ea60 msgr2=0x55840c2f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.651+0000 7fc10bd1db80 1 -- 172.21.15.150:0/1512627839 shutdown_connections 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.651+0000 7fc10bd1db80 1 -- 172.21.15.150:0/1512627839 wait complete. 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.651+0000 7fc10bd1db80 1 -- start start 2024-05-10T04:41:24.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.651+0000 7fc10bd1db80 1 Processor -- start 2024-05-10T04:41:24.657 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c31fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c320410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c327370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c1b8560 con 0x55840c3066d0 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c1b8200 con 0x55840c305d80 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c09f6a0 con 0x55840c304eb0 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc10bd1db80 10 client.0 _ll_register_callbacks cb 0x55840c1c1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c327370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c327370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45134/0 (socket says 172.21.15.150:45134) 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc0f9f76700 1 -- 172.21.15.150:0/2442894091 learned_addr learned my addr 172.21.15.150:0/2442894091 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.652+0000 7fc0f8f74700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c320410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9775700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c31fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.658 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9f76700 1 -- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 msgr2=0x55840c31fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c31fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9f76700 1 -- 172.21.15.150:0/2442894091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 msgr2=0x55840c320410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c320410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9f76700 1 -- 172.21.15.150:0/2442894091 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec009050 con 0x55840c3066d0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9775700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c31fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c327370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000f80 tx=0x7fc0ec016d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb360f5b478016c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0dcff9700 1 -- 172.21.15.150:0/2442894091 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec019040 con 0x55840c3066d0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0dcff9700 1 -- 172.21.15.150:0/2442894091 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc0ec01e870 con 0x55840c3066d0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55840c175270 con 0x55840c3066d0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc0dcff9700 1 -- 172.21.15.150:0/2442894091 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0ec004d60 con 0x55840c3066d0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc10bd1db80 10 client.16233 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.653+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55840c049b60 con 0x55840c3066d0 2024-05-10T04:41:24.659 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc0dcff9700 1 -- 172.21.15.150:0/2442894091 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55840c175270 con 0x55840c3066d0 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc0dcff9700 1 -- 172.21.15.150:0/2442894091 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0ec014070 con 0x55840c3066d0 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc10bd1db80 10 client.16233 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc10bd1db80 10 client.16233 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc10bd1db80 20 client.16233 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c32bb80 0x55840c32dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc10bd1db80 4 client.16233 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55840c2cb780 con 0x55840c32bb80 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc0f9775700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c32bb80 0x55840c32dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.654+0000 7fc0dcff9700 10 client.16233 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.655+0000 7fc0f9775700 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c32bb80 0x55840c32dfd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c9837c14889986c server_cookie=f099ea5ecf702565 in_seq=0 out_seq=0 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.655+0000 7fc0dcff9700 1 -- 172.21.15.150:0/2442894091 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55840c2cb780 con 0x55840c32bb80 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.655+0000 7fc0dcff9700 10 client.16233 handle_command_reply: tid=0 2024-05-10T04:41:24.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.655+0000 7fc10bd1db80 1 client.16233 shutdown 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 msgr2=0x55840c327370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c327370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000f80 tx=0x7fc0ec016d40 comp rx=0 tx=0).stop 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 shutdown_connections 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c32bb80 0x55840c32dfd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c327370 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c31fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc0dcff9700 0 client.16233 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/2442894091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c320410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 >> 172.21.15.150:0/2442894091 conn(0x55840c25ea60 msgr2=0x55840c2f6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 shutdown_connections 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 1 -- 172.21.15.150:0/2442894091 wait complete. 2024-05-10T04:41:24.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.656+0000 7fc10bd1db80 20 client.16233 trim_cache size 0 max 16384 2024-05-10T04:41:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:24 smithi150 ceph-mon[42404]: pgmap v326: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:24.662 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:24 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1953013491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:41:24.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.669+0000 7fc10bd1db80 1 Processor -- start 2024-05-10T04:41:24.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.669+0000 7fc10bd1db80 1 -- start start 2024-05-10T04:41:24.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c327930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c327330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c308fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c2f3d30 con 0x55840c3066d0 2024-05-10T04:41:24.672 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c320d30 con 0x55840c305d80 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c2f8210 con 0x55840c304eb0 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c308fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9775700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c327930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c308fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45146/0 (socket says 172.21.15.150:45146) 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9f76700 1 -- 172.21.15.150:0/873465954 learned_addr learned my addr 172.21.15.150:0/873465954 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9775700 1 -- 172.21.15.150:0/873465954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 msgr2=0x55840c327330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9775700 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c327330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9775700 1 -- 172.21.15.150:0/873465954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 msgr2=0x55840c308fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9775700 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c308fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9775700 1 -- 172.21.15.150:0/873465954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840c1e2560 con 0x55840c304eb0 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.670+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c308fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.671+0000 7fc0f9775700 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c327930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00cb750 tx=0x7fc0f0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec428ac85850cb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.673 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.671+0000 7fc0dffff700 1 -- 172.21.15.150:0/873465954 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0084070 con 0x55840c304eb0 2024-05-10T04:41:24.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.671+0000 7fc0dffff700 1 -- 172.21.15.150:0/873465954 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f00a0e90 con 0x55840c304eb0 2024-05-10T04:41:24.674 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.671+0000 7fc0dffff700 1 -- 172.21.15.150:0/873465954 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f009b630 con 0x55840c304eb0 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 -- 172.21.15.150:0/873465954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 msgr2=0x55840c327930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c327930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00cb750 tx=0x7fc0f0009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 -- 172.21.15.150:0/873465954 shutdown_connections 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c308fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c327930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/873465954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c327330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 -- 172.21.15.150:0/873465954 >> 172.21.15.150:0/873465954 conn(0x55840c25ea60 msgr2=0x55840c2f6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 -- 172.21.15.150:0/873465954 shutdown_connections 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 -- 172.21.15.150:0/873465954 wait complete. 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 -- start start 2024-05-10T04:41:24.677 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.672+0000 7fc10bd1db80 1 Processor -- start 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c324a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c324f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc10bd1db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c325e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c326370 con 0x55840c3066d0 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c3264e0 con 0x55840c305d80 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc10bd1db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840c3266a0 con 0x55840c304eb0 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc10bd1db80 10 client.0 _ll_register_callbacks cb 0x55840c1c1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c325e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.678 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc0f9f76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c325e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45158/0 (socket says 172.21.15.150:45158) 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc0f9f76700 1 -- 172.21.15.150:0/4205830639 learned_addr learned my addr 172.21.15.150:0/4205830639 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc0f9775700 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c324a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.673+0000 7fc0f9f76700 1 -- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 msgr2=0x55840c324a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c324a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0f9f76700 1 -- 172.21.15.150:0/4205830639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 msgr2=0x55840c324f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0f8f74700 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c324f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c324f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0f9f76700 1 -- 172.21.15.150:0/4205830639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55840c049b60 con 0x55840c3066d0 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0f9f76700 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c325e30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec01da20 tx=0x7fc0ec009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b16d15701ee3fd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0dcff9700 1 -- 172.21.15.150:0/4205830639 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec019040 con 0x55840c3066d0 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0dcff9700 1 -- 172.21.15.150:0/4205830639 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fc0ec01dd10 con 0x55840c3066d0 2024-05-10T04:41:24.679 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0ec011040 con 0x55840c3066d0 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.674+0000 7fc0dcff9700 1 -- 172.21.15.150:0/4205830639 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0ec004ea0 con 0x55840c3066d0 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.675+0000 7fc10bd1db80 10 client.16239 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.675+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fc0f41a25f0 con 0x55840c3066d0 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.675+0000 7fc0dcff9700 1 -- 172.21.15.150:0/4205830639 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0ec03a020 con 0x55840c3066d0 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.675+0000 7fc0dcff9700 1 -- 172.21.15.150:0/4205830639 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0ec014070 con 0x55840c3066d0 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc10bd1db80 10 client.16239 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc10bd1db80 10 client.16239 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc10bd1db80 20 client.16239 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c382480 0x55840c3848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc10bd1db80 4 client.16239 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:24.680 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55840c214ec0 con 0x55840c382480 2024-05-10T04:41:24.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc0f9775700 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c382480 0x55840c3848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.681 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc0dcff9700 10 client.16239 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.676+0000 7fc0f9775700 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c382480 0x55840c3848a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c36a932e8622afe1 server_cookie=842296e7653a26da in_seq=0 out_seq=0 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc0dcff9700 1 -- 172.21.15.150:0/4205830639 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55840c214ec0 con 0x55840c382480 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc0dcff9700 10 client.16239 handle_command_reply: tid=0 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 client.16239 shutdown 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 msgr2=0x55840c325e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c325e30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec01da20 tx=0x7fc0ec009e60 comp rx=0 tx=0).stop 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 shutdown_connections 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55840c382480 0x55840c3848a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55840c3066d0 0x55840c325e30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55840c304eb0 0x55840c324a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.682 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc0dcff9700 0 client.16239 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:24.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 --2- 172.21.15.150:0/4205830639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55840c305d80 0x55840c324f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 >> 172.21.15.150:0/4205830639 conn(0x55840c25ea60 msgr2=0x55840c327890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 shutdown_connections 2024-05-10T04:41:24.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 1 -- 172.21.15.150:0/4205830639 wait complete. 2024-05-10T04:41:24.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.677+0000 7fc10bd1db80 20 client.16239 trim_cache size 0 max 16384 2024-05-10T04:41:24.683 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc0e00a1240 msgr2=0x7fc0e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc0e00a1240 0x7fc0e00a3750 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005950 tx=0x7fc0e40058e0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 msgr2=0x7fc0f410b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f410b180 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00098e0 tx=0x7fc0f0005b00 comp rx=0 tx=0).stop 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 shutdown_connections 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fc0e00a1240 0x7fc0e00a3750 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f4102580 0x7fc0f4109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0f4100ac0 0x7fc0f410b180 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 --2- 172.21.15.150:0/3415846414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0f4101be0 0x7fc0f4109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 >> 172.21.15.150:0/3415846414 conn(0x7fc0f40fc0d0 msgr2=0x7fc0f4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 shutdown_connections 2024-05-10T04:41:24.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.678+0000 7fc0fb9d9700 1 -- 172.21.15.150:0/3415846414 wait complete. 2024-05-10T04:41:24.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:41:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:24 smithi161 ceph-mon[46912]: pgmap v326: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:24.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:24 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1953013491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/1762272512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 msgr2=0x7fe3e4108630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/1762272512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 0x7fe3e4108630 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00b0a0 tx=0x7fe3dc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/1762272512 shutdown_connections 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/1762272512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 0x7fe3e4108630 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/1762272512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41002c0 0x7fe3e41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/1762272512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e41013e0 0x7fe3e41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/1762272512 >> 172.21.15.150:0/1762272512 conn(0x7fe3e40fb8f0 msgr2=0x7fe3e40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/1762272512 shutdown_connections 2024-05-10T04:41:24.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.843+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/1762272512 wait complete. 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 Processor -- start 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 -- start start 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41002c0 0x7fe3e4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e41013e0 0x7fe3e4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 0x7fe3e419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419fae0 con 0x7fe3e4101d80 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419fc50 con 0x7fe3e41002c0 2024-05-10T04:41:24.847 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.844+0000 7fe3e8eb7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419fe10 con 0x7fe3e41013e0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41002c0 0x7fe3e4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41002c0 0x7fe3e4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41144/0 (socket says 172.21.15.150:41144) 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e259c700 1 -- 172.21.15.150:0/2058136726 learned_addr learned my addr 172.21.15.150:0/2058136726 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 0x7fe3e419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e41013e0 0x7fe3e4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/2058136726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41002c0 msgr2=0x7fe3e4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41002c0 0x7fe3e4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 msgr2=0x7fe3e419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 0x7fe3e419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/2058136726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc009d00 con 0x7fe3e41013e0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.845+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e41013e0 0x7fe3e4198540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009ef0 tx=0x7fe3d800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.846+0000 7fe3d37fe700 1 -- 172.21.15.150:0/2058136726 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80139d0 con 0x7fe3e41013e0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.846+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e4109d70 con 0x7fe3e41013e0 2024-05-10T04:41:24.848 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.846+0000 7fe3d37fe700 1 -- 172.21.15.150:0/2058136726 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d800c7c0 con 0x7fe3e41013e0 2024-05-10T04:41:24.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.846+0000 7fe3d37fe700 1 -- 172.21.15.150:0/2058136726 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d801b860 con 0x7fe3e41013e0 2024-05-10T04:41:24.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.846+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e410a280 con 0x7fe3e41013e0 2024-05-10T04:41:24.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.849+0000 7fe3d37fe700 1 -- 172.21.15.150:0/2058136726 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe3d800c930 con 0x7fe3e41013e0 2024-05-10T04:41:24.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.850+0000 7fe3d37fe700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3cc0a11c0 0x7fe3cc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.850+0000 7fe3d37fe700 1 -- 172.21.15.150:0/2058136726 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe3d80d0e60 con 0x7fe3e41013e0 2024-05-10T04:41:24.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 Processor -- start 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 -- start start 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1e6d0 0x563388f24fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388e3b150 con 0x563388f1dd80 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388e38b10 con 0x563388f1e6d0 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.851+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388e34400 con 0x563388f1ceb0 2024-05-10T04:41:24.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e259c700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3cc0a11c0 0x7fe3cc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1e6d0 0x563388f24fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e1d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1e6d0 0x563388f24fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41148/0 (socket says 172.21.15.150:41148) 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e2d9d700 1 -- 172.21.15.150:0/2222092820 learned_addr learned my addr 172.21.15.150:0/2222092820 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e259c700 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/2222092820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 msgr2=0x563388f1d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/2222092820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1e6d0 msgr2=0x563388f24fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1e6d0 0x563388f24fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/2222092820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563388cb53b0 con 0x563388f1dd80 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e259c700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3cc0a11c0 0x7fe3cc0a36d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4013570 tx=0x7fe3d4013930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1e6d0 0x563388f24fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.852+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f1e190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e9c0 tx=0x7fe3d809d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59baabf7f5f70053 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.853+0000 7fe3d0ff9700 1 -- 172.21.15.150:0/2222092820 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d809df10 con 0x563388f1dd80 2024-05-10T04:41:24.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.853+0000 7fe3d0ff9700 1 -- 172.21.15.150:0/2222092820 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d80a1d40 con 0x563388f1dd80 2024-05-10T04:41:24.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.853+0000 7fe3d0ff9700 1 -- 172.21.15.150:0/2222092820 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d809d640 con 0x563388f1dd80 2024-05-10T04:41:24.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/2222092820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 msgr2=0x563388f1e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.862 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f1e190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e9c0 tx=0x7fe3d809d870 comp rx=0 tx=0).stop 2024-05-10T04:41:24.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/2222092820 shutdown_connections 2024-05-10T04:41:24.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1e6d0 0x563388f24fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/2222092820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f1e190 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/2222092820 >> 172.21.15.150:0/2222092820 conn(0x563388e76a60 msgr2=0x563388f0efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.863 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/2222092820 shutdown_connections 2024-05-10T04:41:24.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.854+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/2222092820 wait complete. 2024-05-10T04:41:24.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 -- start start 2024-05-10T04:41:24.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 Processor -- start 2024-05-10T04:41:24.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1ceb0 0x563388f26f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1dd80 0x563388f274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1e6d0 0x563388f2e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.864 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388dd0560 con 0x563388f1ceb0 2024-05-10T04:41:24.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388dd0200 con 0x563388f1dd80 2024-05-10T04:41:24.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388cb76a0 con 0x563388f1e6d0 2024-05-10T04:41:24.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3f91fbb80 10 client.0 _ll_register_callbacks cb 0x563388dd9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.855+0000 7fe3e1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1dd80 0x563388f274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1dd80 0x563388f274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41152/0 (socket says 172.21.15.150:41152) 2024-05-10T04:41:24.865 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/1914935722 learned_addr learned my addr 172.21.15.150:0/1914935722 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e259c700 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1ceb0 0x563388f26f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1e6d0 0x563388f2e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/1914935722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1e6d0 msgr2=0x563388f2e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1e6d0 0x563388f2e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/1914935722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1ceb0 msgr2=0x563388f26f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1ceb0 0x563388f26f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/1914935722 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4068020 con 0x563388f1dd80 2024-05-10T04:41:24.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1dd80 0x563388f274c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80ce930 tx=0x7fe3d800cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cae30cead54ca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.856+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/1914935722 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80a6100 con 0x563388f1dd80 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563388d8d270 con 0x563388f1dd80 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/1914935722 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe3d809df10 con 0x563388f1dd80 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/1914935722 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d80bc2c0 con 0x563388f1dd80 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 10 client.25880 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe3e410a280 con 0x563388f1dd80 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/1914935722 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563388d8d270 con 0x563388f1dd80 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/1914935722 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3d8099e90 con 0x563388f1dd80 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 10 client.25880 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 20 client.25880 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f15210 0x563388f17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 4 client.25880 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.857+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563388ee3780 con 0x563388f15210 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.858+0000 7fe3e259c700 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f15210 0x563388f17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.858+0000 7fe3c9ffb700 10 client.25880 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.858+0000 7fe3e259c700 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f15210 0x563388f17630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e937eacc07540454 server_cookie=82fe88ee1da7b0b5 in_seq=0 out_seq=0 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/1914935722 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563388ee3780 con 0x563388f15210 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3c9ffb700 10 client.25880 handle_command_reply: tid=0 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 client.25880 shutdown 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1dd80 msgr2=0x563388f274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1dd80 0x563388f274c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80ce930 tx=0x7fe3d800cee0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 shutdown_connections 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f15210 0x563388f17630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1e6d0 0x563388f2e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1ceb0 0x563388f26f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3c9ffb700 0 client.25880 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:24.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1914935722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1dd80 0x563388f274c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 >> 172.21.15.150:0/1914935722 conn(0x563388e76a60 msgr2=0x563388f0efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 shutdown_connections 2024-05-10T04:41:24.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1914935722 wait complete. 2024-05-10T04:41:24.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.859+0000 7fe3f91fbb80 20 client.25880 trim_cache size 0 max 16384 2024-05-10T04:41:24.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 Processor -- start 2024-05-10T04:41:24.874 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 -- start start 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f15210 0x563388f17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f26f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388f1eb20 con 0x563388f1dd80 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388f0ec90 con 0x563388f15210 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.873+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388f107a0 con 0x563388f1ceb0 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f15210 0x563388f17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f15210 0x563388f17630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41162/0 (socket says 172.21.15.150:41162) 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e259c700 1 -- 172.21.15.150:0/1367759274 learned_addr learned my addr 172.21.15.150:0/1367759274 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.876 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f26f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/1367759274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f15210 msgr2=0x563388f17630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f15210 0x563388f17630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/1367759274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 msgr2=0x563388f26f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f26f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/1367759274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563388c61b60 con 0x563388f1ceb0 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e259c700 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f15210 0x563388f17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f26f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.874+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809dea0 tx=0x7fe3d809c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bafd09cb892ac451 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.875+0000 7fe3d17fa700 1 -- 172.21.15.150:0/1367759274 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80a6100 con 0x563388f1ceb0 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.875+0000 7fe3d17fa700 1 -- 172.21.15.150:0/1367759274 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d809d870 con 0x563388f1ceb0 2024-05-10T04:41:24.877 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.875+0000 7fe3d17fa700 1 -- 172.21.15.150:0/1367759274 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d809b860 con 0x563388f1ceb0 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.875+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1367759274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 msgr2=0x563388f1d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.875+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809dea0 tx=0x7fe3d809c4c0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.875+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1367759274 shutdown_connections 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f26f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f15210 0x563388f17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/1367759274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f1ceb0 0x563388f1d510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1367759274 >> 172.21.15.150:0/1367759274 conn(0x563388e76a60 msgr2=0x563388f0fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1367759274 shutdown_connections 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/1367759274 wait complete. 2024-05-10T04:41:24.880 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 -- start start 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 Processor -- start 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f15210 0x563388f1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1ceb0 0x563388f8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f8f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388f8fd20 con 0x563388f1dd80 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388f8fe90 con 0x563388f1ceb0 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563388f90050 con 0x563388f15210 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.876+0000 7fe3f91fbb80 10 client.0 _ll_register_callbacks cb 0x563388dd9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1ceb0 0x563388f8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f8f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1ceb0 0x563388f8d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41176/0 (socket says 172.21.15.150:41176) 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f8f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45222/0 (socket says 172.21.15.150:45222) 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e1d9b700 1 -- 172.21.15.150:0/3493478033 learned_addr learned my addr 172.21.15.150:0/3493478033 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 -- 172.21.15.150:0/3493478033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f15210 msgr2=0x563388f1c590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:41:24.881 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f15210 0x563388f1c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 -- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1ceb0 msgr2=0x563388f8d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1ceb0 0x563388f8d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 -- 172.21.15.150:0/3493478033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d4025040 con 0x563388f1dd80 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e1d9b700 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1ceb0 0x563388f8d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3e2d9d700 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f8f6e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00b780 tx=0x7fe3dc00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22c200d752a9e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/3493478033 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc00e040 con 0x563388f1dd80 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/3493478033 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe3dc003e10 con 0x563388f1dd80 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.877+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3d80a1d40 con 0x563388f1dd80 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/3493478033 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3dc026480 con 0x563388f1dd80 2024-05-10T04:41:24.882 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 10 client.16257 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe3e410a280 con 0x563388f1dd80 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/3493478033 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d80a1d40 con 0x563388f1dd80 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/3493478033 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3dc00b0d0 con 0x563388f1dd80 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 10 client.16257 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 20 client.16257 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f1e6d0 0x563388f310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 4 client.16257 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:24.883 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.878+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563388e2cec0 con 0x563388f1e6d0 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.879+0000 7fe3e259c700 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f1e6d0 0x563388f310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3c9ffb700 10 client.16257 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3e259c700 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f1e6d0 0x563388f310b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6976bb5ebe35393d server_cookie=a561b6324e7cf748 in_seq=0 out_seq=0 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3c9ffb700 1 -- 172.21.15.150:0/3493478033 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563388e2cec0 con 0x563388f1e6d0 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3c9ffb700 10 client.16257 handle_command_reply: tid=0 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 client.16257 shutdown 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 msgr2=0x563388f8f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f8f6e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00b780 tx=0x7fe3dc00bce0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 shutdown_connections 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x563388f1e6d0 0x563388f310b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563388f1dd80 0x563388f8f6e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.885 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563388f15210 0x563388f1c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 --2- 172.21.15.150:0/3493478033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563388f1ceb0 0x563388f8d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 >> 172.21.15.150:0/3493478033 conn(0x563388e76a60 msgr2=0x563388f1d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3c9ffb700 0 client.16257 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:24.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 shutdown_connections 2024-05-10T04:41:24.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.880+0000 7fe3f91fbb80 1 -- 172.21.15.150:0/3493478033 wait complete. 2024-05-10T04:41:24.886 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.881+0000 7fe3f91fbb80 20 client.16257 trim_cache size 0 max 16384 2024-05-10T04:41:24.886 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3cc0a11c0 msgr2=0x7fe3cc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3cc0a11c0 0x7fe3cc0a36d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4013570 tx=0x7fe3d4013930 comp rx=0 tx=0).stop 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e41013e0 msgr2=0x7fe3e4198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e41013e0 0x7fe3e4198540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009ef0 tx=0x7fe3d800ecd0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 shutdown_connections 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3e4101d80 0x7fe3e419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41002c0 0x7fe3e4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe3cc0a11c0 0x7fe3cc0a36d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.887 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 --2- 172.21.15.150:0/2058136726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e41013e0 0x7fe3e4198540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 >> 172.21.15.150:0/2058136726 conn(0x7fe3e40fb8f0 msgr2=0x7fe3e4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 shutdown_connections 2024-05-10T04:41:24.888 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:24.883+0000 7fe3e8eb7700 1 -- 172.21.15.150:0/2058136726 wait complete. 2024-05-10T04:41:24.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:24 smithi181 ceph-mon[47314]: pgmap v326: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:24.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:24 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1953013491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:41:25.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:25 smithi161 ceph-mon[46912]: pgmap v327: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:25.899 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:41:25.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:41:25.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:25 smithi181 ceph-mon[47314]: pgmap v327: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:25.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:25 smithi150 ceph-mon[42404]: pgmap v327: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:26.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 -- 172.21.15.150:0/1277067518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278101c00 msgr2=0x7fe278102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/1277067518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278101c00 0x7fe278102010 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 -- 172.21.15.150:0/1277067518 shutdown_connections 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/1277067518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2781025a0 0x7fe278108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/1277067518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe278100ae0 0x7fe278100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/1277067518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278101c00 0x7fe278102010 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 -- 172.21.15.150:0/1277067518 >> 172.21.15.150:0/1277067518 conn(0x7fe2780fc0f0 msgr2=0x7fe2780fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 -- 172.21.15.150:0/1277067518 shutdown_connections 2024-05-10T04:41:26.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.052+0000 7fe27cfe1700 1 -- 172.21.15.150:0/1277067518 wait complete. 2024-05-10T04:41:26.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.053+0000 7fe27cfe1700 1 Processor -- start 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.053+0000 7fe27cfe1700 1 -- start start 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27cfe1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278100ae0 0x7fe278195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27cfe1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe278101c00 0x7fe2781962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27cfe1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2781025a0 0x7fe27819d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27cfe1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27819d870 con 0x7fe278100ae0 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27cfe1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27819d9e0 con 0x7fe2781025a0 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27cfe1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27819dba0 con 0x7fe278101c00 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278100ae0 0x7fe278195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278100ae0 0x7fe278195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45234/0 (socket says 172.21.15.150:45234) 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 -- 172.21.15.150:0/2051649995 learned_addr learned my addr 172.21.15.150:0/2051649995 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe275d9b700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe278101c00 0x7fe2781962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe276d9d700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2781025a0 0x7fe27819d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 -- 172.21.15.150:0/2051649995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe278101c00 msgr2=0x7fe2781962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe278101c00 0x7fe2781962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 -- 172.21.15.150:0/2051649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2781025a0 msgr2=0x7fe27819d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2781025a0 0x7fe27819d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe27659c700 1 -- 172.21.15.150:0/2051649995 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c0095f0 con 0x7fe278100ae0 2024-05-10T04:41:26.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.054+0000 7fe276d9d700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2781025a0 0x7fe27819d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:26.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.055+0000 7fe27659c700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278100ae0 0x7fe278195d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe26800d950 tx=0x7fe26800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.055+0000 7fe2677fe700 1 -- 172.21.15.150:0/2051649995 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268004220 con 0x7fe278100ae0 2024-05-10T04:41:26.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.055+0000 7fe2677fe700 1 -- 172.21.15.150:0/2051649995 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe268004380 con 0x7fe278100ae0 2024-05-10T04:41:26.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.055+0000 7fe2677fe700 1 -- 172.21.15.150:0/2051649995 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268003bf0 con 0x7fe278100ae0 2024-05-10T04:41:26.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.056+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27819de80 con 0x7fe278100ae0 2024-05-10T04:41:26.057 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.056+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27819e1f0 con 0x7fe278100ae0 2024-05-10T04:41:26.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.061+0000 7fe28d325b80 1 Processor -- start 2024-05-10T04:41:26.061 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.061+0000 7fe28d325b80 1 -- start start 2024-05-10T04:41:26.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.062+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.062+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d35f0 0x55ccee5d3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.062+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d3f40 0x55ccee5dec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.062+0000 7fe28d325b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee4f91d0 con 0x55ccee5d35f0 2024-05-10T04:41:26.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.062+0000 7fe28d325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee4f6b90 con 0x55ccee5d2720 2024-05-10T04:41:26.062 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.062+0000 7fe28d325b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee4f2480 con 0x55ccee5d3f40 2024-05-10T04:41:26.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d3f40 0x55ccee5dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe276d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d3f40 0x55ccee5dec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:37926/0 (socket says 172.21.15.150:37926) 2024-05-10T04:41:26.063 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 -- 172.21.15.150:0/2108208396 learned_addr learned my addr 172.21.15.150:0/2108208396 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 -- 172.21.15.150:0/2108208396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 msgr2=0x55ccee5d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 --2- 172.21.15.150:0/2108208396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 -- 172.21.15.150:0/2108208396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d35f0 msgr2=0x55ccee5d3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 --2- 172.21.15.150:0/2108208396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d35f0 0x55ccee5d3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 -- 172.21.15.150:0/2108208396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccee373430 con 0x55ccee5d3f40 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.063+0000 7fe275d9b700 1 --2- 172.21.15.150:0/2108208396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d3f40 0x55ccee5dec50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ea1c7f2c454ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.064+0000 7fe264ff9700 1 -- 172.21.15.150:0/2108208396 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c003680 con 0x55ccee5d3f40 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.064+0000 7fe264ff9700 1 -- 172.21.15.150:0/2108208396 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe26c01e040 con 0x55ccee5d3f40 2024-05-10T04:41:26.064 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.064+0000 7fe264ff9700 1 -- 172.21.15.150:0/2108208396 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00fe70 con 0x55ccee5d3f40 2024-05-10T04:41:26.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.064+0000 7fe2677fe700 1 -- 172.21.15.150:0/2051649995 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe268003d50 con 0x7fe278100ae0 2024-05-10T04:41:26.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe2677fe700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe2600a1370 0x7fe2600a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe28d325b80 1 -- 172.21.15.150:0/2108208396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d3f40 msgr2=0x55ccee5dec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2108208396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d3f40 0x55ccee5dec50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c005fb0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe2677fe700 1 -- 172.21.15.150:0/2051649995 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe2680ce7e0 con 0x7fe278100ae0 2024-05-10T04:41:26.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe28d325b80 1 -- 172.21.15.150:0/2108208396 shutdown_connections 2024-05-10T04:41:26.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2108208396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d3f40 0x55ccee5dec50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2108208396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2108208396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d35f0 0x55ccee5d3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe28d325b80 1 -- 172.21.15.150:0/2108208396 >> 172.21.15.150:0/2108208396 conn(0x55ccee534ae0 msgr2=0x55ccee5cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.065+0000 7fe276d9d700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe2600a1370 0x7fe2600a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.066+0000 7fe28d325b80 1 -- 172.21.15.150:0/2108208396 shutdown_connections 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.066+0000 7fe28d325b80 1 -- 172.21.15.150:0/2108208396 wait complete. 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.066+0000 7fe28d325b80 1 -- start start 2024-05-10T04:41:26.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.066+0000 7fe28d325b80 1 Processor -- start 2024-05-10T04:41:26.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee5dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe28d325b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee48e5e0 con 0x55ccee5d2720 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe28d325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee48e280 con 0x55ccee5d3f40 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe28d325b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee375720 con 0x55ccee5d35f0 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe28d325b80 10 client.0 _ll_register_callbacks cb 0x55ccee497bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe276d9d700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe2600a1370 0x7fe2600a3880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55ccee5d2400 tx=0x7fe270009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.067+0000 7fe276d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5ddb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41216/0 (socket says 172.21.15.150:41216) 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 -- 172.21.15.150:0/1579215868 learned_addr learned my addr 172.21.15.150:0/1579215868 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 -- 172.21.15.150:0/1579215868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 msgr2=0x55ccee5dcbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5dcbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 -- 172.21.15.150:0/1579215868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 msgr2=0x55ccee5dc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.070 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee5dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 -- 172.21.15.150:0/1579215868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2781098e0 con 0x55ccee5d3f40 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe276d9d700 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5ddb00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe270018e70 tx=0x7fe270004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a8f3bbaf9219b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe25dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe25dffb700 1 -- 172.21.15.150:0/1579215868 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270016cc0 con 0x55ccee5d3f40 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe25dffb700 1 -- 172.21.15.150:0/1579215868 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe270007e40 con 0x55ccee5d3f40 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.068+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccee44b2f0 con 0x55ccee5d3f40 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe25dffb700 1 -- 172.21.15.150:0/1579215868 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe270016e20 con 0x55ccee5d3f40 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 10 client.25898 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ccee31fbe0 con 0x55ccee5d3f40 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe25dffb700 1 -- 172.21.15.150:0/1579215868 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ccee44b2f0 con 0x55ccee5d3f40 2024-05-10T04:41:26.071 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe25dffb700 1 -- 172.21.15.150:0/1579215868 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe270015450 con 0x55ccee5d3f40 2024-05-10T04:41:26.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:26.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:26.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 20 client.25898 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:26.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee601c20 0x55ccee604070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 4 client.25898 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:26.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccee5a1800 con 0x55ccee601c20 2024-05-10T04:41:26.072 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.069+0000 7fe27659c700 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee601c20 0x55ccee604070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.070+0000 7fe25dffb700 10 client.25898 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.070+0000 7fe27659c700 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee601c20 0x55ccee604070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7f6f47f2c861b2 server_cookie=93d05570d5f3e8d4 in_seq=0 out_seq=0 2024-05-10T04:41:26.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe25dffb700 1 -- 172.21.15.150:0/1579215868 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ccee5a1800 con 0x55ccee601c20 2024-05-10T04:41:26.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe25dffb700 10 client.25898 handle_command_reply: tid=0 2024-05-10T04:41:26.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 client.25898 shutdown 2024-05-10T04:41:26.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 msgr2=0x55ccee5ddb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.073 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5ddb00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe270018e70 tx=0x7fe270004ad0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 shutdown_connections 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee601c20 0x55ccee604070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5ddb00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee5dc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe25dffb700 0 client.25898 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 --2- 172.21.15.150:0/1579215868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 >> 172.21.15.150:0/1579215868 conn(0x55ccee534ae0 msgr2=0x55ccee5cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 shutdown_connections 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 1 -- 172.21.15.150:0/1579215868 wait complete. 2024-05-10T04:41:26.074 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.071+0000 7fe28d325b80 20 client.25898 trim_cache size 0 max 16384 2024-05-10T04:41:26.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.104+0000 7fe28d325b80 1 Processor -- start 2024-05-10T04:41:26.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe28d325b80 1 -- start start 2024-05-10T04:41:26.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee6089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe28d325b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee5ff050 con 0x55ccee5d2720 2024-05-10T04:41:26.105 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe28d325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee5d4350 con 0x55ccee5d3f40 2024-05-10T04:41:26.106 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe28d325b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee5dd580 con 0x55ccee5d35f0 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe27659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee6089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe276d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe27659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee6089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45260/0 (socket says 172.21.15.150:45260) 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.105+0000 7fe27659c700 1 -- 172.21.15.150:0/2003129530 learned_addr learned my addr 172.21.15.150:0/2003129530 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe276d9d700 1 -- 172.21.15.150:0/2003129530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 msgr2=0x55ccee5dc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe276d9d700 1 --2- 172.21.15.150:0/2003129530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe276d9d700 1 -- 172.21.15.150:0/2003129530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 msgr2=0x55ccee6089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe276d9d700 1 --2- 172.21.15.150:0/2003129530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee6089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe276d9d700 1 -- 172.21.15.150:0/2003129530 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccee5c94d0 con 0x55ccee5d3f40 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe276d9d700 1 --2- 172.21.15.150:0/2003129530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5dcbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe270018e40 tx=0x7fe270009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33fdc245e1ec30c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe2657fa700 1 -- 172.21.15.150:0/2003129530 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270039070 con 0x55ccee5d3f40 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe2657fa700 1 -- 172.21.15.150:0/2003129530 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27001c070 con 0x55ccee5d3f40 2024-05-10T04:41:26.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.106+0000 7fe2657fa700 1 -- 172.21.15.150:0/2003129530 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270015070 con 0x55ccee5d3f40 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.107+0000 7fe28d325b80 1 -- 172.21.15.150:0/2003129530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 msgr2=0x55ccee5dcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.107+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2003129530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5dcbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe270018e40 tx=0x7fe270009b90 comp rx=0 tx=0).stop 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.107+0000 7fe28d325b80 1 -- 172.21.15.150:0/2003129530 shutdown_connections 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.107+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2003129530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d3f40 0x55ccee5dcbd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.107+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2003129530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d2720 0x55ccee6089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.107+0000 7fe28d325b80 1 --2- 172.21.15.150:0/2003129530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5dc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.107+0000 7fe28d325b80 1 -- 172.21.15.150:0/2003129530 >> 172.21.15.150:0/2003129530 conn(0x55ccee534ae0 msgr2=0x55ccee5cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.108+0000 7fe28d325b80 1 -- 172.21.15.150:0/2003129530 shutdown_connections 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.108+0000 7fe28d325b80 1 -- 172.21.15.150:0/2003129530 wait complete. 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.108+0000 7fe28d325b80 1 -- start start 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.108+0000 7fe28d325b80 1 Processor -- start 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.108+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.109 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.108+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5eb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.108+0000 7fe28d325b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d3f40 0x55ccee5ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.109+0000 7fe28d325b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee5ec3c0 con 0x55ccee5d3f40 2024-05-10T04:41:26.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.109+0000 7fe28d325b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee5ec530 con 0x55ccee5d2720 2024-05-10T04:41:26.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.109+0000 7fe28d325b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccee5ec6f0 con 0x55ccee5d35f0 2024-05-10T04:41:26.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.109+0000 7fe28d325b80 10 client.0 _ll_register_callbacks cb 0x55ccee497bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:26.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.109+0000 7fe276d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d3f40 0x55ccee5ebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.109+0000 7fe276d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d3f40 0x55ccee5ebe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45270/0 (socket says 172.21.15.150:45270) 2024-05-10T04:41:26.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.109+0000 7fe276d9d700 1 -- 172.21.15.150:0/3537376972 learned_addr learned my addr 172.21.15.150:0/3537376972 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe27659c700 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe276d9d700 1 -- 172.21.15.150:0/3537376972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 msgr2=0x55ccee5eb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe276d9d700 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe276d9d700 1 -- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 msgr2=0x55ccee5ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe276d9d700 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe276d9d700 1 -- 172.21.15.150:0/3537376972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccee31fbe0 con 0x55ccee5d3f40 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe276d9d700 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d3f40 0x55ccee5ebe40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe270004d00 tx=0x7fe270004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80cc5687cb8f7cf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe25dffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe25dffb700 1 -- 172.21.15.150:0/3537376972 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270039070 con 0x55ccee5d3f40 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe26c01e040 con 0x55ccee5d3f40 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.110+0000 7fe25dffb700 1 -- 172.21.15.150:0/3537376972 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe270015ca0 con 0x55ccee5d3f40 2024-05-10T04:41:26.113 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe25dffb700 1 -- 172.21.15.150:0/3537376972 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe270015e60 con 0x55ccee5d3f40 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 10 client.16269 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ccee4b85e0 con 0x55ccee5d3f40 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe25dffb700 1 -- 172.21.15.150:0/3537376972 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe26c01e040 con 0x55ccee5d3f40 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe25dffb700 1 -- 172.21.15.150:0/3537376972 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe27003eb10 con 0x55ccee5d3f40 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 20 client.16269 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee659510 0x55ccee65b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 4 client.16269 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ccee4eaf40 con 0x55ccee659510 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.111+0000 7fe27659c700 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee659510 0x55ccee65b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.112+0000 7fe25dffb700 10 client.16269 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.112+0000 7fe27659c700 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee659510 0x55ccee65b930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84314a3d6f751ba server_cookie=bafa9147c4c8a3cc in_seq=0 out_seq=0 2024-05-10T04:41:26.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.113+0000 7fe25dffb700 1 -- 172.21.15.150:0/3537376972 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ccee4eaf40 con 0x55ccee659510 2024-05-10T04:41:26.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.113+0000 7fe25dffb700 10 client.16269 handle_command_reply: tid=0 2024-05-10T04:41:26.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.113+0000 7fe28d325b80 1 client.16269 shutdown 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d3f40 msgr2=0x55ccee5ebe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d3f40 0x55ccee5ebe40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe270004d00 tx=0x7fe270004de0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 shutdown_connections 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ccee659510 0x55ccee65b930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ccee5d3f40 0x55ccee5ebe40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ccee5d2720 0x55ccee5ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 --2- 172.21.15.150:0/3537376972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ccee5d35f0 0x55ccee5eb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe25dffb700 0 client.16269 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 >> 172.21.15.150:0/3537376972 conn(0x55ccee534ae0 msgr2=0x55ccee5cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 shutdown_connections 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 1 -- 172.21.15.150:0/3537376972 wait complete. 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.114+0000 7fe28d325b80 20 client.16269 trim_cache size 0 max 16384 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "aa2b86e6-7518-43a9-984e-f885b0c7fb30", 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:41:26.117 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe2600a1370 msgr2=0x7fe2600a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe2600a1370 0x7fe2600a3880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55ccee5d2400 tx=0x7fe270009040 comp rx=0 tx=0).stop 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278100ae0 msgr2=0x7fe278195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278100ae0 0x7fe278195d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe26800d950 tx=0x7fe26800dc60 comp rx=0 tx=0).stop 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 shutdown_connections 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2781025a0 0x7fe27819d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe278100ae0 0x7fe278195d90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe2600a1370 0x7fe2600a3880 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 --2- 172.21.15.150:0/2051649995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe278101c00 0x7fe2781962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 >> 172.21.15.150:0/2051649995 conn(0x7fe2780fc0f0 msgr2=0x7fe278104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 shutdown_connections 2024-05-10T04:41:26.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.115+0000 7fe27cfe1700 1 -- 172.21.15.150:0/2051649995 wait complete. 2024-05-10T04:41:26.133 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 -- 172.21.15.150:0/4105404907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 msgr2=0x7f2314100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 --2- 172.21.15.150:0/4105404907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 0x7f2314100ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23080099c0 tx=0x7f2308009cd0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 -- 172.21.15.150:0/4105404907 shutdown_connections 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 --2- 172.21.15.150:0/4105404907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2314102580 0x7f2314108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 --2- 172.21.15.150:0/4105404907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 0x7f2314100ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 --2- 172.21.15.150:0/4105404907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 0x7f2314101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 -- 172.21.15.150:0/4105404907 >> 172.21.15.150:0/4105404907 conn(0x7f23140fc0d0 msgr2=0x7f23140fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 -- 172.21.15.150:0/4105404907 shutdown_connections 2024-05-10T04:41:26.362 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.361+0000 7f231b1a2700 1 -- 172.21.15.150:0/4105404907 wait complete. 2024-05-10T04:41:26.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.362+0000 7f231b1a2700 1 Processor -- start 2024-05-10T04:41:26.363 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.362+0000 7f231b1a2700 1 -- start start 2024-05-10T04:41:26.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f231b1a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 0x7f231419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f231b1a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 0x7f231419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.364 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f231b1a2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2314102580 0x7f23141a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f231b1a2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a1be0 con 0x7f2314101be0 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f231b1a2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a1d50 con 0x7f2314102580 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f231b1a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141a1f10 con 0x7f2314100ac0 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 0x7f231419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 0x7f231419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45288/0 (socket says 172.21.15.150:45288) 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 -- 172.21.15.150:0/67394132 learned_addr learned my addr 172.21.15.150:0/67394132 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2318f3e700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 0x7f231419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f231973f700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2314102580 0x7f23141a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 -- 172.21.15.150:0/67394132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 msgr2=0x7f231419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 0x7f231419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 -- 172.21.15.150:0/67394132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2314102580 msgr2=0x7f23141a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2314102580 0x7f23141a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 -- 172.21.15.150:0/67394132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080095f0 con 0x7f2314101be0 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.363+0000 7f2313fff700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 0x7f231419a640 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2300009a20 tx=0x7f2300009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.365 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.364+0000 7f2311ffb700 1 -- 172.21.15.150:0/67394132 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300012070 con 0x7f2314101be0 2024-05-10T04:41:26.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.364+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23141a21f0 con 0x7f2314101be0 2024-05-10T04:41:26.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.364+0000 7f2311ffb700 1 -- 172.21.15.150:0/67394132 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f230000d650 con 0x7f2314101be0 2024-05-10T04:41:26.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.364+0000 7f2311ffb700 1 -- 172.21.15.150:0/67394132 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300016c50 con 0x7f2314101be0 2024-05-10T04:41:26.366 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.364+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23141a2750 con 0x7f2314101be0 2024-05-10T04:41:26.368 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.369+0000 7f2311ffb700 1 -- 172.21.15.150:0/67394132 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2300016410 con 0x7f2314101be0 2024-05-10T04:41:26.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.369+0000 7f2311ffb700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23040a1240 0x7f23040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.369 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.369+0000 7f2311ffb700 1 -- 172.21.15.150:0/67394132 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f23000cc0a0 con 0x7f2314101be0 2024-05-10T04:41:26.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.370+0000 7f2318f3e700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23040a1240 0x7f23040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.370 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.370+0000 7f232b4e6b80 1 Processor -- start 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.370+0000 7f232b4e6b80 1 -- start start 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f546a0 0x55cca1f54af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f55570 0x55cca1f55980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2318f3e700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23040a1240 0x7f23040a3750 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f5c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1e7b150 con 0x55cca1f55ec0 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1e78b10 con 0x55cca1f55570 2024-05-10T04:41:26.372 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1e74400 con 0x55cca1f546a0 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f55570 0x55cca1f55980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f55570 0x55cca1f55980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41240/0 (socket says 172.21.15.150:41240) 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 -- 172.21.15.150:0/4228824888 learned_addr learned my addr 172.21.15.150:0/4228824888 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2318f3e700 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f546a0 0x55cca1f54af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 -- 172.21.15.150:0/4228824888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f546a0 msgr2=0x55cca1f54af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f546a0 0x55cca1f54af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 -- 172.21.15.150:0/4228824888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 msgr2=0x55cca1f5c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f5c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 -- 172.21.15.150:0/4228824888 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cca1cf53b0 con 0x55cca1f55570 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.371+0000 7f2313fff700 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f55570 0x55cca1f55980 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23000955f0 tx=0x7f2300098d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2737ae1ad45adeb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.372+0000 7f22feffd700 1 -- 172.21.15.150:0/4228824888 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230009cf20 con 0x55cca1f55570 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.372+0000 7f22feffd700 1 -- 172.21.15.150:0/4228824888 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23000a1570 con 0x55cca1f55570 2024-05-10T04:41:26.373 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.372+0000 7f22feffd700 1 -- 172.21.15.150:0/4228824888 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000a10e0 con 0x55cca1f55570 2024-05-10T04:41:26.376 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 -- 172.21.15.150:0/4228824888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f55570 msgr2=0x55cca1f55980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f55570 0x55cca1f55980 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23000955f0 tx=0x7f2300098d60 comp rx=0 tx=0).stop 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 -- 172.21.15.150:0/4228824888 shutdown_connections 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f5c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f546a0 0x55cca1f54af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/4228824888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f55570 0x55cca1f55980 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 -- 172.21.15.150:0/4228824888 >> 172.21.15.150:0/4228824888 conn(0x55cca1eb6a60 msgr2=0x55cca1f4efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 -- 172.21.15.150:0/4228824888 shutdown_connections 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.373+0000 7f232b4e6b80 1 -- 172.21.15.150:0/4228824888 wait complete. 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 -- start start 2024-05-10T04:41:26.377 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 Processor -- start 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f66e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1e10560 con 0x55cca1f55570 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1e10200 con 0x55cca1f546a0 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1cf76a0 con 0x55cca1f55ec0 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.374+0000 7f232b4e6b80 10 client.0 _ll_register_callbacks cb 0x55cca1e19b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2318f3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f66e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2313fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2318f3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f66e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:41252/0 (socket says 172.21.15.150:41252) 2024-05-10T04:41:26.378 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2318f3e700 1 -- 172.21.15.150:0/3585300122 learned_addr learned my addr 172.21.15.150:0/3585300122 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f231973f700 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2313fff700 1 -- 172.21.15.150:0/3585300122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 msgr2=0x55cca1f6e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2313fff700 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2313fff700 1 -- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 msgr2=0x55cca1f66e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2313fff700 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f66e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2313fff700 1 -- 172.21.15.150:0/3585300122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23141a2750 con 0x55cca1f55570 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2318f3e700 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f2313fff700 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f673c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2300098df0 tx=0x7f2300095560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2353439ad1aa41b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.379 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.375+0000 7f22f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f22f3fff700 1 -- 172.21.15.150:0/3585300122 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000a12e0 con 0x55cca1f55570 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f22f3fff700 1 -- 172.21.15.150:0/3585300122 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f230009cf20 con 0x55cca1f55570 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cca1dcd270 con 0x55cca1f55570 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f22f3fff700 1 -- 172.21.15.150:0/3585300122 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2300094ee0 con 0x55cca1f55570 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 10 client.16278 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cca1ca1b60 con 0x55cca1f55570 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f22f3fff700 1 -- 172.21.15.150:0/3585300122 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cca1dcd270 con 0x55cca1f55570 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f22f3fff700 1 -- 172.21.15.150:0/3585300122 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2300099a80 con 0x55cca1f55570 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 20 client.16278 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:26.380 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1f727a0 0x55cca1f74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 4 client.16278 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:26.381 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.376+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cca1f23780 con 0x55cca1f727a0 2024-05-10T04:41:26.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.377+0000 7f2318f3e700 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1f727a0 0x55cca1f74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.377+0000 7f22f3fff700 10 client.16278 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.377+0000 7f2318f3e700 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1f727a0 0x55cca1f74bc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac35c639cbfd76f4 server_cookie=42a0751d5e50fc52 in_seq=0 out_seq=0 2024-05-10T04:41:26.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f22f3fff700 1 -- 172.21.15.150:0/3585300122 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cca1f23780 con 0x55cca1f727a0 2024-05-10T04:41:26.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f22f3fff700 10 client.16278 handle_command_reply: tid=0 2024-05-10T04:41:26.382 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 client.16278 shutdown 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 msgr2=0x55cca1f673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f673c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2300098df0 tx=0x7f2300095560 comp rx=0 tx=0).stop 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 shutdown_connections 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1f727a0 0x55cca1f74bc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f22f3fff700 0 client.16278 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/3585300122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f673c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 >> 172.21.15.150:0/3585300122 conn(0x55cca1eb6a60 msgr2=0x55cca1f4efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 shutdown_connections 2024-05-10T04:41:26.383 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.378+0000 7f232b4e6b80 1 -- 172.21.15.150:0/3585300122 wait complete. 2024-05-10T04:41:26.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.379+0000 7f232b4e6b80 20 client.16278 trim_cache size 0 max 16384 2024-05-10T04:41:26.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.400+0000 7f232b4e6b80 1 Processor -- start 2024-05-10T04:41:26.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.400+0000 7f232b4e6b80 1 -- start start 2024-05-10T04:41:26.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f55980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1f49810 con 0x55cca1f55570 2024-05-10T04:41:26.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1f66a50 con 0x55cca1f546a0 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1f66c10 con 0x55cca1f55ec0 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:38016/0 (socket says 172.21.15.150:38016) 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 -- 172.21.15.150:0/2826238819 learned_addr learned my addr 172.21.15.150:0/2826238819 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f2318f3e700 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f55980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 -- 172.21.15.150:0/2826238819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 msgr2=0x55cca1f55980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f55980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 -- 172.21.15.150:0/2826238819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 msgr2=0x55cca1f74bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f231973f700 1 -- 172.21.15.150:0/2826238819 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cca1e3a560 con 0x55cca1f55ec0 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.401+0000 7f2318f3e700 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f55980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.402+0000 7f231973f700 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f230c00a790 tx=0x7f230c00aaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=941cb85202a1a12c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.402+0000 7f22ff7fe700 1 -- 172.21.15.150:0/2826238819 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c00f040 con 0x55cca1f55ec0 2024-05-10T04:41:26.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.402+0000 7f22ff7fe700 1 -- 172.21.15.150:0/2826238819 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f230c010040 con 0x55cca1f55ec0 2024-05-10T04:41:26.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.402+0000 7f22ff7fe700 1 -- 172.21.15.150:0/2826238819 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c0048f0 con 0x55cca1f55ec0 2024-05-10T04:41:26.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2826238819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 msgr2=0x55cca1f6e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f230c00a790 tx=0x7f230c00aaa0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2826238819 shutdown_connections 2024-05-10T04:41:26.406 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55ec0 0x55cca1f6e2e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f55980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2826238819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55570 0x55cca1f74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2826238819 >> 172.21.15.150:0/2826238819 conn(0x55cca1eb6a60 msgr2=0x55cca1f4b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2826238819 shutdown_connections 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2826238819 wait complete. 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 -- start start 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.403+0000 7f232b4e6b80 1 Processor -- start 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.404+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f6c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.404+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55570 0x55cca1f6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.404+0000 7f232b4e6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f6d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.404+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1f6d990 con 0x55cca1f55ec0 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.404+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1f6db00 con 0x55cca1f546a0 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.404+0000 7f232b4e6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca1f6dcc0 con 0x55cca1f55570 2024-05-10T04:41:26.407 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.404+0000 7f232b4e6b80 10 client.0 _ll_register_callbacks cb 0x55cca1e19b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f6d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f6d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45306/0 (socket says 172.21.15.150:45306) 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 -- 172.21.15.150:0/2451068989 learned_addr learned my addr 172.21.15.150:0/2451068989 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f2313fff700 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55570 0x55cca1f6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f2318f3e700 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f6c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 -- 172.21.15.150:0/2451068989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55570 msgr2=0x55cca1f6c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55570 0x55cca1f6c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 -- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 msgr2=0x55cca1f6c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f6c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 -- 172.21.15.150:0/2451068989 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23141098c0 con 0x55cca1f55ec0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f231973f700 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f6d450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f230c004430 tx=0x7f230c0048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3901b91f5c3b5f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f22f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f22f3fff700 1 -- 172.21.15.150:0/2451068989 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c00f040 con 0x55cca1f55ec0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f22f3fff700 1 -- 172.21.15.150:0/2451068989 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f230c003c10 con 0x55cca1f55ec0 2024-05-10T04:41:26.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.405+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23000a1570 con 0x55cca1f55ec0 2024-05-10T04:41:26.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.406+0000 7f22f3fff700 1 -- 172.21.15.150:0/2451068989 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f230c003dd0 con 0x55cca1f55ec0 2024-05-10T04:41:26.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.406+0000 7f232b4e6b80 10 client.16281 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:26.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.406+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f230c00a410 con 0x55cca1f55ec0 2024-05-10T04:41:26.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.406+0000 7f22f3fff700 1 -- 172.21.15.150:0/2451068989 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23000a1570 con 0x55cca1f55ec0 2024-05-10T04:41:26.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.406+0000 7f22f3fff700 1 -- 172.21.15.150:0/2451068989 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f230c026020 con 0x55cca1f55ec0 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.407+0000 7f232b4e6b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.407+0000 7f232b4e6b80 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.407+0000 7f232b4e6b80 20 client.16281 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.407+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1fc90a0 0x55cca1fcb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.407+0000 7f232b4e6b80 4 client.16281 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.407+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cca1e6cec0 con 0x55cca1fc90a0 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.407+0000 7f2318f3e700 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1fc90a0 0x55cca1fcb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f22f3fff700 10 client.16281 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f2318f3e700 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1fc90a0 0x55cca1fcb4c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cba39061903f38 server_cookie=d3a2c1e1f6461ae7 in_seq=0 out_seq=0 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f22f3fff700 1 -- 172.21.15.150:0/2451068989 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cca1e6cec0 con 0x55cca1fc90a0 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f22f3fff700 10 client.16281 handle_command_reply: tid=0 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 client.16281 shutdown 2024-05-10T04:41:26.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 msgr2=0x55cca1f6d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f6d450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f230c004430 tx=0x7f230c0048e0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 shutdown_connections 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55cca1fc90a0 0x55cca1fcb4c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cca1f55ec0 0x55cca1f6d450 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cca1f546a0 0x55cca1f6c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f22f3fff700 0 client.16281 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 --2- 172.21.15.150:0/2451068989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cca1f55570 0x55cca1f6c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 >> 172.21.15.150:0/2451068989 conn(0x55cca1eb6a60 msgr2=0x55cca1f4b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 shutdown_connections 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 1 -- 172.21.15.150:0/2451068989 wait complete. 2024-05-10T04:41:26.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.408+0000 7f232b4e6b80 20 client.16281 trim_cache size 0 max 16384 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "aa2b86e6-7518-43a9-984e-f885b0c7fb30": { 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "aa2b86e6-7518-43a9-984e-f885b0c7fb30", 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:41:26.413 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:41:26.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23040a1240 msgr2=0x7f23040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23040a1240 0x7f23040a3750 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308005fb0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 msgr2=0x7f231419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 0x7f231419a640 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2300009a20 tx=0x7f2300009d30 comp rx=0 tx=0).stop 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 shutdown_connections 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2314102580 0x7f23141a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f23040a1240 0x7f23040a3750 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2314100ac0 0x7f231419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 --2- 172.21.15.150:0/67394132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2314101be0 0x7f231419a640 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.409+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 >> 172.21.15.150:0/67394132 conn(0x7f23140fc0d0 msgr2=0x7f2314104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.410+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 shutdown_connections 2024-05-10T04:41:26.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:26.410+0000 7f231b1a2700 1 -- 172.21.15.150:0/67394132 wait complete. 2024-05-10T04:41:26.427 INFO:tasks.cephfs.filesystem:scrub status for tag:aa2b86e6-7518-43a9-984e-f885b0c7fb30 - {'path': '/', 'tag': 'aa2b86e6-7518-43a9-984e-f885b0c7fb30', 'options': 'recursive,force'} 2024-05-10T04:41:27.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:26 smithi181 ceph-mon[47314]: scrub queued for path: / 2024-05-10T04:41:27.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:26 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:41:27.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:26 smithi181 ceph-mon[47314]: scrub summary: active paths [/] 2024-05-10T04:41:27.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:26 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:41:27.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:26 smithi150 ceph-mon[42404]: scrub queued for path: / 2024-05-10T04:41:27.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:26 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:41:27.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:26 smithi150 ceph-mon[42404]: scrub summary: active paths [/] 2024-05-10T04:41:27.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:26 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:41:27.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:26 smithi161 ceph-mon[46912]: scrub queued for path: / 2024-05-10T04:41:27.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:26 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:41:27.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:26 smithi161 ceph-mon[46912]: scrub summary: active paths [/] 2024-05-10T04:41:27.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:26 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:41:28.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:27 smithi181 ceph-mon[47314]: pgmap v328: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T04:41:28.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:27 smithi181 ceph-mon[47314]: scrub summary: idle 2024-05-10T04:41:28.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:27 smithi150 ceph-mon[42404]: pgmap v328: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T04:41:28.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:27 smithi150 ceph-mon[42404]: scrub summary: idle 2024-05-10T04:41:28.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:27 smithi161 ceph-mon[46912]: pgmap v328: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-10T04:41:28.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:27 smithi161 ceph-mon[46912]: scrub summary: idle 2024-05-10T04:41:28.976 INFO:tasks.workunit.client.0.smithi150.stderr:Updating files: 96% (11837/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr:Note: switching to '50174015a7e79f951b988949abe236791f52d4da'. 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr:state without impacting any branches by switching back to a branch. 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-10T04:41:29.933 INFO:tasks.workunit.client.0.smithi150.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr: git switch -c 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr:Or undo this operation with: 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr: git switch - 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:29.934 INFO:tasks.workunit.client.0.smithi150.stderr:HEAD is now at 50174015a7e Merge PR #57343 into wip-pdonnell-testing-20240509.171335-reef-debug 2024-05-10T04:41:29.939 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-10T04:41:30.054 INFO:tasks.workunit.client.0.smithi150.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-10T04:41:30.056 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T04:41:30.056 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-10T04:41:30.557 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-10T04:41:30.611 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-10T04:41:30.649 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-10T04:41:30.650 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T04:41:30.650 INFO:tasks.workunit.client.0.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-10T04:41:30.687 INFO:tasks.workunit.client.0.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-10T04:41:30.691 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:41:30.691 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-10T04:41:30.708 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-05-10T04:41:30.709 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-05-10T04:41:30.710 DEBUG:teuthology.orchestra.run.smithi150:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=50174015a7e79f951b988949abe236791f52d4da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-10T04:41:30.771 INFO:tasks.workunit.client.0.smithi150.stderr:+ set -e 2024-05-10T04:41:30.771 INFO:tasks.workunit.client.0.smithi150.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-05-10T04:41:30.804 INFO:tasks.workunit.client.0.smithi150.stderr:--2024-05-10 04:41:30-- http://download.ceph.com/qa/fsync-tester.c 2024-05-10T04:41:30.805 INFO:tasks.workunit.client.0.smithi150.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-10T04:41:30.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:30 smithi161 ceph-mon[46912]: pgmap v329: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:30.841 INFO:tasks.workunit.client.0.smithi150.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-10T04:41:30.877 INFO:tasks.workunit.client.0.smithi150.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-10T04:41:30.877 INFO:tasks.workunit.client.0.smithi150.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-05-10T04:41:30.878 INFO:tasks.workunit.client.0.smithi150.stderr:Saving to: ‘fsync-tester.c’ 2024-05-10T04:41:30.881 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:30.881 INFO:tasks.workunit.client.0.smithi150.stderr: 0K .. 100% 165M=0s 2024-05-10T04:41:30.881 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:30.881 INFO:tasks.workunit.client.0.smithi150.stderr:2024-05-10 04:41:30 (165 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-05-10T04:41:30.881 INFO:tasks.workunit.client.0.smithi150.stderr: 2024-05-10T04:41:30.882 INFO:tasks.workunit.client.0.smithi150.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-05-10T04:41:30.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:30 smithi181 ceph-mon[47314]: pgmap v329: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:30.913 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:30 smithi150 ceph-mon[42404]: pgmap v329: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:30.933 INFO:tasks.workunit.client.0.smithi150.stderr:+ ./fsync-tester 2024-05-10T04:41:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 -- 172.21.15.150:0/204277715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e8105060 msgr2=0x7fe9e8107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 --2- 172.21.15.150:0/204277715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e8105060 0x7fe9e8107480 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009990 tx=0x7fe9dc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 -- 172.21.15.150:0/204277715 shutdown_connections 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 --2- 172.21.15.150:0/204277715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e81079c0 0x7fe9e8109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 --2- 172.21.15.150:0/204277715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9e8102700 0x7fe9e8104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 --2- 172.21.15.150:0/204277715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e8105060 0x7fe9e8107480 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 -- 172.21.15.150:0/204277715 >> 172.21.15.150:0/204277715 conn(0x7fe9e80fc120 msgr2=0x7fe9e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 -- 172.21.15.150:0/204277715 shutdown_connections 2024-05-10T04:41:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.236+0000 7fe9ed713700 1 -- 172.21.15.150:0/204277715 wait complete. 2024-05-10T04:41:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 Processor -- start 2024-05-10T04:41:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 -- start start 2024-05-10T04:41:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9e8102700 0x7fe9e8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e8105060 0x7fe9e8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e81079c0 0x7fe9e819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e819fad0 con 0x7fe9e81079c0 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e819fc10 con 0x7fe9e8102700 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.237+0000 7fe9ed713700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e819fda0 con 0x7fe9e8105060 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e81079c0 0x7fe9e819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e81079c0 0x7fe9e819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42492/0 (socket says 172.21.15.150:42492) 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 -- 172.21.15.150:0/1479223282 learned_addr learned my addr 172.21.15.150:0/1479223282 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e8105060 0x7fe9e8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9e8102700 0x7fe9e8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 -- 172.21.15.150:0/1479223282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e8105060 msgr2=0x7fe9e8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e8105060 0x7fe9e8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 -- 172.21.15.150:0/1479223282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9e8102700 msgr2=0x7fe9e8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9e8102700 0x7fe9e8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 -- 172.21.15.150:0/1479223282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0095f0 con 0x7fe9e81079c0 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e81079c0 0x7fe9e819f500 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000efc0 tx=0x7fe9e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9cffff700 1 -- 172.21.15.150:0/1479223282 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0018070 con 0x7fe9e81079c0 2024-05-10T04:41:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e81a0020 con 0x7fe9e81079c0 2024-05-10T04:41:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9cffff700 1 -- 172.21.15.150:0/1479223282 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e000f040 con 0x7fe9e81079c0 2024-05-10T04:41:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9cffff700 1 -- 172.21.15.150:0/1479223282 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000ce40 con 0x7fe9e81079c0 2024-05-10T04:41:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.238+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e81a0580 con 0x7fe9e81079c0 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9cffff700 1 -- 172.21.15.150:0/1479223282 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe9e0007500 con 0x7fe9e81079c0 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 Processor -- start 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 -- start start 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9cffff700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe9d00a1120 0x7fe9d00a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7ef180 0x55980f7ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f09a0 0x55980f7f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f7160f0 con 0x55980f7f09a0 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f713ab0 con 0x55980f7f0050 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f70f3a0 con 0x55980f7ef180 2024-05-10T04:41:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9cffff700 1 -- 172.21.15.150:0/1479223282 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe9e00cc770 con 0x7fe9e81079c0 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.242+0000 7fe9e77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f09a0 0x55980f7f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe9d00a1120 0x7fe9d00a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f09a0 0x55980f7f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42506/0 (socket says 172.21.15.150:42506) 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 -- 172.21.15.150:0/2903066623 learned_addr learned my addr 172.21.15.150:0/2903066623 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 -- 172.21.15.150:0/2903066623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7ef180 msgr2=0x55980f7ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7ef180 0x55980f7ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 -- 172.21.15.150:0/2903066623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 msgr2=0x55980f7f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 -- 172.21.15.150:0/2903066623 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980f590350 con 0x55980f7f09a0 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe9d00a1120 0x7fe9d00a3630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b2e0 tx=0x7fe9dc004500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f09a0 0x55980f7f72a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00a1770 tx=0x7fe9e0099700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e45a63fc436ea55 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9cd7fa700 1 -- 172.21.15.150:0/2903066623 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0099210 con 0x55980f7f09a0 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9cd7fa700 1 -- 172.21.15.150:0/2903066623 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e0099fb0 con 0x55980f7f09a0 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.243+0000 7fe9cd7fa700 1 -- 172.21.15.150:0/2903066623 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e00b90d0 con 0x55980f7f09a0 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 -- 172.21.15.150:0/2903066623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f09a0 msgr2=0x55980f7f72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f09a0 0x55980f7f72a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00a1770 tx=0x7fe9e0099700 comp rx=0 tx=0).stop 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 -- 172.21.15.150:0/2903066623 shutdown_connections 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f09a0 0x55980f7f72a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7ef180 0x55980f7ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/2903066623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 -- 172.21.15.150:0/2903066623 >> 172.21.15.150:0/2903066623 conn(0x55980f751a00 msgr2=0x55980f7e9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 -- 172.21.15.150:0/2903066623 shutdown_connections 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.244+0000 7fe9fda57b80 1 -- 172.21.15.150:0/2903066623 wait complete. 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 -- start start 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 Processor -- start 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f800680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f6ab500 con 0x55980f7ef180 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f6ab1a0 con 0x55980f7f0050 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f592640 con 0x55980f7f09a0 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9fda57b80 10 client.0 _ll_register_callbacks cb 0x55980f6b4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9e6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9e6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42512/0 (socket says 172.21.15.150:42512) 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/3155746203 learned_addr learned my addr 172.21.15.150:0/3155746203 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.245+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f800680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/3155746203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 msgr2=0x55980f800680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f800680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 msgr2=0x55980f7f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/3155746203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e81a0580 con 0x55980f7ef180 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7f91e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800be90 tx=0x7fe9d80106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f38195b7b31eb18 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9caff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9caff5700 1 -- 172.21.15.150:0/3155746203 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8004030 con 0x55980f7ef180 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9caff5700 1 -- 172.21.15.150:0/3155746203 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe9d8017070 con 0x55980f7ef180 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55980f668210 con 0x55980f7ef180 2024-05-10T04:41:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9caff5700 1 -- 172.21.15.150:0/3155746203 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d8004190 con 0x55980f7ef180 2024-05-10T04:41:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9fda57b80 10 client.16293 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55980f53cb00 con 0x55980f7ef180 2024-05-10T04:41:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9caff5700 1 -- 172.21.15.150:0/3155746203 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55980f668210 con 0x55980f7ef180 2024-05-10T04:41:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.246+0000 7fe9caff5700 1 -- 172.21.15.150:0/3155746203 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9d80114d0 con 0x55980f7ef180 2024-05-10T04:41:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9fda57b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9fda57b80 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9fda57b80 20 client.16293 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f804ee0 0x55980f807330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9fda57b80 4 client.16293 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55980f7be720 con 0x55980f804ee0 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f804ee0 0x55980f807330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9caff5700 10 client.16293 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.247+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f804ee0 0x55980f807330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ae1ce0e51edb67 server_cookie=fb3a4301ef253869 in_seq=0 out_seq=0 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9caff5700 1 -- 172.21.15.150:0/3155746203 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55980f7be720 con 0x55980f804ee0 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9caff5700 10 client.16293 handle_command_reply: tid=0 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 client.16293 shutdown 2024-05-10T04:41:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 msgr2=0x55980f7f91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7f91e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800be90 tx=0x7fe9d80106b0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 shutdown_connections 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f804ee0 0x55980f807330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f800680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7f91e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3155746203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 >> 172.21.15.150:0/3155746203 conn(0x55980f751a00 msgr2=0x55980f7e9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9caff5700 0 client.16293 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 shutdown_connections 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.248+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3155746203 wait complete. 2024-05-10T04:41:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.249+0000 7fe9fda57b80 20 client.16293 trim_cache size 0 max 16384 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 Processor -- start 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 -- start start 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7ef180 0x55980f800c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f0050 0x55980f800640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f806ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f7f7590 con 0x55980f7f0050 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f7f7210 con 0x55980f7ef180 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f7f4ae0 con 0x55980f7f09a0 2024-05-10T04:41:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f0050 0x55980f800640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f0050 0x55980f800640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42526/0 (socket says 172.21.15.150:42526) 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 -- 172.21.15.150:0/3089638745 learned_addr learned my addr 172.21.15.150:0/3089638745 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f806ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7ef180 0x55980f800c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 -- 172.21.15.150:0/3089638745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 msgr2=0x55980f806ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f806ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 -- 172.21.15.150:0/3089638745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7ef180 msgr2=0x55980f800c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7ef180 0x55980f800c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.264+0000 7fe9e67fc700 1 -- 172.21.15.150:0/3089638745 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980f6d5500 con 0x55980f7f0050 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.265+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f0050 0x55980f800640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009990 tx=0x7fe9dc02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62f52dd21187712 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.265+0000 7fe9cdffb700 1 -- 172.21.15.150:0/3089638745 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0057b0 con 0x55980f7f0050 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.265+0000 7fe9cdffb700 1 -- 172.21.15.150:0/3089638745 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9dc0223a0 con 0x55980f7f0050 2024-05-10T04:41:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.265+0000 7fe9cdffb700 1 -- 172.21.15.150:0/3089638745 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc022bc0 con 0x55980f7f0050 2024-05-10T04:41:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3089638745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f0050 msgr2=0x55980f800640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f0050 0x55980f800640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009990 tx=0x7fe9dc02f000 comp rx=0 tx=0).stop 2024-05-10T04:41:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3089638745 shutdown_connections 2024-05-10T04:41:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f806ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7ef180 0x55980f800c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/3089638745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7f0050 0x55980f800640 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3089638745 >> 172.21.15.150:0/3089638745 conn(0x55980f751a00 msgr2=0x55980f7f3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3089638745 shutdown_connections 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 -- 172.21.15.150:0/3089638745 wait complete. 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.266+0000 7fe9fda57b80 1 -- start start 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 1 Processor -- start 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7feed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f7ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f8593a0 con 0x55980f7ef180 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f8594e0 con 0x55980f7f0050 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980f859620 con 0x55980f7f09a0 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9fda57b80 10 client.0 _ll_register_callbacks cb 0x55980f6b4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7feed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42536/0 (socket says 172.21.15.150:42536) 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/379996906 learned_addr learned my addr 172.21.15.150:0/379996906 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e77fe700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f7ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/379996906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 msgr2=0x55980f7ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f7ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 msgr2=0x55980f7ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 -- 172.21.15.150:0/379996906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc02a040 con 0x55980f7ef180 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9e6ffd700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7feed0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005950 tx=0x7fe9d8011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930f9d9f002e4f5a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.267+0000 7fe9caff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9caff5700 1 -- 172.21.15.150:0/379996906 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8005240 con 0x55980f7ef180 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9caff5700 1 -- 172.21.15.150:0/379996906 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe9d8017040 con 0x55980f7ef180 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e0099fb0 con 0x55980f7ef180 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9caff5700 1 -- 172.21.15.150:0/379996906 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9d80053a0 con 0x55980f7ef180 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 10 client.16299 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55980f594c70 con 0x55980f7ef180 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9caff5700 1 -- 172.21.15.150:0/379996906 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9e0099fb0 con 0x55980f7ef180 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9caff5700 1 -- 172.21.15.150:0/379996906 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9d801b420 con 0x55980f7ef180 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 20 client.16299 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f85afa0 0x55980f85d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 4 client.16299 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:41:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.268+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55980f707e60 con 0x55980f85afa0 2024-05-10T04:41:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.269+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f85afa0 0x55980f85d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.269+0000 7fe9e67fc700 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f85afa0 0x55980f85d3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb31f09a2a91695e server_cookie=519c8e2c3402f707 in_seq=0 out_seq=0 2024-05-10T04:41:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.269+0000 7fe9caff5700 10 client.16299 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9caff5700 1 -- 172.21.15.150:0/379996906 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55980f707e60 con 0x55980f85afa0 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9caff5700 10 client.16299 handle_command_reply: tid=0 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 client.16299 shutdown 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 msgr2=0x55980f7feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7feed0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005950 tx=0x7fe9d8011a40 comp rx=0 tx=0).stop 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 shutdown_connections 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55980f85afa0 0x55980f85d3c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55980f7f09a0 0x55980f7ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55980f7ef180 0x55980f7feed0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9caff5700 0 client.16299 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 --2- 172.21.15.150:0/379996906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55980f7f0050 0x55980f7ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 >> 172.21.15.150:0/379996906 conn(0x55980f751a00 msgr2=0x55980f806ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 shutdown_connections 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 1 -- 172.21.15.150:0/379996906 wait complete. 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.281+0000 7fe9fda57b80 20 client.16299 trim_cache size 0 max 16384 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:41:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:41:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe9d00a1120 msgr2=0x7fe9d00a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe9d00a1120 0x7fe9d00a3630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b2e0 tx=0x7fe9dc004500 comp rx=0 tx=0).stop 2024-05-10T04:41:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e81079c0 msgr2=0x7fe9e819f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e81079c0 0x7fe9e819f500 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000efc0 tx=0x7fe9e000c3a0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 shutdown_connections 2024-05-10T04:41:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9e81079c0 0x7fe9e819f500 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9e8102700 0x7fe9e8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe9d00a1120 0x7fe9d00a3630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 --2- 172.21.15.150:0/1479223282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e8105060 0x7fe9e8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 >> 172.21.15.150:0/1479223282 conn(0x7fe9e80fc120 msgr2=0x7fe9e8104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 shutdown_connections 2024-05-10T04:41:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:32.282+0000 7fe9ed713700 1 -- 172.21.15.150:0/1479223282 wait complete. 2024-05-10T04:41:32.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:32 smithi161 ceph-mon[46912]: pgmap v330: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:32.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:32 smithi181 ceph-mon[47314]: pgmap v330: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:32.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:32 smithi150 ceph-mon[42404]: pgmap v330: 129 pgs: 129 active+clean; 674 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:41:34.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:34 smithi161 ceph-mon[46912]: pgmap v331: 129 pgs: 129 active+clean; 728 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-05-10T04:41:34.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:34 smithi181 ceph-mon[47314]: pgmap v331: 129 pgs: 129 active+clean; 728 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-05-10T04:41:34.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:34 smithi150 ceph-mon[42404]: pgmap v331: 129 pgs: 129 active+clean; 728 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-05-10T04:41:36.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:36 smithi161 ceph-mon[46912]: pgmap v332: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 433 KiB/s wr, 15 op/s 2024-05-10T04:41:36.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:36 smithi181 ceph-mon[47314]: pgmap v332: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 433 KiB/s wr, 15 op/s 2024-05-10T04:41:36.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:36 smithi150 ceph-mon[42404]: pgmap v332: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 433 KiB/s wr, 15 op/s 2024-05-10T04:41:38.836 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:38 smithi161 ceph-mon[46912]: pgmap v333: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 436 KiB/s wr, 17 op/s 2024-05-10T04:41:38.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:38 smithi181 ceph-mon[47314]: pgmap v333: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 436 KiB/s wr, 17 op/s 2024-05-10T04:41:38.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:38 smithi150 ceph-mon[42404]: pgmap v333: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 436 KiB/s wr, 17 op/s 2024-05-10T04:41:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:39 smithi161 ceph-mon[46912]: pgmap v334: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 436 KiB/s wr, 16 op/s 2024-05-10T04:41:39.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:39 smithi181 ceph-mon[47314]: pgmap v334: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 436 KiB/s wr, 16 op/s 2024-05-10T04:41:39.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:39 smithi150 ceph-mon[42404]: pgmap v334: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 436 KiB/s wr, 16 op/s 2024-05-10T04:41:42.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:42 smithi161 ceph-mon[46912]: pgmap v335: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 16 op/s 2024-05-10T04:41:42.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:42 smithi181 ceph-mon[47314]: pgmap v335: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 16 op/s 2024-05-10T04:41:42.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:42 smithi150 ceph-mon[42404]: pgmap v335: 129 pgs: 129 active+clean; 1.7 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 862 KiB/s wr, 16 op/s 2024-05-10T04:41:44.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:44 smithi161 ceph-mon[46912]: pgmap v336: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 865 KiB/s wr, 17 op/s 2024-05-10T04:41:44.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:44 smithi181 ceph-mon[47314]: pgmap v336: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 865 KiB/s wr, 17 op/s 2024-05-10T04:41:44.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:44 smithi150 ceph-mon[42404]: pgmap v336: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 865 KiB/s wr, 17 op/s 2024-05-10T04:41:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 -- 172.21.15.150:0/647567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050101be0 msgr2=0x7f4050101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 --2- 172.21.15.150:0/647567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050101be0 0x7f4050101ff0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4040009990 tx=0x7f4040009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 -- 172.21.15.150:0/647567489 shutdown_connections 2024-05-10T04:41:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 --2- 172.21.15.150:0/647567489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4050102580 0x7f4050108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 --2- 172.21.15.150:0/647567489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050100ac0 0x7f4050100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 --2- 172.21.15.150:0/647567489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050101be0 0x7f4050101ff0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 -- 172.21.15.150:0/647567489 >> 172.21.15.150:0/647567489 conn(0x7f40500fc0f0 msgr2=0x7f40500fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 -- 172.21.15.150:0/647567489 shutdown_connections 2024-05-10T04:41:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.409+0000 7f4058131700 1 -- 172.21.15.150:0/647567489 wait complete. 2024-05-10T04:41:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.410+0000 7f4058131700 1 Processor -- start 2024-05-10T04:41:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4058131700 1 -- start start 2024-05-10T04:41:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4058131700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050100ac0 0x7f40500690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4058131700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4050101be0 0x7f405019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4058131700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050102580 0x7f405019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4058131700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019fa20 con 0x7f4050102580 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4058131700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019fb90 con 0x7f4050100ac0 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4058131700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019fd50 con 0x7f4050101be0 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050102580 0x7f405019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40556cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4050101be0 0x7f405019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050102580 0x7f405019f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42340/0 (socket says 172.21.15.150:42340) 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 -- 172.21.15.150:0/839337595 learned_addr learned my addr 172.21.15.150:0/839337595 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4055ecd700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050100ac0 0x7f40500690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 -- 172.21.15.150:0/839337595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4050101be0 msgr2=0x7f405019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4050101be0 0x7f405019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 -- 172.21.15.150:0/839337595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050100ac0 msgr2=0x7f40500690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050100ac0 0x7f40500690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f40566ce700 1 -- 172.21.15.150:0/839337595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40400095f0 con 0x7f4050102580 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.411+0000 7f4055ecd700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050100ac0 0x7f40500690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.412+0000 7f40566ce700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050102580 0x7f405019f320 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4048007ee0 tx=0x7f404800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.412+0000 7f4046ffd700 1 -- 172.21.15.150:0/839337595 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048017070 con 0x7f4050102580 2024-05-10T04:41:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.412+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405019ffd0 con 0x7f4050102580 2024-05-10T04:41:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.412+0000 7f4046ffd700 1 -- 172.21.15.150:0/839337595 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404800f040 con 0x7f4050102580 2024-05-10T04:41:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.412+0000 7f4046ffd700 1 -- 172.21.15.150:0/839337595 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048005740 con 0x7f4050102580 2024-05-10T04:41:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.412+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40501a0450 con 0x7f4050102580 2024-05-10T04:41:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.414+0000 7f4046ffd700 1 -- 172.21.15.150:0/839337595 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f40480058a0 con 0x7f4050102580 2024-05-10T04:41:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.415+0000 7f4046ffd700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f403c0a1120 0x7f403c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.415+0000 7f4046ffd700 1 -- 172.21.15.150:0/839337595 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f40480cc4f0 con 0x7f4050102580 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.415+0000 7f4055ecd700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f403c0a1120 0x7f403c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.415+0000 7f4068475b80 1 Processor -- start 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.415+0000 7f4055ecd700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f403c0a1120 0x7f403c0a3630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4050101ae0 tx=0x7f404c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4068475b80 1 -- start start 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4068475b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca0c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4068475b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca0d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4068475b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d9a0 0x560dbca142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4068475b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbc9330f0 con 0x560dbca0c180 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4068475b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbc930ab0 con 0x560dbca0d9a0 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4068475b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbc92c3a0 con 0x560dbca0d050 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4055ecd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca0c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4055ecd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca0c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42356/0 (socket says 172.21.15.150:42356) 2024-05-10T04:41:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4055ecd700 1 -- 172.21.15.150:0/627098100 learned_addr learned my addr 172.21.15.150:0/627098100 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f40556cc700 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca0d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f40566ce700 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d9a0 0x560dbca142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4055ecd700 1 -- 172.21.15.150:0/627098100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 msgr2=0x560dbca0d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4055ecd700 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca0d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4055ecd700 1 -- 172.21.15.150:0/627098100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d9a0 msgr2=0x560dbca142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.416+0000 7f4055ecd700 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d9a0 0x560dbca142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.417+0000 7f4055ecd700 1 -- 172.21.15.150:0/627098100 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dbc7ad350 con 0x560dbca0c180 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.417+0000 7f4055ecd700 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca0c5b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f404c011ea0 tx=0x7f404c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f421647ea817949e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.417+0000 7f403bfff700 1 -- 172.21.15.150:0/627098100 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c012dc0 con 0x560dbca0c180 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.417+0000 7f403bfff700 1 -- 172.21.15.150:0/627098100 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c016070 con 0x560dbca0c180 2024-05-10T04:41:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.417+0000 7f403bfff700 1 -- 172.21.15.150:0/627098100 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c01d610 con 0x560dbca0c180 2024-05-10T04:41:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 -- 172.21.15.150:0/627098100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 msgr2=0x560dbca0c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca0c5b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f404c011ea0 tx=0x7f404c014000 comp rx=0 tx=0).stop 2024-05-10T04:41:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 -- 172.21.15.150:0/627098100 shutdown_connections 2024-05-10T04:41:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d9a0 0x560dbca142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca0c5b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 --2- 172.21.15.150:0/627098100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca0d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 -- 172.21.15.150:0/627098100 >> 172.21.15.150:0/627098100 conn(0x560dbc96ea00 msgr2=0x560dbca06a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 -- 172.21.15.150:0/627098100 shutdown_connections 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 -- 172.21.15.150:0/627098100 wait complete. 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 -- start start 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.418+0000 7f4068475b80 1 Processor -- start 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4068475b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca16210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4068475b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d050 0x560dbca16750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4068475b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca1d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4068475b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbc8c8500 con 0x560dbca0c180 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4068475b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbc8c81a0 con 0x560dbca0d050 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4068475b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbc7af640 con 0x560dbca0d9a0 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4068475b80 10 client.0 _ll_register_callbacks cb 0x560dbc8d1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4055ecd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca16210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4055ecd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca16210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42362/0 (socket says 172.21.15.150:42362) 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f4055ecd700 1 -- 172.21.15.150:0/2930802438 learned_addr learned my addr 172.21.15.150:0/2930802438 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f40566ce700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca1d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.419+0000 7f40556cc700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d050 0x560dbca16750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40556cc700 1 -- 172.21.15.150:0/2930802438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 msgr2=0x560dbca1d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40556cc700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca1d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40556cc700 1 -- 172.21.15.150:0/2930802438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 msgr2=0x560dbca16210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40556cc700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca16210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40556cc700 1 -- 172.21.15.150:0/2930802438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c011c30 con 0x560dbca0d050 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f4055ecd700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca16210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40566ce700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca1d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40556cc700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d050 0x560dbca16750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f404000b7b0 tx=0x7f404000bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f302120a4f29dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40397fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:41:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40397fa700 1 -- 172.21.15.150:0/2930802438 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040004700 con 0x560dbca0d050 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dbc885210 con 0x560dbca0d050 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40397fa700 1 -- 172.21.15.150:0/2930802438 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4040004860 con 0x560dbca0d050 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f40397fa700 1 -- 172.21.15.150:0/2930802438 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40400056f0 con 0x560dbca0d050 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f4068475b80 10 client.25949 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.420+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560dbc759b00 con 0x560dbca0d050 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f40397fa700 1 -- 172.21.15.150:0/2930802438 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dbc885210 con 0x560dbca0d050 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f40397fa700 1 -- 172.21.15.150:0/2930802438 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4040024b40 con 0x560dbca0d050 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f4068475b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f4068475b80 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f4068475b80 20 client.25949 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f4068475b80 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca21e80 0x560dbca242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f4068475b80 4 client.25949 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dbc9db720 con 0x560dbca21e80 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.421+0000 7f4055ecd700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca21e80 0x560dbca242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.422+0000 7f40397fa700 10 client.25949 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.422+0000 7f4055ecd700 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca21e80 0x560dbca242d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3b4fed0f0f23e8 server_cookie=7dc7f3a52d7be79b in_seq=0 out_seq=0 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.422+0000 7f40397fa700 1 -- 172.21.15.150:0/2930802438 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560dbc9db720 con 0x560dbca21e80 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.422+0000 7f40397fa700 10 client.25949 handle_command_reply: tid=0 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.422+0000 7f4068475b80 1 client.25949 shutdown 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.422+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d050 msgr2=0x560dbca16750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.422+0000 7f4068475b80 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d050 0x560dbca16750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f404000b7b0 tx=0x7f404000bfa0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 shutdown_connections 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca21e80 0x560dbca242d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca1d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0c180 0x560dbca16210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 --2- 172.21.15.150:0/2930802438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0d050 0x560dbca16750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f40397fa700 0 client.25949 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 >> 172.21.15.150:0/2930802438 conn(0x560dbc96ea00 msgr2=0x560dbca06a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 shutdown_connections 2024-05-10T04:41:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 1 -- 172.21.15.150:0/2930802438 wait complete. 2024-05-10T04:41:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.423+0000 7f4068475b80 20 client.25949 trim_cache size 0 max 16384 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.438+0000 7f4068475b80 1 Processor -- start 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.438+0000 7f4068475b80 1 -- start start 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.438+0000 7f4068475b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.438+0000 7f4068475b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d050 0x560dbca1d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.438+0000 7f4068475b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca15780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f4068475b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbca01360 con 0x560dbca0d050 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f4068475b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbca07560 con 0x560dbca0c180 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f4068475b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbca14240 con 0x560dbca0d9a0 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d050 0x560dbca1d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d050 0x560dbca1d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42378/0 (socket says 172.21.15.150:42378) 2024-05-10T04:41:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 -- 172.21.15.150:0/880761046 learned_addr learned my addr 172.21.15.150:0/880761046 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40566ce700 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca15780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f4055ecd700 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 -- 172.21.15.150:0/880761046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 msgr2=0x560dbca15780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca15780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 -- 172.21.15.150:0/880761046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 msgr2=0x560dbca242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 -- 172.21.15.150:0/880761046 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dbc8f2500 con 0x560dbca0d050 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f4055ecd700 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f40556cc700 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d050 0x560dbca1d670 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4040000c00 tx=0x7f4040009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6d37bd6f19df34 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.439+0000 7f4044ff9700 1 -- 172.21.15.150:0/880761046 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404000f620 con 0x560dbca0d050 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4044ff9700 1 -- 172.21.15.150:0/880761046 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404001e040 con 0x560dbca0d050 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4044ff9700 1 -- 172.21.15.150:0/880761046 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404000fac0 con 0x560dbca0d050 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 -- 172.21.15.150:0/880761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d050 msgr2=0x560dbca1d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d050 0x560dbca1d670 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4040000c00 tx=0x7f4040009310 comp rx=0 tx=0).stop 2024-05-10T04:41:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 -- 172.21.15.150:0/880761046 shutdown_connections 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d9a0 0x560dbca15780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 --2- 172.21.15.150:0/880761046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d050 0x560dbca1d670 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 -- 172.21.15.150:0/880761046 >> 172.21.15.150:0/880761046 conn(0x560dbc96ea00 msgr2=0x560dbca0c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.440+0000 7f4068475b80 1 -- 172.21.15.150:0/880761046 shutdown_connections 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.441+0000 7f4068475b80 1 -- 172.21.15.150:0/880761046 wait complete. 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.441+0000 7f4068475b80 1 -- start start 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.441+0000 7f4068475b80 1 Processor -- start 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4068475b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4068475b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca19d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4068475b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d9a0 0x560dbca1a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4068475b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbca1d9e0 con 0x560dbca0d9a0 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4068475b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbca1a810 con 0x560dbca0c180 2024-05-10T04:41:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4068475b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbca1a9d0 con 0x560dbca0d050 2024-05-10T04:41:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4068475b80 10 client.0 _ll_register_callbacks cb 0x560dbc8d1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d9a0 0x560dbca1a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d9a0 0x560dbca1a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42392/0 (socket says 172.21.15.150:42392) 2024-05-10T04:41:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 -- 172.21.15.150:0/2923133204 learned_addr learned my addr 172.21.15.150:0/2923133204 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f4055ecd700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40556cc700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca19d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 -- 172.21.15.150:0/2923133204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 msgr2=0x560dbca19d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca19d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 -- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 msgr2=0x560dbca1d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca1d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 -- 172.21.15.150:0/2923133204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dbc759b00 con 0x560dbca0d9a0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40556cc700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca19d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40566ce700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d9a0 0x560dbca1a2d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4048095fb0 tx=0x7f4048004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddbae668bc292d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.442+0000 7f40397fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f40397fa700 1 -- 172.21.15.150:0/2923133204 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480b1070 con 0x560dbca0d9a0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f40397fa700 1 -- 172.21.15.150:0/2923133204 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4048099540 con 0x560dbca0d9a0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f404c016070 con 0x560dbca0d9a0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f40397fa700 1 -- 172.21.15.150:0/2923133204 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f404809d370 con 0x560dbca0d9a0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 10 client.16317 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560dbc7b1c70 con 0x560dbca0d9a0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f40397fa700 1 -- 172.21.15.150:0/2923133204 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404c016070 con 0x560dbca0d9a0 2024-05-10T04:41:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f40397fa700 1 -- 172.21.15.150:0/2923133204 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4048099830 con 0x560dbca0d9a0 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 20 client.16317 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca77b90 0x560dbca79fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 4 client.16317 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560dbc924e60 con 0x560dbca77b90 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.443+0000 7f4055ecd700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca77b90 0x560dbca79fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.444+0000 7f40397fa700 10 client.16317 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.444+0000 7f4055ecd700 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca77b90 0x560dbca79fb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19637b133ebce9d5 server_cookie=54e1e79e22d42b56 in_seq=0 out_seq=0 2024-05-10T04:41:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.455+0000 7f40397fa700 1 -- 172.21.15.150:0/2923133204 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560dbc924e60 con 0x560dbca77b90 2024-05-10T04:41:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.455+0000 7f40397fa700 10 client.16317 handle_command_reply: tid=0 2024-05-10T04:41:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.455+0000 7f4068475b80 1 client.16317 shutdown 2024-05-10T04:41:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.455+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d9a0 msgr2=0x560dbca1a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.455+0000 7f4068475b80 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d9a0 0x560dbca1a2d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4048095fb0 tx=0x7f4048004c40 comp rx=0 tx=0).stop 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 shutdown_connections 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560dbca77b90 0x560dbca79fb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560dbca0d9a0 0x560dbca1a2d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dbca0c180 0x560dbca1d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f40397fa700 0 client.16317 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 --2- 172.21.15.150:0/2923133204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560dbca0d050 0x560dbca19d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 >> 172.21.15.150:0/2923133204 conn(0x560dbc96ea00 msgr2=0x560dbca10eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 shutdown_connections 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 1 -- 172.21.15.150:0/2923133204 wait complete. 2024-05-10T04:41:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4068475b80 20 client.16317 trim_cache size 0 max 16384 2024-05-10T04:41:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:41:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:41:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:41:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:41:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f403c0a1120 msgr2=0x7f403c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f403c0a1120 0x7f403c0a3630 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4050101ae0 tx=0x7f404c009260 comp rx=0 tx=0).stop 2024-05-10T04:41:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050102580 msgr2=0x7f405019f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050102580 0x7f405019f320 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4048007ee0 tx=0x7f404800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 shutdown_connections 2024-05-10T04:41:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4050102580 0x7f405019f320 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4050100ac0 0x7f40500690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f403c0a1120 0x7f403c0a3630 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 --2- 172.21.15.150:0/839337595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4050101be0 0x7f405019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.456+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 >> 172.21.15.150:0/839337595 conn(0x7f40500fc0f0 msgr2=0x7f4050104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.457+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 shutdown_connections 2024-05-10T04:41:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:45.457+0000 7f4058131700 1 -- 172.21.15.150:0/839337595 wait complete. 2024-05-10T04:41:46.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:46 smithi161 ceph-mon[46912]: pgmap v337: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-10T04:41:46.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:46 smithi181 ceph-mon[47314]: pgmap v337: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-10T04:41:46.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:46 smithi150 ceph-mon[42404]: pgmap v337: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-10T04:41:48.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:48 smithi161 ceph-mon[46912]: pgmap v338: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 15 op/s 2024-05-10T04:41:48.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:48 smithi181 ceph-mon[47314]: pgmap v338: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 15 op/s 2024-05-10T04:41:48.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:48 smithi150 ceph-mon[42404]: pgmap v338: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 15 op/s 2024-05-10T04:41:49.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:49 smithi161 ceph-mon[46912]: pgmap v339: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 860 KiB/s wr, 13 op/s 2024-05-10T04:41:49.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:49 smithi181 ceph-mon[47314]: pgmap v339: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 860 KiB/s wr, 13 op/s 2024-05-10T04:41:49.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:49 smithi150 ceph-mon[42404]: pgmap v339: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 860 KiB/s wr, 13 op/s 2024-05-10T04:41:52.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:52 smithi161 ceph-mon[46912]: pgmap v340: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:41:52.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:52 smithi181 ceph-mon[47314]: pgmap v340: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:41:52.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:52 smithi150 ceph-mon[42404]: pgmap v340: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:41:54.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:54 smithi161 ceph-mon[46912]: pgmap v341: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 778 KiB/s wr, 16 op/s 2024-05-10T04:41:54.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:54 smithi181 ceph-mon[47314]: pgmap v341: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 778 KiB/s wr, 16 op/s 2024-05-10T04:41:54.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:54 smithi150 ceph-mon[42404]: pgmap v341: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 778 KiB/s wr, 16 op/s 2024-05-10T04:41:56.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:41:56.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/440553969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641026d0 msgr2=0x7f3a64104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/440553969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641026d0 0x7f3a64104af0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009990 tx=0x7f3a54009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/440553969 shutdown_connections 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/440553969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a64107990 0x7f3a64109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/440553969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a641026d0 0x7f3a64104af0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/440553969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a64105030 0x7f3a64107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/440553969 >> 172.21.15.150:0/440553969 conn(0x7f3a640fc0f0 msgr2=0x7f3a640fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/440553969 shutdown_connections 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/440553969 wait complete. 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 Processor -- start 2024-05-10T04:41:56.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.587+0000 7f3a6a2e6700 1 -- start start 2024-05-10T04:41:56.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a6a2e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a641026d0 0x7f3a64101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a6a2e6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a64105030 0x7f3a641005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a6a2e6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64107990 0x7f3a64100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a6a2e6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641011d0 con 0x7f3a64107990 2024-05-10T04:41:56.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a6a2e6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a64101310 con 0x7f3a64105030 2024-05-10T04:41:56.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a6a2e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641a20d0 con 0x7f3a641026d0 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64107990 0x7f3a64100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64107990 0x7f3a64100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42438/0 (socket says 172.21.15.150:42438) 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 -- 172.21.15.150:0/2961137583 learned_addr learned my addr 172.21.15.150:0/2961137583 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a637fe700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a64105030 0x7f3a641005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a63fff700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a641026d0 0x7f3a64101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 -- 172.21.15.150:0/2961137583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a641026d0 msgr2=0x7f3a64101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a641026d0 0x7f3a64101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 -- 172.21.15.150:0/2961137583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a64105030 msgr2=0x7f3a641005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a64105030 0x7f3a641005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 -- 172.21.15.150:0/2961137583 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a540095f0 con 0x7f3a64107990 2024-05-10T04:41:56.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.588+0000 7f3a68883700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64107990 0x7f3a64100b20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00ada0 tx=0x7f3a5c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.589+0000 7f3a617fa700 1 -- 172.21.15.150:0/2961137583 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00f070 con 0x7f3a64107990 2024-05-10T04:41:56.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.589+0000 7f3a617fa700 1 -- 172.21.15.150:0/2961137583 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a5c007b00 con 0x7f3a64107990 2024-05-10T04:41:56.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.589+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a641a22d0 con 0x7f3a64107990 2024-05-10T04:41:56.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.589+0000 7f3a617fa700 1 -- 172.21.15.150:0/2961137583 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c005280 con 0x7f3a64107990 2024-05-10T04:41:56.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.589+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a641a2760 con 0x7f3a64107990 2024-05-10T04:41:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.592+0000 7f3a617fa700 1 -- 172.21.15.150:0/2961137583 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3a5c00c8d0 con 0x7f3a64107990 2024-05-10T04:41:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.592+0000 7f3a617fa700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a4c0a11c0 0x7f3a4c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.593+0000 7f3a617fa700 1 -- 172.21.15.150:0/2961137583 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3a5c0cc120 con 0x7f3a64107990 2024-05-10T04:41:56.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.593+0000 7f3a63fff700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a4c0a11c0 0x7f3a4c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.593+0000 7f3a63fff700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a4c0a11c0 0x7f3a4c0a36d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b5c0 tx=0x7f3a54003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 Processor -- start 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 -- start start 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a31af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32570 0x556262a32980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32ec0 0x556262a397c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262958150 con 0x556262a32ec0 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262955b10 con 0x556262a32570 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262951400 con 0x556262a316a0 2024-05-10T04:41:56.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32ec0 0x556262a397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32ec0 0x556262a397c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42450/0 (socket says 172.21.15.150:42450) 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 -- 172.21.15.150:0/1505723700 learned_addr learned my addr 172.21.15.150:0/1505723700 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a63fff700 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a31af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a637fe700 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32570 0x556262a32980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 -- 172.21.15.150:0/1505723700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 msgr2=0x556262a31af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a31af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 -- 172.21.15.150:0/1505723700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32570 msgr2=0x556262a32980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32570 0x556262a32980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.594+0000 7f3a68883700 1 -- 172.21.15.150:0/1505723700 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562627d23b0 con 0x556262a32ec0 2024-05-10T04:41:56.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.595+0000 7f3a68883700 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32ec0 0x556262a397c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c095c70 tx=0x7f3a5c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bbda2028ed2e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.595+0000 7f3a4a7fc700 1 -- 172.21.15.150:0/1505723700 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c09d030 con 0x556262a32ec0 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.595+0000 7f3a4a7fc700 1 -- 172.21.15.150:0/1505723700 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a5c0a15f0 con 0x556262a32ec0 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.595+0000 7f3a4a7fc700 1 -- 172.21.15.150:0/1505723700 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c09f2e0 con 0x556262a32ec0 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1505723700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32ec0 msgr2=0x556262a397c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32ec0 0x556262a397c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c095c70 tx=0x7f3a5c004b50 comp rx=0 tx=0).stop 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1505723700 shutdown_connections 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32ec0 0x556262a397c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a31af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1505723700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32570 0x556262a32980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1505723700 >> 172.21.15.150:0/1505723700 conn(0x556262993a60 msgr2=0x556262a2bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1505723700 shutdown_connections 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1505723700 wait complete. 2024-05-10T04:41:56.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 -- start start 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.596+0000 7f3a7a62ab80 1 Processor -- start 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a316a0 0x556262a3b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32570 0x556262a3bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562628ed560 con 0x556262a316a0 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562628ed200 con 0x556262a32ec0 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562627d46a0 con 0x556262a32570 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a7a62ab80 10 client.0 _ll_register_callbacks cb 0x5562628f6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a316a0 0x556262a3b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a316a0 0x556262a3b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42464/0 (socket says 172.21.15.150:42464) 2024-05-10T04:41:56.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 -- 172.21.15.150:0/2021996785 learned_addr learned my addr 172.21.15.150:0/2021996785 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a637fe700 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32570 0x556262a3bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 -- 172.21.15.150:0/2021996785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32570 msgr2=0x556262a3bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32570 0x556262a3bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 -- 172.21.15.150:0/2021996785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 msgr2=0x556262a42be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 -- 172.21.15.150:0/2021996785 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a641a2760 con 0x556262a316a0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a63fff700 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a316a0 0x556262a3b740 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009990 tx=0x7f3a54009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7329b44b21523d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.597+0000 7f3a437fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a437fe700 1 -- 172.21.15.150:0/2021996785 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5401d040 con 0x556262a316a0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a437fe700 1 -- 172.21.15.150:0/2021996785 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3a54005690 con 0x556262a316a0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562628aa270 con 0x556262a316a0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a437fe700 1 -- 172.21.15.150:0/2021996785 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a5400f870 con 0x556262a316a0 2024-05-10T04:41:56.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 10 client.16329 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55626277eb60 con 0x556262a316a0 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a437fe700 1 -- 172.21.15.150:0/2021996785 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562628aa270 con 0x556262a316a0 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a437fe700 1 -- 172.21.15.150:0/2021996785 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3a54030bd0 con 0x556262a316a0 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 10 client.16329 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 20 client.16329 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a47490 0x556262a498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 4 client.16329 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.598+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556262a00780 con 0x556262a47490 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.599+0000 7f3a637fe700 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a47490 0x556262a498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.599+0000 7f3a637fe700 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a47490 0x556262a498e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6735d247602acfa2 server_cookie=ea011f0a37391ac in_seq=0 out_seq=0 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.599+0000 7f3a437fe700 10 client.16329 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a437fe700 1 -- 172.21.15.150:0/2021996785 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556262a00780 con 0x556262a47490 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a437fe700 10 client.16329 handle_command_reply: tid=0 2024-05-10T04:41:56.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 client.16329 shutdown 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a316a0 msgr2=0x556262a3b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a316a0 0x556262a3b740 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009990 tx=0x7f3a54009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 shutdown_connections 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a47490 0x556262a498e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a316a0 0x556262a3b740 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a437fe700 0 client.16329 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/2021996785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32570 0x556262a3bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 >> 172.21.15.150:0/2021996785 conn(0x556262993a60 msgr2=0x556262a2bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 shutdown_connections 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/2021996785 wait complete. 2024-05-10T04:41:56.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.600+0000 7f3a7a62ab80 20 client.16329 trim_cache size 0 max 16384 2024-05-10T04:41:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.617+0000 7f3a7a62ab80 1 Processor -- start 2024-05-10T04:41:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a7a62ab80 1 -- start start 2024-05-10T04:41:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a316a0 0x556262a431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a42ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32ec0 0x556262a2cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262a47110 con 0x556262a32570 2024-05-10T04:41:56.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262a2d770 con 0x556262a316a0 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262a2d930 con 0x556262a32ec0 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a42ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a42ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42476/0 (socket says 172.21.15.150:42476) 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 -- 172.21.15.150:0/1875110781 learned_addr learned my addr 172.21.15.150:0/1875110781 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a68883700 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32ec0 0x556262a2cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a63fff700 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a316a0 0x556262a431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 -- 172.21.15.150:0/1875110781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32ec0 msgr2=0x556262a2cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32ec0 0x556262a2cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 -- 172.21.15.150:0/1875110781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a316a0 msgr2=0x556262a431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a316a0 0x556262a431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.618+0000 7f3a637fe700 1 -- 172.21.15.150:0/1875110781 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556262917560 con 0x556262a32570 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.619+0000 7f3a637fe700 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a42ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x556262a4a160 tx=0x7f3a58011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85712d567cecb2ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.619+0000 7f3a4affd700 1 -- 172.21.15.150:0/1875110781 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58017040 con 0x556262a32570 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.619+0000 7f3a4affd700 1 -- 172.21.15.150:0/1875110781 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a58011e30 con 0x556262a32570 2024-05-10T04:41:56.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.619+0000 7f3a4affd700 1 -- 172.21.15.150:0/1875110781 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58005280 con 0x556262a32570 2024-05-10T04:41:56.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1875110781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 msgr2=0x556262a42ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a42ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x556262a4a160 tx=0x7f3a58011720 comp rx=0 tx=0).stop 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1875110781 shutdown_connections 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a32ec0 0x556262a2cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a316a0 0x556262a431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/1875110781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a42ba0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1875110781 >> 172.21.15.150:0/1875110781 conn(0x556262993a60 msgr2=0x556262a2bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1875110781 shutdown_connections 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/1875110781 wait complete. 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 -- start start 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.620+0000 7f3a7a62ab80 1 Processor -- start 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a99e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a7a62ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a9c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262a9c8d0 con 0x556262a32570 2024-05-10T04:41:56.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262a9ca10 con 0x556262a32ec0 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a7a62ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556262a9cba0 con 0x556262a316a0 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a7a62ab80 10 client.0 _ll_register_callbacks cb 0x5562628f6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a99e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a99e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42494/0 (socket says 172.21.15.150:42494) 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 -- 172.21.15.150:0/3258219841 learned_addr learned my addr 172.21.15.150:0/3258219841 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a63fff700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a68883700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a9c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 -- 172.21.15.150:0/3258219841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 msgr2=0x556262a978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 -- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 msgr2=0x556262a9c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a9c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 -- 172.21.15.150:0/3258219841 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5800ad00 con 0x556262a32570 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a68883700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a9c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a637fe700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a99e40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3a58004a60 tx=0x7f3a58004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265df99e2de6ac53 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.621+0000 7f3a437fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a437fe700 1 -- 172.21.15.150:0/3258219841 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58017050 con 0x556262a32570 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a437fe700 1 -- 172.21.15.150:0/3258219841 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3a5801c870 con 0x556262a32570 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a5c0a15f0 con 0x556262a32570 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a437fe700 1 -- 172.21.15.150:0/3258219841 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a58004020 con 0x556262a32570 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 10 client.16335 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5562627d6cd0 con 0x556262a32570 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a437fe700 1 -- 172.21.15.150:0/3258219841 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a5c0a15f0 con 0x556262a32570 2024-05-10T04:41:56.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a437fe700 1 -- 172.21.15.150:0/3258219841 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3a58003680 con 0x556262a32570 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 10 client.16335 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 20 client.16335 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a9e4a0 0x556262aa08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 4 client.16335 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.622+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556262949ec0 con 0x556262a9e4a0 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a63fff700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a9e4a0 0x556262aa08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a437fe700 10 client.16335 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a63fff700 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a9e4a0 0x556262aa08c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db39351ca4baa2be server_cookie=b0112b522e243d7e in_seq=0 out_seq=0 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a437fe700 1 -- 172.21.15.150:0/3258219841 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556262949ec0 con 0x556262a9e4a0 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a437fe700 10 client.16335 handle_command_reply: tid=0 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a7a62ab80 1 client.16335 shutdown 2024-05-10T04:41:56.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 msgr2=0x556262a99e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a99e40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3a58004a60 tx=0x7f3a58004a90 comp rx=0 tx=0).stop 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 shutdown_connections 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556262a9e4a0 0x556262aa08c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.623+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556262a32ec0 0x556262a9c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a437fe700 0 client.16335 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556262a316a0 0x556262a978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a7a62ab80 1 --2- 172.21.15.150:0/3258219841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556262a32570 0x556262a99e40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 >> 172.21.15.150:0/3258219841 conn(0x556262993a60 msgr2=0x556262a302b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 shutdown_connections 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a7a62ab80 1 -- 172.21.15.150:0/3258219841 wait complete. 2024-05-10T04:41:56.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a7a62ab80 20 client.16335 trim_cache size 0 max 16384 2024-05-10T04:41:56.632 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:41:56.632 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:41:56.632 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:41:56.632 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:41:56.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a4c0a11c0 msgr2=0x7f3a4c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a4c0a11c0 0x7f3a4c0a36d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b5c0 tx=0x7f3a54003790 comp rx=0 tx=0).stop 2024-05-10T04:41:56.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64107990 msgr2=0x7f3a64100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64107990 0x7f3a64100b20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00ada0 tx=0x7f3a5c00c330 comp rx=0 tx=0).stop 2024-05-10T04:41:56.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 shutdown_connections 2024-05-10T04:41:56.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a64107990 0x7f3a64100b20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3a4c0a11c0 0x7f3a4c0a36d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a641026d0 0x7f3a64101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 --2- 172.21.15.150:0/2961137583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a64105030 0x7f3a641005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 >> 172.21.15.150:0/2961137583 conn(0x7f3a640fc0f0 msgr2=0x7f3a641035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:56.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 shutdown_connections 2024-05-10T04:41:56.634 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.624+0000 7f3a6a2e6700 1 -- 172.21.15.150:0/2961137583 wait complete. 2024-05-10T04:41:56.639 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:41:56.640 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 -- 172.21.15.150:0/885546554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290102590 msgr2=0x7f9290108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 --2- 172.21.15.150:0/885546554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290102590 0x7f9290108e40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f928800b0a0 tx=0x7f928800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 -- 172.21.15.150:0/885546554 shutdown_connections 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 --2- 172.21.15.150:0/885546554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290102590 0x7f9290108e40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 --2- 172.21.15.150:0/885546554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9290100ad0 0x7f9290100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 --2- 172.21.15.150:0/885546554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290101bf0 0x7f9290102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 -- 172.21.15.150:0/885546554 >> 172.21.15.150:0/885546554 conn(0x7f92900fc0c0 msgr2=0x7f92900fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 -- 172.21.15.150:0/885546554 shutdown_connections 2024-05-10T04:41:56.784 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.782+0000 7f929590c700 1 -- 172.21.15.150:0/885546554 wait complete. 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 Processor -- start 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 -- start start 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290100ad0 0x7f9290196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290101bf0 0x7f92901974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9290102590 0x7f929019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929019ea50 con 0x7f9290101bf0 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929019ebc0 con 0x7f9290102590 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.783+0000 7f929590c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929019ed80 con 0x7f9290100ad0 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290101bf0 0x7f92901974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290101bf0 0x7f92901974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42514/0 (socket says 172.21.15.150:42514) 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 -- 172.21.15.150:0/3694186111 learned_addr learned my addr 172.21.15.150:0/3694186111 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928effd700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290100ad0 0x7f9290196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928f7fe700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9290102590 0x7f929019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 -- 172.21.15.150:0/3694186111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290100ad0 msgr2=0x7f9290196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290100ad0 0x7f9290196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 -- 172.21.15.150:0/3694186111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9290102590 msgr2=0x7f929019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9290102590 0x7f929019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 -- 172.21.15.150:0/3694186111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288009d00 con 0x7f9290101bf0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928f7fe700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9290102590 0x7f929019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f928e7fc700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290101bf0 0x7f92901974b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f928400d980 tx=0x7f928400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f929490a700 1 -- 172.21.15.150:0/3694186111 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284004a30 con 0x7f9290101bf0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929019f060 con 0x7f9290101bf0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f929490a700 1 -- 172.21.15.150:0/3694186111 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92840054e0 con 0x7f9290101bf0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.784+0000 7f929490a700 1 -- 172.21.15.150:0/3694186111 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284009c90 con 0x7f9290101bf0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.785+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929019f520 con 0x7f9290101bf0 2024-05-10T04:41:56.788 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.786+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9290107ce0 con 0x7f9290101bf0 2024-05-10T04:41:56.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.788+0000 7f929490a700 1 -- 172.21.15.150:0/3694186111 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9284004c20 con 0x7f9290101bf0 2024-05-10T04:41:56.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.788+0000 7f929490a700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92780a1120 0x7f92780a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:56.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.789+0000 7f929490a700 1 -- 172.21.15.150:0/3694186111 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f92840ccf80 con 0x7f9290101bf0 2024-05-10T04:41:56.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.789+0000 7f928effd700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92780a1120 0x7f92780a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:56.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.789+0000 7f928effd700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92780a1120 0x7f92780a3630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9290101af0 tx=0x7f9280009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:56.794 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:56.793+0000 7f929490a700 1 -- 172.21.15.150:0/3694186111 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9284096370 con 0x7f9290101bf0 2024-05-10T04:41:56.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:56 smithi161 ceph-mon[46912]: pgmap v342: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:41:56.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:56 smithi181 ceph-mon[47314]: pgmap v342: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:41:56.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:56 smithi150 ceph-mon[42404]: pgmap v342: 129 pgs: 129 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:41:57.048 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.047+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92900678b0 con 0x7f9290101bf0 2024-05-10T04:41:57.049 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.047+0000 7f929490a700 1 -- 172.21.15.150:0/3694186111 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f9284017070 con 0x7f9290101bf0 2024-05-10T04:41:57.049 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:41:57.049 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:41:57.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92780a1120 msgr2=0x7f92780a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92780a1120 0x7f92780a3630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9290101af0 tx=0x7f9280009260 comp rx=0 tx=0).stop 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290101bf0 msgr2=0x7f92901974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290101bf0 0x7f92901974b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f928400d980 tx=0x7f928400dc90 comp rx=0 tx=0).stop 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 shutdown_connections 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9290102590 0x7f929019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f92780a1120 0x7f92780a3630 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9290100ad0 0x7f9290196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 --2- 172.21.15.150:0/3694186111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9290101bf0 0x7f92901974b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 >> 172.21.15.150:0/3694186111 conn(0x7f92900fc0c0 msgr2=0x7f9290104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 shutdown_connections 2024-05-10T04:41:57.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.050+0000 7f929590c700 1 -- 172.21.15.150:0/3694186111 wait complete. 2024-05-10T04:41:57.053 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:41:57.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T04:41:57.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 -- 172.21.15.150:0/1692731337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158101700 msgr2=0x7f8158109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1692731337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158101700 0x7f8158109010 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f815000b0a0 tx=0x7f815000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 -- 172.21.15.150:0/1692731337 shutdown_connections 2024-05-10T04:41:57.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1692731337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158101700 0x7f8158109010 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1692731337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81581020a0 0x7f81581024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1692731337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8158100db0 0x7f81581011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 -- 172.21.15.150:0/1692731337 >> 172.21.15.150:0/1692731337 conn(0x7f81580fb8f0 msgr2=0x7f81580fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 -- 172.21.15.150:0/1692731337 shutdown_connections 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.212+0000 7f815d3b6700 1 -- 172.21.15.150:0/1692731337 wait complete. 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 Processor -- start 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 -- start start 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158100db0 0x7f8158197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158101700 0x7f8158198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81581020a0 0x7f815819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819fad0 con 0x7f8158100db0 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819fc40 con 0x7f8158101700 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.213+0000 7f815d3b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815819fe00 con 0x7f81581020a0 2024-05-10T04:41:57.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158100db0 0x7f8158197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158100db0 0x7f8158197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42544/0 (socket says 172.21.15.150:42544) 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 -- 172.21.15.150:0/1928213127 learned_addr learned my addr 172.21.15.150:0/1928213127 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f81567fc700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158101700 0x7f8158198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f81577fe700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81581020a0 0x7f815819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 -- 172.21.15.150:0/1928213127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81581020a0 msgr2=0x7f815819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81581020a0 0x7f815819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 -- 172.21.15.150:0/1928213127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158101700 msgr2=0x7f8158198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158101700 0x7f8158198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 -- 172.21.15.150:0/1928213127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8150009d00 con 0x7f8158100db0 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f8156ffd700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158100db0 0x7f8158197ff0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f814800d980 tx=0x7f814800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f813ffff700 1 -- 172.21.15.150:0/1928213127 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148009930 con 0x7f8158100db0 2024-05-10T04:41:57.218 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f813ffff700 1 -- 172.21.15.150:0/1928213127 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8148010460 con 0x7f8158100db0 2024-05-10T04:41:57.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f813ffff700 1 -- 172.21.15.150:0/1928213127 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814800f620 con 0x7f8158100db0 2024-05-10T04:41:57.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81581a00e0 con 0x7f8158100db0 2024-05-10T04:41:57.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.214+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81581a05a0 con 0x7f8158100db0 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.217+0000 7f813ffff700 1 -- 172.21.15.150:0/1928213127 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f814800f780 con 0x7f8158100db0 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.217+0000 7f813ffff700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f81400a11c0 0x7f81400a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.217+0000 7f813ffff700 1 -- 172.21.15.150:0/1928213127 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f81480cc640 con 0x7f8158100db0 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.217+0000 7f81567fc700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f81400a11c0 0x7f81400a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f81567fc700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f81400a11c0 0x7f81400a36d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8158199090 tx=0x7f814c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 Processor -- start 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 -- start start 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fd6a0 0x562e604fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e604fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e605057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e60424150 con 0x562e604feec0 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e60421b10 con 0x562e604fd6a0 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.218+0000 7f816d6fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6041d400 con 0x562e604fe570 2024-05-10T04:41:57.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e605057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e605057c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42556/0 (socket says 172.21.15.150:42556) 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 -- 172.21.15.150:0/1870229797 learned_addr learned my addr 172.21.15.150:0/1870229797 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f8156ffd700 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fd6a0 0x562e604fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81567fc700 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e604fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 -- 172.21.15.150:0/1870229797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 msgr2=0x562e604fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e604fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 -- 172.21.15.150:0/1870229797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fd6a0 msgr2=0x562e604fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fd6a0 0x562e604fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 -- 172.21.15.150:0/1870229797 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e6029e3b0 con 0x562e604feec0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f81577fe700 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e605057c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8150015000 tx=0x7f8150009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4701cf72f3af1fad server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f813d7fa700 1 -- 172.21.15.150:0/1870229797 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815000e050 con 0x562e604feec0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f813d7fa700 1 -- 172.21.15.150:0/1870229797 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8150007840 con 0x562e604feec0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.219+0000 7f813d7fa700 1 -- 172.21.15.150:0/1870229797 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150005540 con 0x562e604feec0 2024-05-10T04:41:57.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 -- 172.21.15.150:0/1870229797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 msgr2=0x562e605057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e605057c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8150015000 tx=0x7f8150009c60 comp rx=0 tx=0).stop 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 -- 172.21.15.150:0/1870229797 shutdown_connections 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e605057c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fd6a0 0x562e604fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 --2- 172.21.15.150:0/1870229797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e604fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 -- 172.21.15.150:0/1870229797 >> 172.21.15.150:0/1870229797 conn(0x562e6045fa60 msgr2=0x562e604f7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 -- 172.21.15.150:0/1870229797 shutdown_connections 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.220+0000 7f816d6fab80 1 -- 172.21.15.150:0/1870229797 wait complete. 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 -- start start 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 Processor -- start 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e60507610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fe570 0x562e60507b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e6050eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e603b9560 con 0x562e604fe570 2024-05-10T04:41:57.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e603b9200 con 0x562e604feec0 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e602a06a0 con 0x562e604fd6a0 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f816d6fab80 10 client.0 _ll_register_callbacks cb 0x562e603c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f81567fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fe570 0x562e60507b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f81567fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fe570 0x562e60507b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:42568/0 (socket says 172.21.15.150:42568) 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f81567fc700 1 -- 172.21.15.150:0/2306656478 learned_addr learned my addr 172.21.15.150:0/2306656478 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f81577fe700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e6050eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.221+0000 7f8156ffd700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e60507610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f81567fc700 1 -- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 msgr2=0x562e60507610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f81567fc700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e60507610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f81567fc700 1 -- 172.21.15.150:0/2306656478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 msgr2=0x562e6050eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f81567fc700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e6050eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f81567fc700 1 -- 172.21.15.150:0/2306656478 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8158109900 con 0x562e604fe570 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f8156ffd700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e60507610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:57.228 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f81577fe700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e6050eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f81567fc700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fe570 0x562e60507b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f814c0042e0 tx=0x7f814c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5eca4916ea1727b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f813aff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f813aff5700 1 -- 172.21.15.150:0/2306656478 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c010eb0 con 0x562e604fe570 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f813aff5700 1 -- 172.21.15.150:0/2306656478 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f814c017070 con 0x562e604fe570 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e60376270 con 0x562e604fe570 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f813aff5700 1 -- 172.21.15.150:0/2306656478 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f814c011440 con 0x562e604fe570 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f816d6fab80 10 client.16353 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562e6024ab60 con 0x562e604fe570 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f813aff5700 1 -- 172.21.15.150:0/2306656478 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e60376270 con 0x562e604fe570 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.222+0000 7f813aff5700 1 -- 172.21.15.150:0/2306656478 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f814c01f6a0 con 0x562e604fe570 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f816d6fab80 10 client.16353 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f816d6fab80 10 client.16353 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f816d6fab80 20 client.16353 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60513290 0x562e605156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f816d6fab80 4 client.16353 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e604cc780 con 0x562e60513290 2024-05-10T04:41:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f8156ffd700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60513290 0x562e605156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f813aff5700 10 client.16353 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:57.230 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.223+0000 7f8156ffd700 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60513290 0x562e605156e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c171651cf197cb4 server_cookie=cb4269be3d522be7 in_seq=0 out_seq=0 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f813aff5700 1 -- 172.21.15.150:0/2306656478 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e604cc780 con 0x562e60513290 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f813aff5700 10 client.16353 handle_command_reply: tid=0 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 client.16353 shutdown 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fe570 msgr2=0x562e60507b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fe570 0x562e60507b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f814c0042e0 tx=0x7f814c0047c0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 shutdown_connections 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60513290 0x562e605156e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e6050eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f813aff5700 0 client.16353 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:57.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e60507610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2306656478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fe570 0x562e60507b50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 >> 172.21.15.150:0/2306656478 conn(0x562e6045fa60 msgr2=0x562e604f7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 shutdown_connections 2024-05-10T04:41:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.224+0000 7f816d6fab80 1 -- 172.21.15.150:0/2306656478 wait complete. 2024-05-10T04:41:57.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.225+0000 7f816d6fab80 20 client.16353 trim_cache size 0 max 16384 2024-05-10T04:41:57.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 Processor -- start 2024-05-10T04:41:57.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 -- start start 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e6050f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fe570 0x562e6050ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e6051a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6050fbb0 con 0x562e604feec0 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e604f9770 con 0x562e604fe570 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f816d6fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e60512f90 con 0x562e604fd6a0 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f81577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e6051a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f81577fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e6051a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43654/0 (socket says 172.21.15.150:43654) 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f81577fe700 1 -- 172.21.15.150:0/4167819151 learned_addr learned my addr 172.21.15.150:0/4167819151 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f8156ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e6050f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.238+0000 7f81567fc700 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fe570 0x562e6050ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f81577fe700 1 -- 172.21.15.150:0/4167819151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 msgr2=0x562e6050f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f81577fe700 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e6050f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.241 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f81577fe700 1 -- 172.21.15.150:0/4167819151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fe570 msgr2=0x562e6050ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f81577fe700 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fe570 0x562e6050ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f81577fe700 1 -- 172.21.15.150:0/4167819151 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e603e3560 con 0x562e604feec0 2024-05-10T04:41:57.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f81567fc700 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fe570 0x562e6050ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:57.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f81577fe700 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e6051a010 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8150007b20 tx=0x7f8150006020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dae5a70acc34e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f813dffb700 1 -- 172.21.15.150:0/4167819151 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150004c50 con 0x562e604feec0 2024-05-10T04:41:57.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f813dffb700 1 -- 172.21.15.150:0/4167819151 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8150009460 con 0x562e604feec0 2024-05-10T04:41:57.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.239+0000 7f813dffb700 1 -- 172.21.15.150:0/4167819151 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815001b690 con 0x562e604feec0 2024-05-10T04:41:57.246 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 -- 172.21.15.150:0/4167819151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 msgr2=0x562e6051a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e6051a010 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8150007b20 tx=0x7f8150006020 comp rx=0 tx=0).stop 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 -- 172.21.15.150:0/4167819151 shutdown_connections 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604feec0 0x562e6051a010 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fd6a0 0x562e6050f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 --2- 172.21.15.150:0/4167819151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604fe570 0x562e6050ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 -- 172.21.15.150:0/4167819151 >> 172.21.15.150:0/4167819151 conn(0x562e6045fa60 msgr2=0x562e605017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 -- 172.21.15.150:0/4167819151 shutdown_connections 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.240+0000 7f816d6fab80 1 -- 172.21.15.150:0/4167819151 wait complete. 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 -- start start 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 Processor -- start 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fd6a0 0x562e6050ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e6050bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e60567770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6050c670 con 0x562e604fd6a0 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e60567ce0 con 0x562e604feec0 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e60567ea0 con 0x562e604fe570 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f816d6fab80 10 client.0 _ll_register_callbacks cb 0x562e603c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f8156ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fd6a0 0x562e6050ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f8156ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fd6a0 0x562e6050ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43660/0 (socket says 172.21.15.150:43660) 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f8156ffd700 1 -- 172.21.15.150:0/2475506593 learned_addr learned my addr 172.21.15.150:0/2475506593 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f81567fc700 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e6050bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f81577fe700 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e60567770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f8156ffd700 1 -- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 msgr2=0x562e6050bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.241+0000 7f8156ffd700 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e6050bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f8156ffd700 1 -- 172.21.15.150:0/2475506593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 msgr2=0x562e60567770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f8156ffd700 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e60567770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f8156ffd700 1 -- 172.21.15.150:0/2475506593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e6029e3b0 con 0x562e604fd6a0 2024-05-10T04:41:57.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f8156ffd700 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fd6a0 0x562e6050ba40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8148009520 tx=0x7f81480997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1916bf46e474fae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f813aff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f813aff5700 1 -- 172.21.15.150:0/2475506593 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81480a1b70 con 0x562e604fd6a0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f813aff5700 1 -- 172.21.15.150:0/2475506593 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f81480b90b0 con 0x562e604fd6a0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8150007840 con 0x562e604fd6a0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f813aff5700 1 -- 172.21.15.150:0/2475506593 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81480108d0 con 0x562e604fd6a0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 10 client.16359 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562e602a2cd0 con 0x562e604fd6a0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f813aff5700 1 -- 172.21.15.150:0/2475506593 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8150007840 con 0x562e604fd6a0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f813aff5700 1 -- 172.21.15.150:0/2475506593 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f814809d150 con 0x562e604fd6a0 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 10 client.16359 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 10 client.16359 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 20 client.16359 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60569780 0x562e6056bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 4 client.16359 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:57.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.242+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e60415ec0 con 0x562e60569780 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f81567fc700 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60569780 0x562e6056bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f813aff5700 10 client.16359 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f81567fc700 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60569780 0x562e6056bba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d6ee0795eebc186 server_cookie=7e55250796919625 in_seq=0 out_seq=0 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f813aff5700 1 -- 172.21.15.150:0/2475506593 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e60415ec0 con 0x562e60569780 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f813aff5700 10 client.16359 handle_command_reply: tid=0 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f816d6fab80 1 client.16359 shutdown 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fd6a0 msgr2=0x562e6050ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.243+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fd6a0 0x562e6050ba40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8148009520 tx=0x7f81480997e0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 shutdown_connections 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x562e60569780 0x562e6056bba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562e604feec0 0x562e60567770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e604fd6a0 0x562e6050ba40 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f813aff5700 0 client.16359 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:41:57.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 --2- 172.21.15.150:0/2475506593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e604fe570 0x562e6050bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 >> 172.21.15.150:0/2475506593 conn(0x562e6045fa60 msgr2=0x562e605017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 shutdown_connections 2024-05-10T04:41:57.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 1 -- 172.21.15.150:0/2475506593 wait complete. 2024-05-10T04:41:57.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f816d6fab80 20 client.16359 trim_cache size 0 max 16384 2024-05-10T04:41:57.251 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f81400a11c0 msgr2=0x7f81400a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f81400a11c0 0x7f81400a36d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8158199090 tx=0x7f814c0073d0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158100db0 msgr2=0x7f8158197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158100db0 0x7f8158197ff0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f814800d980 tx=0x7f814800dc90 comp rx=0 tx=0).stop 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 shutdown_connections 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f81400a11c0 0x7f81400a36d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81581020a0 0x7f815819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8158100db0 0x7f8158197ff0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 --2- 172.21.15.150:0/1928213127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8158101700 0x7f8158198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 >> 172.21.15.150:0/1928213127 conn(0x7f81580fb8f0 msgr2=0x7f81580ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.244+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 shutdown_connections 2024-05-10T04:41:57.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.245+0000 7f815d3b6700 1 -- 172.21.15.150:0/1928213127 wait complete. 2024-05-10T04:41:57.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T04:41:57.408 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 -- 172.21.15.150:0/4279432521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f781025a0 msgr2=0x7f5f78108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/4279432521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f781025a0 0x7f5f78108e50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000b0a0 tx=0x7f5f7000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 -- 172.21.15.150:0/4279432521 shutdown_connections 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/4279432521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f781025a0 0x7f5f78108e50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/4279432521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f78100ae0 0x7f5f78100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/4279432521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f78101c00 0x7f5f78102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 -- 172.21.15.150:0/4279432521 >> 172.21.15.150:0/4279432521 conn(0x7f5f780fc0f0 msgr2=0x7f5f780fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 -- 172.21.15.150:0/4279432521 shutdown_connections 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 -- 172.21.15.150:0/4279432521 wait complete. 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.407+0000 7f5f7eedc700 1 Processor -- start 2024-05-10T04:41:57.409 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7eedc700 1 -- start start 2024-05-10T04:41:57.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7eedc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f78100ae0 0x7f5f78196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7eedc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78101c00 0x7f5f78197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7eedc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f781025a0 0x7f5f7819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7eedc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7819ea10 con 0x7f5f78101c00 2024-05-10T04:41:57.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7eedc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7819eb80 con 0x7f5f78100ae0 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7eedc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7819ed40 con 0x7f5f781025a0 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78101c00 0x7f5f78197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78101c00 0x7f5f78197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43676/0 (socket says 172.21.15.150:43676) 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 -- 172.21.15.150:0/3238466474 learned_addr learned my addr 172.21.15.150:0/3238466474 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7d479700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f781025a0 0x7f5f7819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f78100ae0 0x7f5f78196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 -- 172.21.15.150:0/3238466474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f781025a0 msgr2=0x7f5f7819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f781025a0 0x7f5f7819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 -- 172.21.15.150:0/3238466474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f78100ae0 msgr2=0x7f5f78196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f78100ae0 0x7f5f78196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 -- 172.21.15.150:0/3238466474 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f70009d00 con 0x7f5f78101c00 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.408+0000 7f5f77fff700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78101c00 0x7f5f78197470 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400d980 tx=0x7f5f6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.409+0000 7f5f75ffb700 1 -- 172.21.15.150:0/3238466474 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64004a30 con 0x7f5f78101c00 2024-05-10T04:41:57.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.409+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7819efc0 con 0x7f5f78101c00 2024-05-10T04:41:57.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.409+0000 7f5f75ffb700 1 -- 172.21.15.150:0/3238466474 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f640054e0 con 0x7f5f78101c00 2024-05-10T04:41:57.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.409+0000 7f5f75ffb700 1 -- 172.21.15.150:0/3238466474 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400fbc0 con 0x7f5f78101c00 2024-05-10T04:41:57.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.409+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7819f470 con 0x7f5f78101c00 2024-05-10T04:41:57.413 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.412+0000 7f5f75ffb700 1 -- 172.21.15.150:0/3238466474 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f5f6400fd20 con 0x7f5f78101c00 2024-05-10T04:41:57.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f75ffb700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5f680a11c0 0x7f5f680a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f75ffb700 1 -- 172.21.15.150:0/3238466474 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f5f640cca50 con 0x7f5f78101c00 2024-05-10T04:41:57.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5f680a11c0 0x7f5f680a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f8f220b80 1 Processor -- start 2024-05-10T04:41:57.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f8f220b80 1 -- start start 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd6bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5f680a11c0 0x7f5f680a36d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f78101b00 tx=0x7f5f6c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd6ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd73c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.413+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fc8e150 con 0x560f5fd6bb50 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fc8bb10 con 0x560f5fd6d370 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fc87400 con 0x560f5fd6ca20 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd6bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd6bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43680/0 (socket says 172.21.15.150:43680) 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3629439686 learned_addr learned my addr 172.21.15.150:0/3629439686 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f77fff700 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd6ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7d479700 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd73c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3629439686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 msgr2=0x560f5fd6ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd6ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3629439686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 msgr2=0x560f5fd73c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd73c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3629439686 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f5fb083b0 con 0x560f5fd6bb50 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7d479700 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd73c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd6bf80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c010e20 tx=0x7f5f6c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e615579f6e44c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f62ffd700 1 -- 172.21.15.150:0/3629439686 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004220 con 0x560f5fd6bb50 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f62ffd700 1 -- 172.21.15.150:0/3629439686 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f6c015070 con 0x560f5fd6bb50 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.414+0000 7f5f62ffd700 1 -- 172.21.15.150:0/3629439686 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c01e680 con 0x560f5fd6bb50 2024-05-10T04:41:57.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3629439686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 msgr2=0x560f5fd6bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd6bf80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c010e20 tx=0x7f5f6c013000 comp rx=0 tx=0).stop 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3629439686 shutdown_connections 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd73c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd6bf80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3629439686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd6ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3629439686 >> 172.21.15.150:0/3629439686 conn(0x560f5fcc9a60 msgr2=0x560f5fd61fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3629439686 shutdown_connections 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.415+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3629439686 wait complete. 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 -- start start 2024-05-10T04:41:57.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 Processor -- start 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd75b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd76060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fc23560 con 0x560f5fd6bb50 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fc23200 con 0x560f5fd6d370 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fb0a6a0 con 0x560f5fd6ca20 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f8f220b80 10 client.0 _ll_register_callbacks cb 0x560f5fc2cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.416+0000 7f5f7cc78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd75b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd75b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43690/0 (socket says 172.21.15.150:43690) 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3804089053 learned_addr learned my addr 172.21.15.150:0/3804089053 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f77fff700 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd76060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7d479700 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3804089053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 msgr2=0x560f5fd76060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd76060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 msgr2=0x560f5fd7cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 -- 172.21.15.150:0/3804089053 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c010b00 con 0x560f5fd6bb50 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd75b20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0038e0 tx=0x7f5f6c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b0d914243549dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f57fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f57fff700 1 -- 172.21.15.150:0/3804089053 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c005300 con 0x560f5fd6bb50 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f57fff700 1 -- 172.21.15.150:0/3804089053 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f5f6c017070 con 0x560f5fd6bb50 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.417+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f5fbe0270 con 0x560f5fd6bb50 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f57fff700 1 -- 172.21.15.150:0/3804089053 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f6c003ab0 con 0x560f5fd6bb50 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 10 client.16371 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560f5fab4b60 con 0x560f5fd6bb50 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f57fff700 1 -- 172.21.15.150:0/3804089053 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f5fbe0270 con 0x560f5fd6bb50 2024-05-10T04:41:57.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f57fff700 1 -- 172.21.15.150:0/3804089053 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f6c024070 con 0x560f5fd6bb50 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 10 client.16371 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 20 client.16371 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fd816b0 0x560f5fd83b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 4 client.16371 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f5fd36780 con 0x560f5fd816b0 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.418+0000 7f5f77fff700 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fd816b0 0x560f5fd83b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.419+0000 7f5f57fff700 10 client.16371 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.419+0000 7f5f77fff700 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fd816b0 0x560f5fd83b00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb2da4ea76987879 server_cookie=7b665e4b0dd96980 in_seq=0 out_seq=0 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.419+0000 7f5f57fff700 1 -- 172.21.15.150:0/3804089053 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f5fd36780 con 0x560f5fd816b0 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.419+0000 7f5f57fff700 10 client.16371 handle_command_reply: tid=0 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 client.16371 shutdown 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 msgr2=0x560f5fd75b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd75b20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0038e0 tx=0x7f5f6c003910 comp rx=0 tx=0).stop 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 shutdown_connections 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fd816b0 0x560f5fd83b00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f57fff700 0 client.16371 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd75b20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/3804089053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd76060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 >> 172.21.15.150:0/3804089053 conn(0x560f5fcc9a60 msgr2=0x560f5fd61fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 shutdown_connections 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 1 -- 172.21.15.150:0/3804089053 wait complete. 2024-05-10T04:41:57.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.420+0000 7f5f8f220b80 20 client.16371 trim_cache size 0 max 16384 2024-05-10T04:41:57.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 Processor -- start 2024-05-10T04:41:57.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 -- start start 2024-05-10T04:41:57.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6bb50 0x560f5fd83ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6ca20 0x560f5fd7cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6d370 0x560f5fd7d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fd62ad0 con 0x560f5fd6ca20 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fd6c3e0 con 0x560f5fd6bb50 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fd73b90 con 0x560f5fd6d370 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6ca20 0x560f5fd7cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6ca20 0x560f5fd7cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43704/0 (socket says 172.21.15.150:43704) 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f77fff700 1 -- 172.21.15.150:0/2757042296 learned_addr learned my addr 172.21.15.150:0/2757042296 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6bb50 0x560f5fd83ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.434+0000 7f5f7d479700 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6d370 0x560f5fd7d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.435+0000 7f5f7cc78700 1 -- 172.21.15.150:0/2757042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6d370 msgr2=0x560f5fd7d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.435+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6d370 0x560f5fd7d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.435+0000 7f5f7cc78700 1 -- 172.21.15.150:0/2757042296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6ca20 msgr2=0x560f5fd7cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.437 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.435+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6ca20 0x560f5fd7cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.435+0000 7f5f7cc78700 1 -- 172.21.15.150:0/2757042296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f5fc4d560 con 0x560f5fd6bb50 2024-05-10T04:41:57.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.435+0000 7f5f77fff700 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6ca20 0x560f5fd7cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:57.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.435+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6bb50 0x560f5fd83ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c026f50 tx=0x7f5f6c0079b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ccc012717a5e88 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.437+0000 7f5f637fe700 1 -- 172.21.15.150:0/2757042296 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003980 con 0x560f5fd6bb50 2024-05-10T04:41:57.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.437+0000 7f5f637fe700 1 -- 172.21.15.150:0/2757042296 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f6c003ae0 con 0x560f5fd6bb50 2024-05-10T04:41:57.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.437+0000 7f5f637fe700 1 -- 172.21.15.150:0/2757042296 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c026440 con 0x560f5fd6bb50 2024-05-10T04:41:57.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2757042296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6bb50 msgr2=0x560f5fd83ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6bb50 0x560f5fd83ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c026f50 tx=0x7f5f6c0079b0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2757042296 shutdown_connections 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6d370 0x560f5fd7d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6bb50 0x560f5fd83ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2757042296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6ca20 0x560f5fd7cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2757042296 >> 172.21.15.150:0/2757042296 conn(0x560f5fcc9a60 msgr2=0x560f5fd61fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2757042296 shutdown_connections 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2757042296 wait complete. 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 -- start start 2024-05-10T04:41:57.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.438+0000 7f5f8f220b80 1 Processor -- start 2024-05-10T04:41:57.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd7a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd7a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f8f220b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fd7bd90 con 0x560f5fd6bb50 2024-05-10T04:41:57.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fd7bf00 con 0x560f5fd6d370 2024-05-10T04:41:57.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f8f220b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5fd7c0c0 con 0x560f5fd6ca20 2024-05-10T04:41:57.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f8f220b80 10 client.0 _ll_register_callbacks cb 0x560f5fc2cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f7cc78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd7a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd7a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f7d479700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f77fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd7a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:36032/0 (socket says 172.21.15.150:36032) 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f7cc78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd7a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43714/0 (socket says 172.21.15.150:43714) 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.439+0000 7f5f77fff700 1 -- 172.21.15.150:0/2378549038 learned_addr learned my addr 172.21.15.150:0/2378549038 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f77fff700 1 -- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 msgr2=0x560f5fd7b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f77fff700 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f77fff700 1 -- 172.21.15.150:0/2378549038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 msgr2=0x560f5fd7a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f77fff700 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd7a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f77fff700 1 -- 172.21.15.150:0/2378549038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7819f470 con 0x560f5fd6ca20 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f7d479700 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f77fff700 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd7a920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f64096570 tx=0x7f5f6400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99bd5e48d7ff0558 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.440+0000 7f5f57fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f57fff700 1 -- 172.21.15.150:0/2378549038 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6409d930 con 0x560f5fd6ca20 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f57fff700 1 -- 172.21.15.150:0/2378549038 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f5f640a1f80 con 0x560f5fd6ca20 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f6c015070 con 0x560f5fd6ca20 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f57fff700 1 -- 172.21.15.150:0/2378549038 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f64005650 con 0x560f5fd6ca20 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f8f220b80 10 client.25912 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560f5fab4b60 con 0x560f5fd6ca20 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f57fff700 1 -- 172.21.15.150:0/2378549038 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f6407c020 con 0x560f5fd6ca20 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f57fff700 1 -- 172.21.15.150:0/2378549038 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5f64098ff0 con 0x560f5fd6ca20 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f8f220b80 10 client.25912 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f8f220b80 10 client.25912 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.441+0000 7f5f8f220b80 20 client.25912 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fdd7f70 0x560f5fdda390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f8f220b80 4 client.25912 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f5fc7fec0 con 0x560f5fdd7f70 2024-05-10T04:41:57.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fdd7f70 0x560f5fdda390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f57fff700 10 client.25912 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:57.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f7cc78700 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fdd7f70 0x560f5fdda390 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d741064c7f28730 server_cookie=b7c65b64a01da23f in_seq=0 out_seq=0 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f57fff700 1 -- 172.21.15.150:0/2378549038 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f5fc7fec0 con 0x560f5fdd7f70 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f57fff700 10 client.25912 handle_command_reply: tid=0 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.442+0000 7f5f8f220b80 1 client.25912 shutdown 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 msgr2=0x560f5fd7a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd7a920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f64096570 tx=0x7f5f6400f690 comp rx=0 tx=0).stop 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 shutdown_connections 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x560f5fdd7f70 0x560f5fdda390 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f5fd6d370 0x560f5fd7b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f57fff700 0 client.25912 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f5fd6bb50 0x560f5fd7a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 --2- 172.21.15.150:0/2378549038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f5fd6ca20 0x560f5fd7a920 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 >> 172.21.15.150:0/2378549038 conn(0x560f5fcc9a60 msgr2=0x560f5fd7d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 shutdown_connections 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 1 -- 172.21.15.150:0/2378549038 wait complete. 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f8f220b80 20 client.25912 trim_cache size 0 max 16384 2024-05-10T04:41:57.448 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5f680a11c0 msgr2=0x7f5f680a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5f680a11c0 0x7f5f680a36d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f78101b00 tx=0x7f5f6c007400 comp rx=0 tx=0).stop 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78101c00 msgr2=0x7f5f78197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.443+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78101c00 0x7f5f78197470 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400d980 tx=0x7f5f6400dc90 comp rx=0 tx=0).stop 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 shutdown_connections 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f5f680a11c0 0x7f5f680a36d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f781025a0 0x7f5f7819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f78100ae0 0x7f5f78196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 --2- 172.21.15.150:0/3238466474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f78101c00 0x7f5f78197470 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 >> 172.21.15.150:0/3238466474 conn(0x7f5f780fc0f0 msgr2=0x7f5f78104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 shutdown_connections 2024-05-10T04:41:57.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.444+0000 7f5f7eedc700 1 -- 172.21.15.150:0/3238466474 wait complete. 2024-05-10T04:41:57.458 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:41:57.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.602+0000 7f280a12f700 1 -- 172.21.15.150:0/4056042163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28041026b0 msgr2=0x7f2804104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.602+0000 7f280a12f700 1 --2- 172.21.15.150:0/4056042163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28041026b0 0x7f2804104ad0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009990 tx=0x7f27f4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.602+0000 7f280a12f700 1 -- 172.21.15.150:0/4056042163 shutdown_connections 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.602+0000 7f280a12f700 1 --2- 172.21.15.150:0/4056042163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2804107970 0x7f2804109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.602+0000 7f280a12f700 1 --2- 172.21.15.150:0/4056042163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28041026b0 0x7f2804104ad0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.602+0000 7f280a12f700 1 --2- 172.21.15.150:0/4056042163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2804105010 0x7f2804107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.602+0000 7f280a12f700 1 -- 172.21.15.150:0/4056042163 >> 172.21.15.150:0/4056042163 conn(0x7f28040fc0d0 msgr2=0x7f28040fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.603+0000 7f280a12f700 1 -- 172.21.15.150:0/4056042163 shutdown_connections 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.603+0000 7f280a12f700 1 -- 172.21.15.150:0/4056042163 wait complete. 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.603+0000 7f280a12f700 1 Processor -- start 2024-05-10T04:41:57.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.603+0000 7f280a12f700 1 -- start start 2024-05-10T04:41:57.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f280a12f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28041026b0 0x7f2804197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f280a12f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2804105010 0x7f2804198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f280a12f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2804107970 0x7f280419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f280a12f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280419f9c0 con 0x7f2804105010 2024-05-10T04:41:57.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f280a12f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280419fb30 con 0x7f28041026b0 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f280a12f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280419fcf0 con 0x7f2804107970 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2804105010 0x7f2804198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2804105010 0x7f2804198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43736/0 (socket says 172.21.15.150:43736) 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 -- 172.21.15.150:0/259882518 learned_addr learned my addr 172.21.15.150:0/259882518 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2803fff700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2804107970 0x7f280419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f28037fe700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28041026b0 0x7f2804197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 -- 172.21.15.150:0/259882518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2804107970 msgr2=0x7f280419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2804107970 0x7f280419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 -- 172.21.15.150:0/259882518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28041026b0 msgr2=0x7f2804197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28041026b0 0x7f2804197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 -- 172.21.15.150:0/259882518 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f40095f0 con 0x7f2804105010 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f28037fe700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28041026b0 0x7f2804197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2802ffd700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2804105010 0x7f2804198420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9a0 tx=0x7f27f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f2800ff9700 1 -- 172.21.15.150:0/259882518 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8009790 con 0x7f2804105010 2024-05-10T04:41:57.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.604+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280419ffd0 con 0x7f2804105010 2024-05-10T04:41:57.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.605+0000 7f2800ff9700 1 -- 172.21.15.150:0/259882518 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f800cbd0 con 0x7f2804105010 2024-05-10T04:41:57.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.605+0000 7f2800ff9700 1 -- 172.21.15.150:0/259882518 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8018680 con 0x7f2804105010 2024-05-10T04:41:57.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.605+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28041a0450 con 0x7f2804105010 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.607+0000 7f2800ff9700 1 -- 172.21.15.150:0/259882518 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f27f80187e0 con 0x7f2804105010 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.608+0000 7f2800ff9700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27ec0a11c0 0x7f27ec0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.608+0000 7f2800ff9700 1 -- 172.21.15.150:0/259882518 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f27f8014070 con 0x7f2804105010 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.608+0000 7f28037fe700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27ec0a11c0 0x7f27ec0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.608+0000 7f28037fe700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27ec0a11c0 0x7f27ec0a36d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f27f400b2e0 tx=0x7f27f401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.608+0000 7f281a473b80 1 Processor -- start 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f281a473b80 1 -- start start 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f281a473b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f76af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f281a473b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f77980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f281a473b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f7e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f281a473b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86e9d150 con 0x55ae86f77ec0 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f281a473b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86e9ab10 con 0x55ae86f766a0 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f281a473b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86e96400 con 0x55ae86f77570 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2803fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f7e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f77980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2803fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f7e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43752/0 (socket says 172.21.15.150:43752) 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2803fff700 1 -- 172.21.15.150:0/3172828441 learned_addr learned my addr 172.21.15.150:0/3172828441 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f28037fe700 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f76af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2802ffd700 1 -- 172.21.15.150:0/3172828441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 msgr2=0x55ae86f76af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2802ffd700 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f76af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2802ffd700 1 -- 172.21.15.150:0/3172828441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 msgr2=0x55ae86f7e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2802ffd700 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f7e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.609+0000 7f2802ffd700 1 -- 172.21.15.150:0/3172828441 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae86d173b0 con 0x55ae86f77570 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.610+0000 7f2802ffd700 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f77980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27f80ca670 tx=0x7f27f8099920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d90bf8d3ddd37cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.610+0000 7f27e9ffb700 1 -- 172.21.15.150:0/3172828441 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8099fc0 con 0x55ae86f77570 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.610+0000 7f27e9ffb700 1 -- 172.21.15.150:0/3172828441 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f80ade90 con 0x55ae86f77570 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.610+0000 7f27e9ffb700 1 -- 172.21.15.150:0/3172828441 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8099710 con 0x55ae86f77570 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 -- 172.21.15.150:0/3172828441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 msgr2=0x55ae86f77980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f77980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27f80ca670 tx=0x7f27f8099920 comp rx=0 tx=0).stop 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 -- 172.21.15.150:0/3172828441 shutdown_connections 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f7e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f76af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 --2- 172.21.15.150:0/3172828441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f77980 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 -- 172.21.15.150:0/3172828441 >> 172.21.15.150:0/3172828441 conn(0x55ae86ed8a60 msgr2=0x55ae86f70fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 -- 172.21.15.150:0/3172828441 shutdown_connections 2024-05-10T04:41:57.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 -- 172.21.15.150:0/3172828441 wait complete. 2024-05-10T04:41:57.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 -- start start 2024-05-10T04:41:57.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.611+0000 7f281a473b80 1 Processor -- start 2024-05-10T04:41:57.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f281a473b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f766a0 0x55ae86f806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f281a473b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77570 0x55ae86f80c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f281a473b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f281a473b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86e32560 con 0x55ae86f77ec0 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f281a473b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86e32200 con 0x55ae86f77570 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f281a473b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86d196a0 con 0x55ae86f766a0 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f281a473b80 10 client.0 _ll_register_callbacks cb 0x55ae86e3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77570 0x55ae86f80c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f2803fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77570 0x55ae86f80c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:54280/0 (socket says 172.21.15.150:54280) 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f2802ffd700 1 -- 172.21.15.150:0/3801161915 learned_addr learned my addr 172.21.15.150:0/3801161915 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f28037fe700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f766a0 0x55ae86f806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f2803fff700 1 -- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f766a0 msgr2=0x55ae86f806e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f2803fff700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f766a0 0x55ae86f806e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.612+0000 7f2803fff700 1 -- 172.21.15.150:0/3801161915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77570 msgr2=0x55ae86f80c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f2803fff700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77570 0x55ae86f80c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f2803fff700 1 -- 172.21.15.150:0/3801161915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28041a0450 con 0x55ae86f77ec0 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f28037fe700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f766a0 0x55ae86f806e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f2802ffd700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77570 0x55ae86f80c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f2803fff700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f87b30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009fc0 tx=0x7f27fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46c1c2bd9ad1ceb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f27e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f27e2ffd700 1 -- 172.21.15.150:0/3801161915 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc004db0 con 0x55ae86f77ec0 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f27e2ffd700 1 -- 172.21.15.150:0/3801161915 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f27fc007500 con 0x55ae86f77ec0 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae86def270 con 0x55ae86f77ec0 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f27e2ffd700 1 -- 172.21.15.150:0/3801161915 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27fc0076c0 con 0x55ae86f77ec0 2024-05-10T04:41:57.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f281a473b80 10 client.16389 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ae86cc3b60 con 0x55ae86f77ec0 2024-05-10T04:41:57.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f27e2ffd700 1 -- 172.21.15.150:0/3801161915 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae86def270 con 0x55ae86f77ec0 2024-05-10T04:41:57.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.613+0000 7f27e2ffd700 1 -- 172.21.15.150:0/3801161915 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27fc016ab0 con 0x55ae86f77ec0 2024-05-10T04:41:57.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f281a473b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f281a473b80 10 client.16389 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:41:57.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f281a473b80 20 client.16389 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f281a473b80 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86f90750 0x55ae86f92ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f281a473b80 4 client.16389 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae86f45780 con 0x55ae86f90750 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f28037fe700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86f90750 0x55ae86f92ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f27e2ffd700 10 client.16389 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.614+0000 7f28037fe700 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86f90750 0x55ae86f92ba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4ef40cf1b5665cd server_cookie=8c32458b40209687 in_seq=0 out_seq=0 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f27e2ffd700 1 -- 172.21.15.150:0/3801161915 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae86f45780 con 0x55ae86f90750 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f27e2ffd700 10 client.16389 handle_command_reply: tid=0 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 client.16389 shutdown 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 msgr2=0x55ae86f87b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f87b30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009fc0 tx=0x7f27fc004040 comp rx=0 tx=0).stop 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 shutdown_connections 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86f90750 0x55ae86f92ba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77ec0 0x55ae86f87b30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f766a0 0x55ae86f806e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 --2- 172.21.15.150:0/3801161915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77570 0x55ae86f80c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 >> 172.21.15.150:0/3801161915 conn(0x55ae86ed8a60 msgr2=0x55ae86f70fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f27e2ffd700 0 client.16389 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 shutdown_connections 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.615+0000 7f281a473b80 1 -- 172.21.15.150:0/3801161915 wait complete. 2024-05-10T04:41:57.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.616+0000 7f281a473b80 20 client.16389 trim_cache size 0 max 16384 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 Processor -- start 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 -- start start 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f766a0 0x55ae86f7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77ec0 0x55ae86f7b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86f90450 con 0x55ae86f766a0 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86f7ae20 con 0x55ae86f77ec0 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f281a473b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86f97bc0 con 0x55ae86f77570 2024-05-10T04:41:57.632 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f28037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f766a0 0x55ae86f7b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f28037fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f766a0 0x55ae86f7b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43770/0 (socket says 172.21.15.150:43770) 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.629+0000 7f28037fe700 1 -- 172.21.15.150:0/879135473 learned_addr learned my addr 172.21.15.150:0/879135473 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f2802ffd700 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f2803fff700 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77ec0 0x55ae86f7b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f28037fe700 1 -- 172.21.15.150:0/879135473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 msgr2=0x55ae86f879e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f28037fe700 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f879e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f28037fe700 1 -- 172.21.15.150:0/879135473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77ec0 msgr2=0x55ae86f7b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f28037fe700 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77ec0 0x55ae86f7b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f28037fe700 1 -- 172.21.15.150:0/879135473 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae86e5c560 con 0x55ae86f766a0 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f28037fe700 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f766a0 0x55ae86f7b390 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009990 tx=0x7f27f401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3b33b29e93a582 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f27ea7fc700 1 -- 172.21.15.150:0/879135473 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f401d040 con 0x55ae86f766a0 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f27ea7fc700 1 -- 172.21.15.150:0/879135473 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f402b070 con 0x55ae86f766a0 2024-05-10T04:41:57.633 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.630+0000 7f27ea7fc700 1 -- 172.21.15.150:0/879135473 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f40236f0 con 0x55ae86f766a0 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 -- 172.21.15.150:0/879135473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f766a0 msgr2=0x55ae86f7b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f766a0 0x55ae86f7b390 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009990 tx=0x7f27f401e000 comp rx=0 tx=0).stop 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 -- 172.21.15.150:0/879135473 shutdown_connections 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f77ec0 0x55ae86f7b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f766a0 0x55ae86f7b390 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 --2- 172.21.15.150:0/879135473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77570 0x55ae86f879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 -- 172.21.15.150:0/879135473 >> 172.21.15.150:0/879135473 conn(0x55ae86ed8a60 msgr2=0x55ae86f974b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 -- 172.21.15.150:0/879135473 shutdown_connections 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 -- 172.21.15.150:0/879135473 wait complete. 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.631+0000 7f281a473b80 1 -- start start 2024-05-10T04:41:57.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 1 Processor -- start 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f84d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77570 0x55ae86f852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77ec0 0x55ae86f86200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86f86740 con 0x55ae86f77570 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86f868b0 con 0x55ae86f766a0 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae86f86a70 con 0x55ae86f77ec0 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f281a473b80 10 client.0 _ll_register_callbacks cb 0x55ae86e3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77570 0x55ae86f852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f2803fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77ec0 0x55ae86f86200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f28037fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f84d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77570 0x55ae86f852d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43780/0 (socket says 172.21.15.150:43780) 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.632+0000 7f2802ffd700 1 -- 172.21.15.150:0/2867484668 learned_addr learned my addr 172.21.15.150:0/2867484668 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f2803fff700 1 -- 172.21.15.150:0/2867484668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 msgr2=0x55ae86f84d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f2803fff700 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f84d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f2803fff700 1 -- 172.21.15.150:0/2867484668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77570 msgr2=0x55ae86f852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f2803fff700 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77570 0x55ae86f852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f2803fff700 1 -- 172.21.15.150:0/2867484668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280410a9b0 con 0x55ae86f77ec0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f2802ffd700 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77570 0x55ae86f852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f2803fff700 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77ec0 0x55ae86f86200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55ae86f81610 tx=0x7f27fc004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f9dd226265dca server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f27e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f27e2ffd700 1 -- 172.21.15.150:0/2867484668 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc016840 con 0x55ae86f77ec0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f27e2ffd700 1 -- 172.21.15.150:0/2867484668 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f27fc0169a0 con 0x55ae86f77ec0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27f80ade90 con 0x55ae86f77ec0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f27e2ffd700 1 -- 172.21.15.150:0/2867484668 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27fc015390 con 0x55ae86f77ec0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f281a473b80 10 client.25927 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ae86d1bcd0 con 0x55ae86f77ec0 2024-05-10T04:41:57.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.633+0000 7f27e2ffd700 1 -- 172.21.15.150:0/2867484668 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f80ade90 con 0x55ae86f77ec0 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f27e2ffd700 1 -- 172.21.15.150:0/2867484668 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27fc007500 con 0x55ae86f77ec0 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f281a473b80 10 client.25927 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f281a473b80 10 client.25927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f281a473b80 20 client.25927 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f281a473b80 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86fe6f00 0x55ae86fe9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f281a473b80 4 client.25927 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae86e8eec0 con 0x55ae86fe6f00 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f28037fe700 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86fe6f00 0x55ae86fe9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f27e2ffd700 10 client.25927 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.634+0000 7f28037fe700 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86fe6f00 0x55ae86fe9320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d79e5a825c36529 server_cookie=5637d90b8036fedc in_seq=0 out_seq=0 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f27e2ffd700 1 -- 172.21.15.150:0/2867484668 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae86e8eec0 con 0x55ae86fe6f00 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f27e2ffd700 10 client.25927 handle_command_reply: tid=0 2024-05-10T04:41:57.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 client.25927 shutdown 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77ec0 msgr2=0x55ae86f86200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77ec0 0x55ae86f86200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55ae86f81610 tx=0x7f27fc004cf0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 shutdown_connections 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x55ae86fe6f00 0x55ae86fe9320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ae86f77ec0 0x55ae86f86200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae86f766a0 0x55ae86f84d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 --2- 172.21.15.150:0/2867484668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae86f77570 0x55ae86f852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f27e2ffd700 0 client.25927 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 >> 172.21.15.150:0/2867484668 conn(0x55ae86ed8a60 msgr2=0x55ae86f974b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 shutdown_connections 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 1 -- 172.21.15.150:0/2867484668 wait complete. 2024-05-10T04:41:57.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.635+0000 7f281a473b80 20 client.25927 trim_cache size 0 max 16384 2024-05-10T04:41:57.643 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27ec0a11c0 msgr2=0x7f27ec0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27ec0a11c0 0x7f27ec0a36d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f27f400b2e0 tx=0x7f27f401a040 comp rx=0 tx=0).stop 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2804105010 msgr2=0x7f2804198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2804105010 0x7f2804198420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9a0 tx=0x7f27f800ecb0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 shutdown_connections 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f27ec0a11c0 0x7f27ec0a36d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2804107970 0x7f280419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28041026b0 0x7f2804197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 --2- 172.21.15.150:0/259882518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2804105010 0x7f2804198420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 >> 172.21.15.150:0/259882518 conn(0x7f28040fc0d0 msgr2=0x7f28041035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 shutdown_connections 2024-05-10T04:41:57.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.636+0000 7f280a12f700 1 -- 172.21.15.150:0/259882518 wait complete. 2024-05-10T04:41:57.651 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:41:57.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 -- 172.21.15.150:0/563647094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c1079a0 msgr2=0x7f2a2c109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 --2- 172.21.15.150:0/563647094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c1079a0 0x7f2a2c109dc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400b0a0 tx=0x7f2a2400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 -- 172.21.15.150:0/563647094 shutdown_connections 2024-05-10T04:41:57.798 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 --2- 172.21.15.150:0/563647094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c1079a0 0x7f2a2c109dc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.799 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 --2- 172.21.15.150:0/563647094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a2c1026e0 0x7f2a2c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 --2- 172.21.15.150:0/563647094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c105040 0x7f2a2c107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 -- 172.21.15.150:0/563647094 >> 172.21.15.150:0/563647094 conn(0x7f2a2c0fc100 msgr2=0x7f2a2c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 -- 172.21.15.150:0/563647094 shutdown_connections 2024-05-10T04:41:57.800 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.796+0000 7f2a3318b700 1 -- 172.21.15.150:0/563647094 wait complete. 2024-05-10T04:41:57.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.797+0000 7f2a3318b700 1 Processor -- start 2024-05-10T04:41:57.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.797+0000 7f2a3318b700 1 -- start start 2024-05-10T04:41:57.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.797+0000 7f2a3318b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1026e0 0x7f2a2c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a3318b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c105040 0x7f2a2c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a3318b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a2c1079a0 0x7f2a2c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a3318b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c19fae0 con 0x7f2a2c105040 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a3318b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c19fc50 con 0x7f2a2c1026e0 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a3318b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c19fe10 con 0x7f2a2c1079a0 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c105040 0x7f2a2c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c105040 0x7f2a2c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43804/0 (socket says 172.21.15.150:43804) 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2814403729 learned_addr learned my addr 172.21.15.150:0/2814403729 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a31728700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a2c1079a0 0x7f2a2c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a30f27700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1026e0 0x7f2a2c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2814403729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a2c1079a0 msgr2=0x7f2a2c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a2c1079a0 0x7f2a2c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2814403729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1026e0 msgr2=0x7f2a2c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1026e0 0x7f2a2c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2814403729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24009d00 con 0x7f2a2c105040 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a30f27700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1026e0 0x7f2a2c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c105040 0x7f2a2c198540 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800e9a0 tx=0x7f2a1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.798+0000 7f2a29ffb700 1 -- 172.21.15.150:0/2814403729 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18009790 con 0x7f2a2c105040 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.799+0000 7f2a29ffb700 1 -- 172.21.15.150:0/2814403729 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a1800cbd0 con 0x7f2a2c105040 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.799+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c1a00f0 con 0x7f2a2c105040 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.799+0000 7f2a29ffb700 1 -- 172.21.15.150:0/2814403729 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18018680 con 0x7f2a2c105040 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.799+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c1a0600 con 0x7f2a2c105040 2024-05-10T04:41:57.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.801+0000 7f2a29ffb700 1 -- 172.21.15.150:0/2814403729 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f2a180187e0 con 0x7f2a2c105040 2024-05-10T04:41:57.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.802+0000 7f2a29ffb700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2a1c0a11c0 0x7f2a1c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.802+0000 7f2a29ffb700 1 -- 172.21.15.150:0/2814403729 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2a18014070 con 0x7f2a2c105040 2024-05-10T04:41:57.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.802+0000 7f2a30f27700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2a1c0a11c0 0x7f2a1c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.802+0000 7f2a30f27700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2a1c0a11c0 0x7f2a1c0a36d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c06a1e0 tx=0x7f2a20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 Processor -- start 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 -- start start 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95d4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d5ec0 0x5634a95dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a94fb150 con 0x5634a95d46a0 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a94f8b10 con 0x5634a95d5570 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a94f4400 con 0x5634a95d5ec0 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95d4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95d4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43806/0 (socket says 172.21.15.150:43806) 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 -- 172.21.15.150:0/3911264651 learned_addr learned my addr 172.21.15.150:0/3911264651 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a31728700 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d5ec0 0x5634a95dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 -- 172.21.15.150:0/3911264651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d5ec0 msgr2=0x5634a95dc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d5ec0 0x5634a95dc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 -- 172.21.15.150:0/3911264651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 msgr2=0x5634a95d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.803+0000 7f2a30f27700 1 -- 172.21.15.150:0/3911264651 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a93753b0 con 0x5634a95d46a0 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.804+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95d4af0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2a20011f60 tx=0x7f2a20014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d36f964d3b5cb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.804+0000 7f2a16ffd700 1 -- 172.21.15.150:0/3911264651 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20003680 con 0x5634a95d46a0 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.804+0000 7f2a16ffd700 1 -- 172.21.15.150:0/3911264651 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a20003ca0 con 0x5634a95d46a0 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.804+0000 7f2a16ffd700 1 -- 172.21.15.150:0/3911264651 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20012c60 con 0x5634a95d46a0 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3911264651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 msgr2=0x5634a95d4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95d4af0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2a20011f60 tx=0x7f2a20014000 comp rx=0 tx=0).stop 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3911264651 shutdown_connections 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d5ec0 0x5634a95dc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95d4af0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3911264651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3911264651 >> 172.21.15.150:0/3911264651 conn(0x5634a9536a60 msgr2=0x5634a95cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3911264651 shutdown_connections 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3911264651 wait complete. 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 -- start start 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.805+0000 7f2a434cfb80 1 Processor -- start 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d46a0 0x5634a95de700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95dec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d5ec0 0x5634a95e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9490560 con 0x5634a95d5ec0 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9490200 con 0x5634a95d5570 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a93776a0 con 0x5634a95d46a0 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a434cfb80 10 client.0 _ll_register_callbacks cb 0x5634a9499b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a30f27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d46a0 0x5634a95de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95dec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:54328/0 (socket says 172.21.15.150:54328) 2024-05-10T04:41:57.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a30f27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d46a0 0x5634a95de700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:36126/0 (socket says 172.21.15.150:36126) 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a2bfff700 1 -- 172.21.15.150:0/3263449154 learned_addr learned my addr 172.21.15.150:0/3263449154 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a31728700 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d5ec0 0x5634a95e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a30f27700 1 -- 172.21.15.150:0/3263449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 msgr2=0x5634a95dec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95dec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a30f27700 1 -- 172.21.15.150:0/3263449154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d5ec0 msgr2=0x5634a95e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d5ec0 0x5634a95e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a30f27700 1 -- 172.21.15.150:0/3263449154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a20011c60 con 0x5634a95d46a0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.806+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95dec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d46a0 0x5634a95de700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a20023040 tx=0x7f2a20012c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bbec211c276d19 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3263449154 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20003a90 con 0x5634a95d46a0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3263449154 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2a20018070 con 0x5634a95d46a0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634a944d270 con 0x5634a95d46a0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3263449154 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a20005040 con 0x5634a95d46a0 2024-05-10T04:41:57.815 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a434cfb80 10 client.25939 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5634a9321b60 con 0x5634a95d46a0 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3263449154 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634a944d270 con 0x5634a95d46a0 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.807+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3263449154 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a200049a0 con 0x5634a95d46a0 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a434cfb80 10 client.25939 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a434cfb80 10 client.25939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a434cfb80 20 client.25939 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95ea280 0x5634a95ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a434cfb80 4 client.25939 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634a95a3780 con 0x5634a95ea280 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95ea280 0x5634a95ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a0bfff700 10 client.25939 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.808+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95ea280 0x5634a95ec6a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcaa1508d1f12270 server_cookie=b43087d3eb8b854d in_seq=0 out_seq=0 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3263449154 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634a95a3780 con 0x5634a95ea280 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a0bfff700 10 client.25939 handle_command_reply: tid=0 2024-05-10T04:41:57.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 client.25939 shutdown 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d46a0 msgr2=0x5634a95de700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d46a0 0x5634a95de700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a20023040 tx=0x7f2a20012c90 comp rx=0 tx=0).stop 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 shutdown_connections 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95ea280 0x5634a95ec6a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d5ec0 0x5634a95e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a95d46a0 0x5634a95de700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a0bfff700 0 client.25939 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3263449154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95dec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 >> 172.21.15.150:0/3263449154 conn(0x5634a9536a60 msgr2=0x5634a95cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 shutdown_connections 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.809+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3263449154 wait complete. 2024-05-10T04:41:57.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.810+0000 7f2a434cfb80 20 client.25939 trim_cache size 0 max 16384 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 Processor -- start 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 -- start start 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a95dc6e0 con 0x5634a95d46a0 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a95d5240 con 0x5634a95d5570 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a95e6400 con 0x5634a9536a60 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43824/0 (socket says 172.21.15.150:43824) 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.823+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2461287539 learned_addr learned my addr 172.21.15.150:0/2461287539 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a31728700 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a30f27700 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2461287539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 msgr2=0x5634a95d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95d4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2461287539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 msgr2=0x5634a95d9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a2bfff700 1 -- 172.21.15.150:0/2461287539 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a94ba560 con 0x5634a95d46a0 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e5b60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2a180969b0 tx=0x7f2a180cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717364ab57e098b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a177fe700 1 -- 172.21.15.150:0/2461287539 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1809ddc0 con 0x5634a95d46a0 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a177fe700 1 -- 172.21.15.150:0/2461287539 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a180a2410 con 0x5634a95d46a0 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.824+0000 7f2a177fe700 1 -- 172.21.15.150:0/2461287539 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18099480 con 0x5634a95d46a0 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 -- 172.21.15.150:0/2461287539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 msgr2=0x5634a95e5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e5b60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2a180969b0 tx=0x7f2a180cc040 comp rx=0 tx=0).stop 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 -- 172.21.15.150:0/2461287539 shutdown_connections 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95d9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/2461287539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e5b60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 -- 172.21.15.150:0/2461287539 >> 172.21.15.150:0/2461287539 conn(0x5634a95cb450 msgr2=0x5634a95cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 -- 172.21.15.150:0/2461287539 shutdown_connections 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 -- 172.21.15.150:0/2461287539 wait complete. 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.825+0000 7f2a434cfb80 1 -- start start 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 1 Processor -- start 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95e3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a95e4f00 con 0x5634a95d46a0 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a95e5070 con 0x5634a95d5570 2024-05-10T04:41:57.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a95e5230 con 0x5634a9536a60 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a434cfb80 10 client.0 _ll_register_callbacks cb 0x5634a9499b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a2bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43826/0 (socket says 172.21.15.150:43826) 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a30f27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a2bfff700 1 -- 172.21.15.150:0/3518690935 learned_addr learned my addr 172.21.15.150:0/3518690935 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.826+0000 7f2a31728700 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a30f27700 1 -- 172.21.15.150:0/3518690935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 msgr2=0x5634a95e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a30f27700 1 -- 172.21.15.150:0/3518690935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 msgr2=0x5634a95e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a30f27700 1 -- 172.21.15.150:0/3518690935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c1a0600 con 0x5634a9536a60 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a30f27700 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95e3550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a20003ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ce88128754dc092 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3518690935 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20020b00 con 0x5634a9536a60 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a20003ca0 con 0x5634a9536a60 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3518690935 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f2a20018040 con 0x5634a9536a60 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3518690935 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a20012d00 con 0x5634a9536a60 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a434cfb80 10 client.25945 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5634a9379cd0 con 0x5634a9536a60 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.827+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3518690935 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a20003ca0 con 0x5634a9536a60 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3518690935 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a200053e0 con 0x5634a9536a60 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a434cfb80 10 client.25945 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a434cfb80 10 client.25945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a434cfb80 20 client.25945 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95d5ec0 0x5634a9642e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a434cfb80 4 client.25945 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:57.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634a94ecec0 con 0x5634a95d5ec0 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95d5ec0 0x5634a9642e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a0bfff700 10 client.25945 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.828+0000 7f2a2bfff700 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95d5ec0 0x5634a9642e70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62ce25458cab042f server_cookie=3bf73cae598596c0 in_seq=0 out_seq=0 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a0bfff700 1 -- 172.21.15.150:0/3518690935 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634a94ecec0 con 0x5634a95d5ec0 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a0bfff700 10 client.25945 handle_command_reply: tid=0 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 client.25945 shutdown 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 msgr2=0x5634a95e3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95e3550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a20003ac0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 shutdown_connections 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x5634a95d5ec0 0x5634a9642e70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634a95d5570 0x5634a95e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a0bfff700 0 client.25945 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5634a9536a60 0x5634a95e3550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 --2- 172.21.15.150:0/3518690935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5634a95d46a0 0x5634a95e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 >> 172.21.15.150:0/3518690935 conn(0x5634a95cb450 msgr2=0x5634a95dc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 shutdown_connections 2024-05-10T04:41:57.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 1 -- 172.21.15.150:0/3518690935 wait complete. 2024-05-10T04:41:57.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.829+0000 7f2a434cfb80 20 client.25945 trim_cache size 0 max 16384 2024-05-10T04:41:57.837 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2a1c0a11c0 msgr2=0x7f2a1c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2a1c0a11c0 0x7f2a1c0a36d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c06a1e0 tx=0x7f2a20009260 comp rx=0 tx=0).stop 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c105040 msgr2=0x7f2a2c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c105040 0x7f2a2c198540 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800e9a0 tx=0x7f2a1800ecb0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 shutdown_connections 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f2a1c0a11c0 0x7f2a1c0a36d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a2c1079a0 0x7f2a2c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1026e0 0x7f2a2c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 --2- 172.21.15.150:0/2814403729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a2c105040 0x7f2a2c198540 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 >> 172.21.15.150:0/2814403729 conn(0x7f2a2c0fc100 msgr2=0x7f2a2c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 shutdown_connections 2024-05-10T04:41:57.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.830+0000 7f2a3318b700 1 -- 172.21.15.150:0/2814403729 wait complete. 2024-05-10T04:41:57.839 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:57 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3694186111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:41:57.844 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:57 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3694186111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:41:57.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:41:57.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:57 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3694186111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:41:57.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 -- 172.21.15.150:0/1490200797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc101c00 msgr2=0x7fd6cc102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/1490200797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc101c00 0x7fd6cc102010 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009990 tx=0x7fd6bc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.992 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 -- 172.21.15.150:0/1490200797 shutdown_connections 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/1490200797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc1025a0 0x7fd6cc108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/1490200797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6cc100ae0 0x7fd6cc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/1490200797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc101c00 0x7fd6cc102010 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 -- 172.21.15.150:0/1490200797 >> 172.21.15.150:0/1490200797 conn(0x7fd6cc0fc0f0 msgr2=0x7fd6cc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 -- 172.21.15.150:0/1490200797 shutdown_connections 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.988+0000 7fd6d3f33700 1 -- 172.21.15.150:0/1490200797 wait complete. 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 Processor -- start 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 -- start start 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc100ae0 0x7fd6cc06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc101c00 0x7fd6cc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6cc1025a0 0x7fd6cc19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc19fa10 con 0x7fd6cc100ae0 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc19fb80 con 0x7fd6cc101c00 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d3f33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc19fd40 con 0x7fd6cc1025a0 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d1ccf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc100ae0 0x7fd6cc06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d1ccf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc100ae0 0x7fd6cc06a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43850/0 (socket says 172.21.15.150:43850) 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/39514714 learned_addr learned my addr 172.21.15.150:0/39514714 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc101c00 0x7fd6cc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d24d0700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6cc1025a0 0x7fd6cc19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/39514714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6cc1025a0 msgr2=0x7fd6cc19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.989+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6cc1025a0 0x7fd6cc19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/39514714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc101c00 msgr2=0x7fd6cc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc101c00 0x7fd6cc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:57.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/39514714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0095f0 con 0x7fd6cc100ae0 2024-05-10T04:41:57.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6d24d0700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6cc1025a0 0x7fd6cc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:57.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc100ae0 0x7fd6cc06a5b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e940 tx=0x7fd6c800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6c2ffd700 1 -- 172.21.15.150:0/39514714 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800cbd0 con 0x7fd6cc100ae0 2024-05-10T04:41:57.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6c2ffd700 1 -- 172.21.15.150:0/39514714 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6c800ee90 con 0x7fd6cc100ae0 2024-05-10T04:41:57.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc19ffc0 con 0x7fd6cc100ae0 2024-05-10T04:41:57.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6c2ffd700 1 -- 172.21.15.150:0/39514714 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8003cb0 con 0x7fd6cc100ae0 2024-05-10T04:41:57.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.990+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc1a0470 con 0x7fd6cc100ae0 2024-05-10T04:41:57.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.993+0000 7fd6c2ffd700 1 -- 172.21.15.150:0/39514714 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd6c8018430 con 0x7fd6cc100ae0 2024-05-10T04:41:57.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.993+0000 7fd6c2ffd700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd6b80a11c0 0x7fd6b80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.998 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.994+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd6b80a11c0 0x7fd6b80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.994+0000 7fd6c2ffd700 1 -- 172.21.15.150:0/39514714 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fd6c8014070 con 0x7fd6cc100ae0 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.994+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd6b80a11c0 0x7fd6b80a36d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00b2e0 tx=0x7fd6bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.994+0000 7fd6e4277b80 1 Processor -- start 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6e4277b80 1 -- start start 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c76da150 con 0x5645c77b4570 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c76d7b10 con 0x5645c77b36a0 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c76d3400 con 0x5645c77b4ec0 2024-05-10T04:41:57.999 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43866/0 (socket says 172.21.15.150:43866) 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 -- 172.21.15.150:0/262264705 learned_addr learned my addr 172.21.15.150:0/262264705 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d24d0700 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 -- 172.21.15.150:0/262264705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 msgr2=0x5645c77bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 -- 172.21.15.150:0/262264705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 msgr2=0x5645c77b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d14ce700 1 -- 172.21.15.150:0/262264705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645c75543b0 con 0x5645c77b4570 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.995+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.996+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77b4980 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009990 tx=0x7fd6bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec18498cacaf004 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.996+0000 7fd6b7fff700 1 -- 172.21.15.150:0/262264705 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc02b070 con 0x5645c77b4570 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.996+0000 7fd6b7fff700 1 -- 172.21.15.150:0/262264705 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6bc01e040 con 0x5645c77b4570 2024-05-10T04:41:58.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.996+0000 7fd6b7fff700 1 -- 172.21.15.150:0/262264705 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc024830 con 0x5645c77b4570 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 -- 172.21.15.150:0/262264705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 msgr2=0x5645c77b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77b4980 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009990 tx=0x7fd6bc01f000 comp rx=0 tx=0).stop 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 -- 172.21.15.150:0/262264705 shutdown_connections 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/262264705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77b4980 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 -- 172.21.15.150:0/262264705 >> 172.21.15.150:0/262264705 conn(0x5645c7715a60 msgr2=0x5645c77adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 -- 172.21.15.150:0/262264705 shutdown_connections 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 -- 172.21.15.150:0/262264705 wait complete. 2024-05-10T04:41:58.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 -- start start 2024-05-10T04:41:58.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.997+0000 7fd6e4277b80 1 Processor -- start 2024-05-10T04:41:58.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c766f560 con 0x5645c77b4570 2024-05-10T04:41:58.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c766f200 con 0x5645c77b36a0 2024-05-10T04:41:58.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c75566a0 con 0x5645c77b4ec0 2024-05-10T04:41:58.006 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6e4277b80 10 client.0 _ll_register_callbacks cb 0x5645c7678b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77bdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43872/0 (socket says 172.21.15.150:43872) 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 -- 172.21.15.150:0/1406804472 learned_addr learned my addr 172.21.15.150:0/1406804472 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d24d0700 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 -- 172.21.15.150:0/1406804472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 msgr2=0x5645c77c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 -- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 msgr2=0x5645c77bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.998+0000 7fd6d14ce700 1 -- 172.21.15.150:0/1406804472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc1a0470 con 0x5645c77b4570 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77bdb00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc025d90 tx=0x7fd6bc025dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aee6e370b52c0a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:58.007 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6b57fa700 1 -- 172.21.15.150:0/1406804472 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc02b070 con 0x5645c77b4570 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6b57fa700 1 -- 172.21.15.150:0/1406804472 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fd6bc00f670 con 0x5645c77b4570 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645c762c270 con 0x5645c77b4570 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6b57fa700 1 -- 172.21.15.150:0/1406804472 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6bc00f830 con 0x5645c77b4570 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 10 client.16425 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd6cc1098e0 con 0x5645c77b4570 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6b57fa700 1 -- 172.21.15.150:0/1406804472 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6bc042080 con 0x5645c77b4570 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6b57fa700 1 -- 172.21.15.150:0/1406804472 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd6bc037c80 con 0x5645c77b4570 2024-05-10T04:41:58.008 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 10 client.16425 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:58.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:58.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 20 client.16425 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:58.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c77cd1a0 0x5645c77cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 4 client.16425 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:58.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:57.999+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645c7782780 con 0x5645c77cd1a0 2024-05-10T04:41:58.009 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.000+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c77cd1a0 0x5645c77cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.000+0000 7fd6b57fa700 10 client.16425 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.000+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c77cd1a0 0x5645c77cf5c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d66df2144166b0 server_cookie=472f0ab2a613beb6 in_seq=0 out_seq=0 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6b57fa700 1 -- 172.21.15.150:0/1406804472 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645c7782780 con 0x5645c77cd1a0 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6b57fa700 10 client.16425 handle_command_reply: tid=0 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 client.16425 shutdown 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 msgr2=0x5645c77bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77bdb00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc025d90 tx=0x7fd6bc025dc0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 shutdown_connections 2024-05-10T04:41:58.010 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c77cd1a0 0x5645c77cf5c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1406804472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4570 0x5645c77bdb00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 >> 172.21.15.150:0/1406804472 conn(0x5645c7715a60 msgr2=0x5645c77adfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6b57fa700 0 client.16425 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 shutdown_connections 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1406804472 wait complete. 2024-05-10T04:41:58.011 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.001+0000 7fd6e4277b80 20 client.16425 trim_cache size 0 max 16384 2024-05-10T04:41:58.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.014+0000 7fd6e4277b80 1 Processor -- start 2024-05-10T04:41:58.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.014+0000 7fd6e4277b80 1 -- start start 2024-05-10T04:41:58.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4570 0x5645c77c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.017 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4ec0 0x5645c77bd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c77be660 con 0x5645c77b4ec0 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c77c4df0 con 0x5645c77b36a0 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c77bb580 con 0x5645c77b4570 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d24d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4ec0 0x5645c77bd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:54394/0 (socket says 172.21.15.150:54394) 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d24d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4ec0 0x5645c77bd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43878/0 (socket says 172.21.15.150:43878) 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/1615979503 learned_addr learned my addr 172.21.15.150:0/1615979503 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4570 0x5645c77c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/1615979503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4570 msgr2=0x5645c77c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.018 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4570 0x5645c77c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/1615979503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4ec0 msgr2=0x5645c77bd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4ec0 0x5645c77bd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/1615979503 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645c7500b60 con 0x5645c77b36a0 2024-05-10T04:41:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.015+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bcb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8096a20 tx=0x7fd6c80ca530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f616c77cce171a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.016+0000 7fd6c0ff9700 1 -- 172.21.15.150:0/1615979503 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c809dde0 con 0x5645c77b36a0 2024-05-10T04:41:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.016+0000 7fd6c0ff9700 1 -- 172.21.15.150:0/1615979503 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6c80a2430 con 0x5645c77b36a0 2024-05-10T04:41:58.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.016+0000 7fd6c0ff9700 1 -- 172.21.15.150:0/1615979503 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c801f340 con 0x5645c77b36a0 2024-05-10T04:41:58.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1615979503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 msgr2=0x5645c77bcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bcb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8096a20 tx=0x7fd6c80ca530 comp rx=0 tx=0).stop 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1615979503 shutdown_connections 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b4ec0 0x5645c77bd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b36a0 0x5645c77bcb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/1615979503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4570 0x5645c77c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1615979503 >> 172.21.15.150:0/1615979503 conn(0x5645c7715a60 msgr2=0x5645c77b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1615979503 shutdown_connections 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 -- 172.21.15.150:0/1615979503 wait complete. 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 -- start start 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.017+0000 7fd6e4277b80 1 Processor -- start 2024-05-10T04:41:58.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b36a0 0x5645c77c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b4570 0x5645c77c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6e4277b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c77c3570 con 0x5645c77b36a0 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c77c36e0 con 0x5645c77b4570 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6e4277b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645c77c38a0 con 0x5645c77b4ec0 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6e4277b80 10 client.0 _ll_register_callbacks cb 0x5645c7678b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b36a0 0x5645c77c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b36a0 0x5645c77c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43882/0 (socket says 172.21.15.150:43882) 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/3106142157 learned_addr learned my addr 172.21.15.150:0/3106142157 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.026 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b4570 0x5645c77c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d24d0700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/3106142157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 msgr2=0x5645c77c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b4570 msgr2=0x5645c77c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b4570 0x5645c77c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 -- 172.21.15.150:0/3106142157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc19ffc0 con 0x5645c77b36a0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b4570 0x5645c77c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6d1ccf700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b36a0 0x5645c77c1bc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c5b0 tx=0x7fd6c8098350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5183143da3722ab server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.018+0000 7fd6b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6b57fa700 1 -- 172.21.15.150:0/3106142157 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c809dde0 con 0x5645c77b36a0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6b57fa700 1 -- 172.21.15.150:0/3106142157 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fd6c801f340 con 0x5645c77b36a0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6bc01e040 con 0x5645c77b36a0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6b57fa700 1 -- 172.21.15.150:0/3106142157 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6c801f500 con 0x5645c77b36a0 2024-05-10T04:41:58.027 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 10 client.16431 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:58.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5645c7699560 con 0x5645c77b36a0 2024-05-10T04:41:58.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6b57fa700 1 -- 172.21.15.150:0/3106142157 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6c80a9080 con 0x5645c77b36a0 2024-05-10T04:41:58.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6b57fa700 1 -- 172.21.15.150:0/3106142157 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd6c80994a0 con 0x5645c77b36a0 2024-05-10T04:41:58.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:58.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:58.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 20 client.16431 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:58.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c78236b0 0x5645c7825ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 4 client.16431 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:41:58.029 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5645c76cbec0 con 0x5645c78236b0 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.019+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c78236b0 0x5645c7825ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6b57fa700 10 client.16431 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6d14ce700 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c78236b0 0x5645c7825ad0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97483a79b0f7fed server_cookie=fee24661f77055e9 in_seq=0 out_seq=0 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6b57fa700 1 -- 172.21.15.150:0/3106142157 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5645c76cbec0 con 0x5645c78236b0 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6b57fa700 10 client.16431 handle_command_reply: tid=0 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 client.16431 shutdown 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b36a0 msgr2=0x5645c77c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b36a0 0x5645c77c1bc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c5b0 tx=0x7fd6c8098350 comp rx=0 tx=0).stop 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 shutdown_connections 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5645c78236b0 0x5645c7825ad0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5645c77b4ec0 0x5645c77c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5645c77b36a0 0x5645c77c1bc0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.030 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6b57fa700 0 client.16431 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 --2- 172.21.15.150:0/3106142157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645c77b4570 0x5645c77c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 >> 172.21.15.150:0/3106142157 conn(0x5645c7715a60 msgr2=0x5645c77b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.020+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 shutdown_connections 2024-05-10T04:41:58.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6e4277b80 1 -- 172.21.15.150:0/3106142157 wait complete. 2024-05-10T04:41:58.031 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6e4277b80 20 client.16431 trim_cache size 0 max 16384 2024-05-10T04:41:58.031 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:41:58.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd6b80a11c0 msgr2=0x7fd6b80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd6b80a11c0 0x7fd6b80a36d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00b2e0 tx=0x7fd6bc01a040 comp rx=0 tx=0).stop 2024-05-10T04:41:58.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc100ae0 msgr2=0x7fd6cc06a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc100ae0 0x7fd6cc06a5b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e940 tx=0x7fd6c800ec50 comp rx=0 tx=0).stop 2024-05-10T04:41:58.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 shutdown_connections 2024-05-10T04:41:58.032 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd6b80a11c0 0x7fd6b80a36d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6cc1025a0 0x7fd6cc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc100ae0 0x7fd6cc06a5b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 --2- 172.21.15.150:0/39514714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc101c00 0x7fd6cc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 >> 172.21.15.150:0/39514714 conn(0x7fd6cc0fc0f0 msgr2=0x7fd6cc104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 shutdown_connections 2024-05-10T04:41:58.033 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:58.021+0000 7fd6d3f33700 1 -- 172.21.15.150:0/39514714 wait complete. 2024-05-10T04:41:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 -- 172.21.15.150:0/2713724761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc101bf0 msgr2=0x7fb6bc102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 --2- 172.21.15.150:0/2713724761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc101bf0 0x7fb6bc102000 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009990 tx=0x7fb6a8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 -- 172.21.15.150:0/2713724761 shutdown_connections 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 --2- 172.21.15.150:0/2713724761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc102590 0x7fb6bc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 --2- 172.21.15.150:0/2713724761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6bc100ad0 0x7fb6bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 --2- 172.21.15.150:0/2713724761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc101bf0 0x7fb6bc102000 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 -- 172.21.15.150:0/2713724761 >> 172.21.15.150:0/2713724761 conn(0x7fb6bc0fc0c0 msgr2=0x7fb6bc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 -- 172.21.15.150:0/2713724761 shutdown_connections 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 -- 172.21.15.150:0/2713724761 wait complete. 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.584+0000 7fb6c30af700 1 Processor -- start 2024-05-10T04:41:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c30af700 1 -- start start 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c30af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6bc100ad0 0x7fb6bc104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c30af700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc101bf0 0x7fb6bc102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c30af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc102590 0x7fb6bc103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c30af700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc103ad0 con 0x7fb6bc102590 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c30af700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc103c40 con 0x7fb6bc101bf0 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c30af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc103e00 con 0x7fb6bc100ad0 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc102590 0x7fb6bc103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc102590 0x7fb6bc103470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43888/0 (socket says 172.21.15.150:43888) 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 -- 172.21.15.150:0/1765327219 learned_addr learned my addr 172.21.15.150:0/1765327219 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc101bf0 0x7fb6bc102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6bc100ad0 0x7fb6bc104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 -- 172.21.15.150:0/1765327219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6bc100ad0 msgr2=0x7fb6bc104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6bc100ad0 0x7fb6bc104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 -- 172.21.15.150:0/1765327219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc101bf0 msgr2=0x7fb6bc102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc101bf0 0x7fb6bc102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 -- 172.21.15.150:0/1765327219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80095f0 con 0x7fb6bc102590 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc101bf0 0x7fb6bc102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.585+0000 7fb6c164c700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc102590 0x7fb6bc103470 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400ef30 tx=0x7fb6b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.586+0000 7fb6b9ffb700 1 -- 172.21.15.150:0/1765327219 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4018070 con 0x7fb6bc102590 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.586+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc1a21d0 con 0x7fb6bc102590 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.586+0000 7fb6b9ffb700 1 -- 172.21.15.150:0/1765327219 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b400f040 con 0x7fb6bc102590 2024-05-10T04:41:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.586+0000 7fb6b9ffb700 1 -- 172.21.15.150:0/1765327219 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400cd00 con 0x7fb6bc102590 2024-05-10T04:41:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.586+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc1a2690 con 0x7fb6bc102590 2024-05-10T04:41:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.588+0000 7fb6b9ffb700 1 -- 172.21.15.150:0/1765327219 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb6b4004000 con 0x7fb6bc102590 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.589+0000 7fb6b9ffb700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6ac0a1120 0x7fb6ac0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.589+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6ac0a1120 0x7fb6ac0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.589+0000 7fb6b9ffb700 1 -- 172.21.15.150:0/1765327219 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb6b40cc7c0 con 0x7fb6bc102590 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 Processor -- start 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 -- start start 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6ac0a1120 0x7fb6ac0a3630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc101af0 tx=0x7fb6b0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3652c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3652d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a365342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a364530f0 con 0x556a3652c180 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a36450ab0 con 0x556a3652d9a0 2024-05-10T04:41:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3644c3a0 con 0x556a3652d050 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3652c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3652c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43902/0 (socket says 172.21.15.150:43902) 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.590+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2440689104 learned_addr learned my addr 172.21.15.150:0/2440689104 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3652d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c164c700 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a365342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2440689104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 msgr2=0x556a3652d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3652d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2440689104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 msgr2=0x556a365342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a365342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2440689104 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a362cd350 con 0x556a3652c180 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c164c700 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a365342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3652c5b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0010e20 tx=0x7fb6b0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671c6202545d0c96 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6a6ffd700 1 -- 172.21.15.150:0/2440689104 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0004220 con 0x556a3652c180 2024-05-10T04:41:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6a6ffd700 1 -- 172.21.15.150:0/2440689104 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b0015070 con 0x556a3652c180 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.591+0000 7fb6a6ffd700 1 -- 172.21.15.150:0/2440689104 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b001e680 con 0x556a3652c180 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2440689104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 msgr2=0x556a3652c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3652c5b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0010e20 tx=0x7fb6b0013000 comp rx=0 tx=0).stop 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2440689104 shutdown_connections 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a365342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3652c5b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2440689104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3652d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2440689104 >> 172.21.15.150:0/2440689104 conn(0x556a3648ea00 msgr2=0x556a36526a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2440689104 shutdown_connections 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2440689104 wait complete. 2024-05-10T04:41:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.592+0000 7fb6d33f3b80 1 -- start start 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 1 Processor -- start 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a365361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a365366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a363e8500 con 0x556a3652c180 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a363e81a0 con 0x556a3652d9a0 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a362cf640 con 0x556a3652d050 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6d33f3b80 10 client.0 _ll_register_callbacks cb 0x556a363f1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a365361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a365361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43918/0 (socket says 172.21.15.150:43918) 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/536385020 learned_addr learned my addr 172.21.15.150:0/536385020 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a365366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.593+0000 7fb6c164c700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/536385020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 msgr2=0x556a365366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a365366e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 msgr2=0x556a3653d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/536385020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b0010b00 con 0x556a3652c180 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a365366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a365361a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0003920 tx=0x7fb6b0003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5323842736750d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb69bfff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb69bfff700 1 -- 172.21.15.150:0/536385020 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0005520 con 0x556a3652c180 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a363a5210 con 0x556a3652c180 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb69bfff700 1 -- 172.21.15.150:0/536385020 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb6b0017070 con 0x556a3652c180 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb69bfff700 1 -- 172.21.15.150:0/536385020 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b0005680 con 0x556a3652c180 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6d33f3b80 10 client.16443 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556a36279b00 con 0x556a3652c180 2024-05-10T04:41:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.594+0000 7fb69bfff700 1 -- 172.21.15.150:0/536385020 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a363a5210 con 0x556a3652c180 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb69bfff700 1 -- 172.21.15.150:0/536385020 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6b0024070 con 0x556a3652c180 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6d33f3b80 10 client.16443 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6d33f3b80 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6d33f3b80 20 client.16443 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a36541d20 0x556a36544170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6d33f3b80 4 client.16443 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a364fb720 con 0x556a36541d20 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a36541d20 0x556a36544170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a36541d20 0x556a36544170 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f49c8352cd3e07 server_cookie=f9af2c33d0453548 in_seq=0 out_seq=0 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.595+0000 7fb69bfff700 10 client.16443 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb69bfff700 1 -- 172.21.15.150:0/536385020 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a364fb720 con 0x556a36541d20 2024-05-10T04:41:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb69bfff700 10 client.16443 handle_command_reply: tid=0 2024-05-10T04:41:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 client.16443 shutdown 2024-05-10T04:41:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 msgr2=0x556a365361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a365361a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0003920 tx=0x7fb6b0003950 comp rx=0 tx=0).stop 2024-05-10T04:41:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 shutdown_connections 2024-05-10T04:41:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a36541d20 0x556a36544170 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a365361a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb69bfff700 0 client.16443 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/536385020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a365366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.596+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 >> 172.21.15.150:0/536385020 conn(0x556a3648ea00 msgr2=0x556a36526a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.597+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 shutdown_connections 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.597+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/536385020 wait complete. 2024-05-10T04:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.597+0000 7fb6d33f3b80 20 client.16443 trim_cache size 0 max 16384 2024-05-10T04:41:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 Processor -- start 2024-05-10T04:41:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 -- start start 2024-05-10T04:41:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d050 0x556a3653d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d9a0 0x556a365344f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3652ddb0 con 0x556a3652c180 2024-05-10T04:41:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a36521360 con 0x556a3652d050 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a36534170 con 0x556a3652d9a0 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.612+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43924/0 (socket says 172.21.15.150:43924) 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2366707411 learned_addr learned my addr 172.21.15.150:0/2366707411 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d050 0x556a3653d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c164c700 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d9a0 0x556a365344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2366707411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d9a0 msgr2=0x556a365344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d9a0 0x556a365344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2366707411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d050 msgr2=0x556a3653d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d050 0x556a3653d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/2366707411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a36412500 con 0x556a3652c180 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c164c700 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d9a0 0x556a365344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653dc20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0010ad0 tx=0x7fb6b0010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c22502070ec8be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6a77fe700 1 -- 172.21.15.150:0/2366707411 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0026440 con 0x556a3652c180 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6a77fe700 1 -- 172.21.15.150:0/2366707411 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b0003c90 con 0x556a3652c180 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.613+0000 7fb6a77fe700 1 -- 172.21.15.150:0/2366707411 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0033680 con 0x556a3652c180 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2366707411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 msgr2=0x556a3653dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653dc20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0010ad0 tx=0x7fb6b0010a60 comp rx=0 tx=0).stop 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2366707411 shutdown_connections 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d9a0 0x556a365344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653dc20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/2366707411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d050 0x556a3653d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2366707411 >> 172.21.15.150:0/2366707411 conn(0x556a3648ea00 msgr2=0x556a36530e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2366707411 shutdown_connections 2024-05-10T04:41:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.614+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/2366707411 wait complete. 2024-05-10T04:41:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 -- start start 2024-05-10T04:41:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 Processor -- start 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3653a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a36596100 con 0x556a3652c180 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a36596240 con 0x556a3652d9a0 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a36596380 con 0x556a3652d050 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6d33f3b80 10 client.0 _ll_register_callbacks cb 0x556a363f1ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.615+0000 7fb6c0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43936/0 (socket says 172.21.15.150:43936) 2024-05-10T04:41:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/684654910 learned_addr learned my addr 172.21.15.150:0/684654910 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3653a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c164c700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/684654910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 msgr2=0x556a3653a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3653a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 msgr2=0x556a3653a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c0e4b700 1 -- 172.21.15.150:0/684654910 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a36279b00 con 0x556a3652c180 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c164c700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3653a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6c0e4b700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653d5c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00036c0 tx=0x7fb6b0026c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a6d6a32c127e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb69bfff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb69bfff700 1 -- 172.21.15.150:0/684654910 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0026860 con 0x556a3652c180 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb69bfff700 1 -- 172.21.15.150:0/684654910 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb6b0046070 con 0x556a3652c180 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6b0015070 con 0x556a3652c180 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb69bfff700 1 -- 172.21.15.150:0/684654910 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b004b480 con 0x556a3652c180 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6d33f3b80 10 client.16449 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.616+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556a362d1c70 con 0x556a3652c180 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb69bfff700 1 -- 172.21.15.150:0/684654910 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b0015070 con 0x556a3652c180 2024-05-10T04:41:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb69bfff700 1 -- 172.21.15.150:0/684654910 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6b00117d0 con 0x556a3652c180 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb6d33f3b80 10 client.16449 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb6d33f3b80 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb6d33f3b80 20 client.16449 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a365977e0 0x556a36599c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb6d33f3b80 4 client.16449 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a36444e60 con 0x556a365977e0 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.617+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a365977e0 0x556a36599c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.618+0000 7fb69bfff700 10 client.16449 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.618+0000 7fb6bbfff700 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a365977e0 0x556a36599c00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f6668c11b6b15a server_cookie=c0f7497fd21ad3c7 in_seq=0 out_seq=0 2024-05-10T04:41:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb69bfff700 1 -- 172.21.15.150:0/684654910 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a36444e60 con 0x556a365977e0 2024-05-10T04:41:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb69bfff700 10 client.16449 handle_command_reply: tid=0 2024-05-10T04:41:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 client.16449 shutdown 2024-05-10T04:41:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 msgr2=0x556a3653d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653d5c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00036c0 tx=0x7fb6b0026c20 comp rx=0 tx=0).stop 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 shutdown_connections 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x556a365977e0 0x556a36599c00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a3652d9a0 0x556a3653a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a3652c180 0x556a3653d5c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 --2- 172.21.15.150:0/684654910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556a3652d050 0x556a3653a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb69bfff700 0 client.16449 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 >> 172.21.15.150:0/684654910 conn(0x556a3648ea00 msgr2=0x556a365344b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 shutdown_connections 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.628+0000 7fb6d33f3b80 1 -- 172.21.15.150:0/684654910 wait complete. 2024-05-10T04:41:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6d33f3b80 20 client.16449 trim_cache size 0 max 16384 2024-05-10T04:41:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:41:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:41:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:41:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6ac0a1120 msgr2=0x7fb6ac0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6ac0a1120 0x7fb6ac0a3630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc101af0 tx=0x7fb6b0007400 comp rx=0 tx=0).stop 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc102590 msgr2=0x7fb6bc103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc102590 0x7fb6bc103470 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400ef30 tx=0x7fb6b400c3a0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 shutdown_connections 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6bc102590 0x7fb6bc103470 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6ac0a1120 0x7fb6ac0a3630 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6bc100ad0 0x7fb6bc104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 --2- 172.21.15.150:0/1765327219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6bc101bf0 0x7fb6bc102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 >> 172.21.15.150:0/1765327219 conn(0x7fb6bc0fc0c0 msgr2=0x7fb6bc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 shutdown_connections 2024-05-10T04:41:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:41:58.629+0000 7fb6c30af700 1 -- 172.21.15.150:0/1765327219 wait complete. 2024-05-10T04:41:58.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:58 smithi161 ceph-mon[46912]: pgmap v343: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 779 KiB/s wr, 16 op/s 2024-05-10T04:41:58.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:58 smithi181 ceph-mon[47314]: pgmap v343: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 779 KiB/s wr, 16 op/s 2024-05-10T04:41:58.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:58 smithi150 ceph-mon[42404]: pgmap v343: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 779 KiB/s wr, 16 op/s 2024-05-10T04:41:59.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:41:59.038 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 -- 172.21.15.150:0/2908877064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 msgr2=0x7f6170109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 --2- 172.21.15.150:0/2908877064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170109db0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f616800b0a0 tx=0x7f616800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 -- 172.21.15.150:0/2908877064 shutdown_connections 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 --2- 172.21.15.150:0/2908877064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170109db0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 --2- 172.21.15.150:0/2908877064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61701026d0 0x7f6170104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 --2- 172.21.15.150:0/2908877064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6170105030 0x7f6170107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 -- 172.21.15.150:0/2908877064 >> 172.21.15.150:0/2908877064 conn(0x7f61700fc0f0 msgr2=0x7f61700fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 -- 172.21.15.150:0/2908877064 shutdown_connections 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.184+0000 7f61765a5700 1 -- 172.21.15.150:0/2908877064 wait complete. 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 Processor -- start 2024-05-10T04:41:59.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 -- start start 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61701026d0 0x7f6170101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6170105030 0x7f61701005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6170101200 con 0x7f6170107990 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6170101370 con 0x7f61701026d0 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f61765a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61701a2120 con 0x7f6170105030 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f6174b42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f6174b42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43954/0 (socket says 172.21.15.150:43954) 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f6174b42700 1 -- 172.21.15.150:0/3989603173 learned_addr learned my addr 172.21.15.150:0/3989603173 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.185+0000 7f616ffff700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61701026d0 0x7f6170101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f616f7fe700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6170105030 0x7f61701005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.189 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f6174b42700 1 -- 172.21.15.150:0/3989603173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6170105030 msgr2=0x7f61701005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f6174b42700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6170105030 0x7f61701005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f6174b42700 1 -- 172.21.15.150:0/3989603173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61701026d0 msgr2=0x7f6170101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f6174b42700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61701026d0 0x7f6170101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f6174b42700 1 -- 172.21.15.150:0/3989603173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164009520 con 0x7f6170107990 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f616ffff700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61701026d0 0x7f6170101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f6174b42700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170100b20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f616800b070 tx=0x7f61680078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f616d7fa700 1 -- 172.21.15.150:0/3989603173 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e040 con 0x7f6170107990 2024-05-10T04:41:59.190 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6168009d00 con 0x7f6170107990 2024-05-10T04:41:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f616d7fa700 1 -- 172.21.15.150:0/3989603173 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6168004c50 con 0x7f6170107990 2024-05-10T04:41:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f616d7fa700 1 -- 172.21.15.150:0/3989603173 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680052a0 con 0x7f6170107990 2024-05-10T04:41:59.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.186+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701a2660 con 0x7f6170107990 2024-05-10T04:41:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.188+0000 7f616d7fa700 1 -- 172.21.15.150:0/3989603173 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f6168005400 con 0x7f6170107990 2024-05-10T04:41:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.189+0000 7f616d7fa700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f61580a11c0 0x7f61580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.189+0000 7f616d7fa700 1 -- 172.21.15.150:0/3989603173 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f61680ce030 con 0x7f6170107990 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.189+0000 7f616ffff700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f61580a11c0 0x7f61580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f616ffff700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f61580a11c0 0x7f61580a36d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f617006a200 tx=0x7f6160009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 Processor -- start 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 -- start start 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e6720 0x55793e1e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e75f0 0x55793e1e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1ee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e10d1d0 con 0x55793e1e6720 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e10ab90 con 0x55793e1e7f40 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.190+0000 7f61868e9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e106480 con 0x55793e1e75f0 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e6720 0x55793e1e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e6720 0x55793e1e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43966/0 (socket says 172.21.15.150:43966) 2024-05-10T04:41:59.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 -- 172.21.15.150:0/1565826402 learned_addr learned my addr 172.21.15.150:0/1565826402 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f6174b42700 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616f7fe700 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e75f0 0x55793e1e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 -- 172.21.15.150:0/1565826402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e75f0 msgr2=0x55793e1e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e75f0 0x55793e1e7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 -- 172.21.15.150:0/1565826402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 msgr2=0x55793e1ee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1ee840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 -- 172.21.15.150:0/1565826402 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793df87430 con 0x55793e1e6720 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f616ffff700 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e6720 0x55793e1e6b70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f61600043b0 tx=0x7f6160014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66758813c2cf44a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f61567fc700 1 -- 172.21.15.150:0/1565826402 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160003d10 con 0x55793e1e6720 2024-05-10T04:41:59.196 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f61567fc700 1 -- 172.21.15.150:0/1565826402 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6160003e70 con 0x55793e1e6720 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.191+0000 7f61567fc700 1 -- 172.21.15.150:0/1565826402 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000e6f0 con 0x55793e1e6720 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 -- 172.21.15.150:0/1565826402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e6720 msgr2=0x55793e1e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e6720 0x55793e1e6b70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f61600043b0 tx=0x7f6160014000 comp rx=0 tx=0).stop 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 -- 172.21.15.150:0/1565826402 shutdown_connections 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1ee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e6720 0x55793e1e6b70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1565826402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e75f0 0x55793e1e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 -- 172.21.15.150:0/1565826402 >> 172.21.15.150:0/1565826402 conn(0x55793e148ae0 msgr2=0x55793e1e1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 -- 172.21.15.150:0/1565826402 shutdown_connections 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.192+0000 7f61868e9b80 1 -- 172.21.15.150:0/1565826402 wait complete. 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 -- start start 2024-05-10T04:41:59.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 Processor -- start 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e0a25e0 con 0x55793e1e7f40 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e0a2280 con 0x55793e1e75f0 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.193+0000 7f61868e9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793df89720 con 0x55793e1e6720 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f61868e9b80 10 client.0 _ll_register_callbacks cb 0x55793e0abbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:36316/0 (socket says 172.21.15.150:36316) 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 -- 172.21.15.150:0/3116144136 learned_addr learned my addr 172.21.15.150:0/3116144136 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f6174b42700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.203 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616f7fe700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 -- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 msgr2=0x55793e1f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 -- 172.21.15.150:0/3116144136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 msgr2=0x55793e1f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 -- 172.21.15.150:0/3116144136 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6160017070 con 0x55793e1e6720 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f6174b42700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f616ffff700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f0790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6160003e40 tx=0x7f6160005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68b48000a1aef54a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f614f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.194+0000 7f614f7fe700 1 -- 172.21.15.150:0/3116144136 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616002c070 con 0x55793e1e6720 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55793e05f2f0 con 0x55793e1e6720 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f614f7fe700 1 -- 172.21.15.150:0/3116144136 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f616001e3f0 con 0x55793e1e6720 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f614f7fe700 1 -- 172.21.15.150:0/3116144136 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f616001e5b0 con 0x55793e1e6720 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 10 client.25993 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:59.204 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55793df33be0 con 0x55793e1e6720 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f614f7fe700 1 -- 172.21.15.150:0/3116144136 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55793e05f2f0 con 0x55793e1e6720 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f614f7fe700 1 -- 172.21.15.150:0/3116144136 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6160024020 con 0x55793e1e6720 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 10 client.25993 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 20 client.25993 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e1fc2b0 0x55793e1fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 4 client.25993 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.195+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55793e1b5800 con 0x55793e1fc2b0 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.196+0000 7f616f7fe700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e1fc2b0 0x55793e1fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.196+0000 7f614f7fe700 10 client.25993 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.196+0000 7f616f7fe700 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e1fc2b0 0x55793e1fe700 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59757bbb7a48190f server_cookie=68834931646b3d8 in_seq=0 out_seq=0 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.196+0000 7f614f7fe700 1 -- 172.21.15.150:0/3116144136 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55793e1b5800 con 0x55793e1fc2b0 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f614f7fe700 10 client.25993 handle_command_reply: tid=0 2024-05-10T04:41:59.205 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 client.25993 shutdown 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 msgr2=0x55793e1f0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f0790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6160003e40 tx=0x7f6160005400 comp rx=0 tx=0).stop 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 shutdown_connections 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e1fc2b0 0x55793e1fe700 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f0790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 --2- 172.21.15.150:0/3116144136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f614f7fe700 0 client.25993 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 >> 172.21.15.150:0/3116144136 conn(0x55793e148ae0 msgr2=0x55793e1e1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 shutdown_connections 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 1 -- 172.21.15.150:0/3116144136 wait complete. 2024-05-10T04:41:59.206 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.197+0000 7f61868e9b80 20 client.25993 trim_cache size 0 max 16384 2024-05-10T04:41:59.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.214+0000 7f61868e9b80 1 Processor -- start 2024-05-10T04:41:59.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.214+0000 7f61868e9b80 1 -- start start 2024-05-10T04:41:59.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1ef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f61868e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e1fbe10 con 0x55793e1e7f40 2024-05-10T04:41:59.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f61868e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e1e7440 con 0x55793e1e75f0 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f61868e9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e1fbfd0 con 0x55793e1e6720 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1efd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:43994/0 (socket says 172.21.15.150:43994) 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 -- 172.21.15.150:0/4263786426 learned_addr learned my addr 172.21.15.150:0/4263786426 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f616ffff700 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f616f7fe700 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 -- 172.21.15.150:0/4263786426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 msgr2=0x55793e1ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1ef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 -- 172.21.15.150:0/4263786426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 msgr2=0x55793e1f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 -- 172.21.15.150:0/4263786426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793e1dd4d0 con 0x55793e1e7f40 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.215+0000 7f6174b42700 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1efd00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6168009cd0 tx=0x7f616801dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f748b8ddb1a2d9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.216+0000 7f6156ffd700 1 -- 172.21.15.150:0/4263786426 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616809c0e0 con 0x55793e1e7f40 2024-05-10T04:41:59.223 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.216+0000 7f6156ffd700 1 -- 172.21.15.150:0/4263786426 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f616809ff10 con 0x55793e1e7f40 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.216+0000 7f6156ffd700 1 -- 172.21.15.150:0/4263786426 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168011ea0 con 0x55793e1e7f40 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.216+0000 7f61868e9b80 1 -- 172.21.15.150:0/4263786426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 msgr2=0x55793e1efd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.216+0000 7f61868e9b80 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1efd00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6168009cd0 tx=0x7f616801dc60 comp rx=0 tx=0).stop 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 -- 172.21.15.150:0/4263786426 shutdown_connections 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e7f40 0x55793e1efd00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 --2- 172.21.15.150:0/4263786426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e75f0 0x55793e1f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 -- 172.21.15.150:0/4263786426 >> 172.21.15.150:0/4263786426 conn(0x55793e148ae0 msgr2=0x55793e1eeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 -- 172.21.15.150:0/4263786426 shutdown_connections 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 -- 172.21.15.150:0/4263786426 wait complete. 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 -- start start 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 Processor -- start 2024-05-10T04:41:59.224 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e75f0 0x55793e1f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.217+0000 7f61868e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f61868e9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e1f63e0 con 0x55793e1e75f0 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f61868e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e1f6550 con 0x55793e1e7f40 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f61868e9b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e1f6710 con 0x55793e1e6720 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f61868e9b80 10 client.0 _ll_register_callbacks cb 0x55793e0abbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e75f0 0x55793e1f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e75f0 0x55793e1f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44000/0 (socket says 172.21.15.150:44000) 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 -- 172.21.15.150:0/1834937530 learned_addr learned my addr 172.21.15.150:0/1834937530 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616ffff700 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f6174b42700 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 -- 172.21.15.150:0/1834937530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 msgr2=0x55793e1f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 -- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 msgr2=0x55793e1f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 -- 172.21.15.150:0/1834937530 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701a2660 con 0x55793e1e75f0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f616f7fe700 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e75f0 0x55793e1f7b70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6164000c00 tx=0x7f6164012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32f8881fe9ab028 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f614f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f614f7fe700 1 -- 172.21.15.150:0/1834937530 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164017050 con 0x55793e1e75f0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f614f7fe700 1 -- 172.21.15.150:0/1834937530 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f6164012e20 con 0x55793e1e75f0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.218+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6160003e70 con 0x55793e1e75f0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f614f7fe700 1 -- 172.21.15.150:0/1834937530 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61640204c0 con 0x55793e1e75f0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 10 client.16467 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55793e0cc5e0 con 0x55793e1e75f0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f614f7fe700 1 -- 172.21.15.150:0/1834937530 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6160003e70 con 0x55793e1e75f0 2024-05-10T04:41:59.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f614f7fe700 1 -- 172.21.15.150:0/1834937530 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6164029420 con 0x55793e1e75f0 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 10 client.16467 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 20 client.16467 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e253c20 0x55793e256040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 4 client.16467 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55793e0fef40 con 0x55793e253c20 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.219+0000 7f616ffff700 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e253c20 0x55793e256040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.220+0000 7f614f7fe700 10 client.16467 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.220+0000 7f616ffff700 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e253c20 0x55793e256040 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52975e9f552e53ec server_cookie=f727b1fbc1913680 in_seq=0 out_seq=0 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f614f7fe700 1 -- 172.21.15.150:0/1834937530 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55793e0fef40 con 0x55793e253c20 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f614f7fe700 10 client.16467 handle_command_reply: tid=0 2024-05-10T04:41:59.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 client.16467 shutdown 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e75f0 msgr2=0x55793e1f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e75f0 0x55793e1f7b70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6164000c00 tx=0x7f6164012870 comp rx=0 tx=0).stop 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 shutdown_connections 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55793e253c20 0x55793e256040 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55793e1e7f40 0x55793e1f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55793e1e6720 0x55793e1f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f614f7fe700 0 client.16467 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 --2- 172.21.15.150:0/1834937530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55793e1e75f0 0x55793e1f7b70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 >> 172.21.15.150:0/1834937530 conn(0x55793e148ae0 msgr2=0x55793e1e5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 shutdown_connections 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.221+0000 7f61868e9b80 1 -- 172.21.15.150:0/1834937530 wait complete. 2024-05-10T04:41:59.229 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61868e9b80 20 client.16467 trim_cache size 0 max 16384 2024-05-10T04:41:59.230 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:41:59.230 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:41:59.230 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "f09282df-00fa-4769-ac5b-e40368694587", 2024-05-10T04:41:59.230 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:41:59.231 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:41:59.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f61580a11c0 msgr2=0x7f61580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f61580a11c0 0x7f61580a36d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f617006a200 tx=0x7f6160009260 comp rx=0 tx=0).stop 2024-05-10T04:41:59.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 msgr2=0x7f6170100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170100b20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f616800b070 tx=0x7f61680078e0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 shutdown_connections 2024-05-10T04:41:59.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6170107990 0x7f6170100b20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61701026d0 0x7f6170101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f61580a11c0 0x7f61580a36d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 --2- 172.21.15.150:0/3989603173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6170105030 0x7f61701005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 >> 172.21.15.150:0/3989603173 conn(0x7f61700fc0f0 msgr2=0x7f6170103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 shutdown_connections 2024-05-10T04:41:59.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.222+0000 7f61765a5700 1 -- 172.21.15.150:0/3989603173 wait complete. 2024-05-10T04:41:59.238 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:41:59.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 -- 172.21.15.150:0/2225394775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514102580 msgr2=0x7f4514108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2225394775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514102580 0x7f4514108e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f450c00b0a0 tx=0x7f450c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 -- 172.21.15.150:0/2225394775 shutdown_connections 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2225394775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514102580 0x7f4514108e30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2225394775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4514100ac0 0x7f4514100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2225394775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4514101be0 0x7f4514101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 -- 172.21.15.150:0/2225394775 >> 172.21.15.150:0/2225394775 conn(0x7f45140fc0d0 msgr2=0x7f45140fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 -- 172.21.15.150:0/2225394775 shutdown_connections 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 -- 172.21.15.150:0/2225394775 wait complete. 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.383+0000 7f4519cb8700 1 Processor -- start 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4519cb8700 1 -- start start 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4519cb8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514100ac0 0x7f4514068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4519cb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4514101be0 0x7f451419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4519cb8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4514102580 0x7f451419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4519cb8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451419fa10 con 0x7f4514100ac0 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4519cb8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451419fb80 con 0x7f4514102580 2024-05-10T04:41:59.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4519cb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451419fd40 con 0x7f4514101be0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514100ac0 0x7f4514068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514100ac0 0x7f4514068e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44030/0 (socket says 172.21.15.150:44030) 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 -- 172.21.15.150:0/2999055328 learned_addr learned my addr 172.21.15.150:0/2999055328 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4512ffd700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4514101be0 0x7f451419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f4513fff700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4514102580 0x7f451419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 -- 172.21.15.150:0/2999055328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4514101be0 msgr2=0x7f451419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4514101be0 0x7f451419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 -- 172.21.15.150:0/2999055328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4514102580 msgr2=0x7f451419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4514102580 0x7f451419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 -- 172.21.15.150:0/2999055328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c009d00 con 0x7f4514100ac0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.384+0000 7f45137fe700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514100ac0 0x7f4514068e30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f450400d950 tx=0x7f450400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.385+0000 7f4510ff9700 1 -- 172.21.15.150:0/2999055328 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504004220 con 0x7f4514100ac0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.385+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451419ffc0 con 0x7f4514100ac0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.385+0000 7f4510ff9700 1 -- 172.21.15.150:0/2999055328 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4504004380 con 0x7f4514100ac0 2024-05-10T04:41:59.388 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.385+0000 7f4510ff9700 1 -- 172.21.15.150:0/2999055328 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504003bf0 con 0x7f4514100ac0 2024-05-10T04:41:59.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.385+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45141a04f0 con 0x7f4514100ac0 2024-05-10T04:41:59.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.387+0000 7f4510ff9700 1 -- 172.21.15.150:0/2999055328 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4504003d50 con 0x7f4514100ac0 2024-05-10T04:41:59.389 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.388+0000 7f4510ff9700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44fc0a11c0 0x7f44fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.388+0000 7f4510ff9700 1 -- 172.21.15.150:0/2999055328 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f45040cc820 con 0x7f4514100ac0 2024-05-10T04:41:59.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.388+0000 7f4512ffd700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44fc0a11c0 0x7f44fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.390 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.388+0000 7f4512ffd700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44fc0a11c0 0x7f44fc0a36d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f45140687d0 tx=0x7f450800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.389+0000 7f4529ffcb80 1 Processor -- start 2024-05-10T04:41:59.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.389+0000 7f4529ffcb80 1 -- start start 2024-05-10T04:41:59.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.389+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab35af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.389+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab36980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.389+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab3d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.392 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.389+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361aa5c150 con 0x56361ab356a0 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361aa59b10 con 0x56361ab36ec0 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361aa55400 con 0x56361ab36570 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab35af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab35af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44038/0 (socket says 172.21.15.150:44038) 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 -- 172.21.15.150:0/2977337432 learned_addr learned my addr 172.21.15.150:0/2977337432 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f4513fff700 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab3d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f4512ffd700 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab36980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 -- 172.21.15.150:0/2977337432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 msgr2=0x56361ab36980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab36980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 -- 172.21.15.150:0/2977337432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 msgr2=0x56361ab3d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab3d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 -- 172.21.15.150:0/2977337432 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361a8d63b0 con 0x56361ab356a0 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f45137fe700 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab35af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4504099800 tx=0x7f4504003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3d28455f49e64b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.393 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f44f9ffb700 1 -- 172.21.15.150:0/2977337432 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504099870 con 0x56361ab356a0 2024-05-10T04:41:59.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f44f9ffb700 1 -- 172.21.15.150:0/2977337432 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450409d6a0 con 0x56361ab356a0 2024-05-10T04:41:59.394 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.390+0000 7f44f9ffb700 1 -- 172.21.15.150:0/2977337432 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45040b92c0 con 0x56361ab356a0 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2977337432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 msgr2=0x56361ab35af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab35af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4504099800 tx=0x7f4504003680 comp rx=0 tx=0).stop 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2977337432 shutdown_connections 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab3d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab35af0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2977337432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab36980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2977337432 >> 172.21.15.150:0/2977337432 conn(0x56361aa97a60 msgr2=0x56361ab2ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.398 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2977337432 shutdown_connections 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.391+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2977337432 wait complete. 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 -- start start 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 Processor -- start 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab3fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab46ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a9f1560 con 0x56361ab36570 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a9f1200 con 0x56361ab356a0 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361a8d86a0 con 0x56361ab36ec0 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4529ffcb80 10 client.0 _ll_register_callbacks cb 0x56361a9fab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.392+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab3fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab3fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44048/0 (socket says 172.21.15.150:44048) 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 -- 172.21.15.150:0/645223446 learned_addr learned my addr 172.21.15.150:0/645223446 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.399 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f45137fe700 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4513fff700 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab46ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 -- 172.21.15.150:0/645223446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 msgr2=0x56361ab46ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab46ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 -- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 msgr2=0x56361ab3f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 -- 172.21.15.150:0/645223446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45141a04f0 con 0x56361ab36570 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4512ffd700 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab3fb80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4508003680 tx=0x7f45080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd72cf4c03469f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f44f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f44f2ffd700 1 -- 172.21.15.150:0/645223446 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508010840 con 0x56361ab36570 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f44f2ffd700 1 -- 172.21.15.150:0/645223446 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4508003e20 con 0x56361ab36570 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361a9ae270 con 0x56361ab36570 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f44f2ffd700 1 -- 172.21.15.150:0/645223446 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45080109a0 con 0x56361ab36570 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4529ffcb80 10 client.16479 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:59.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.393+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56361a882b60 con 0x56361ab36570 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f44f2ffd700 1 -- 172.21.15.150:0/645223446 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56361a9ae270 con 0x56361ab36570 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f44f2ffd700 1 -- 172.21.15.150:0/645223446 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4508027c20 con 0x56361ab36570 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f4529ffcb80 10 client.16479 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f4529ffcb80 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f4529ffcb80 20 client.16479 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361ab4b370 0x56361ab4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f4529ffcb80 4 client.16479 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361ab04780 con 0x56361ab4b370 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.394+0000 7f45137fe700 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361ab4b370 0x56361ab4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f44f2ffd700 10 client.16479 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f45137fe700 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361ab4b370 0x56361ab4d7c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ddefa73bc9e21d3 server_cookie=f611d341404563fa in_seq=0 out_seq=0 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f44f2ffd700 1 -- 172.21.15.150:0/645223446 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56361ab04780 con 0x56361ab4b370 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f44f2ffd700 10 client.16479 handle_command_reply: tid=0 2024-05-10T04:41:59.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 client.16479 shutdown 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 msgr2=0x56361ab3fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab3fb80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4508003680 tx=0x7f45080047c0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 shutdown_connections 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361ab4b370 0x56361ab4d7c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab46ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f44f2ffd700 0 client.16479 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/645223446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab3fb80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.395+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 >> 172.21.15.150:0/645223446 conn(0x56361aa97a60 msgr2=0x56361ab2ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.396+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 shutdown_connections 2024-05-10T04:41:59.402 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.396+0000 7f4529ffcb80 1 -- 172.21.15.150:0/645223446 wait complete. 2024-05-10T04:41:59.403 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.396+0000 7f4529ffcb80 20 client.16479 trim_cache size 0 max 16384 2024-05-10T04:41:59.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 Processor -- start 2024-05-10T04:41:59.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 -- start start 2024-05-10T04:41:59.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab46aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab3f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ab47be0 con 0x56361ab36570 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ab31770 con 0x56361ab356a0 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ab3d670 con 0x56361ab36ec0 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab46aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab46aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44050/0 (socket says 172.21.15.150:44050) 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 -- 172.21.15.150:0/2152147401 learned_addr learned my addr 172.21.15.150:0/2152147401 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f45137fe700 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4513fff700 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab3f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 -- 172.21.15.150:0/2152147401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 msgr2=0x56361ab3f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab3f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 -- 172.21.15.150:0/2152147401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 msgr2=0x56361ab3ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4512ffd700 1 -- 172.21.15.150:0/2152147401 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361ab30280 con 0x56361ab36570 2024-05-10T04:41:59.418 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.414+0000 7f4513fff700 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab3f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.415+0000 7f4512ffd700 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab46aa0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f450800fc80 tx=0x7f4508005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b975906cc506a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.415+0000 7f44fa7fc700 1 -- 172.21.15.150:0/2152147401 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450801e680 con 0x56361ab36570 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.415+0000 7f44fa7fc700 1 -- 172.21.15.150:0/2152147401 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450801ed80 con 0x56361ab36570 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.415+0000 7f44fa7fc700 1 -- 172.21.15.150:0/2152147401 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508010ac0 con 0x56361ab36570 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2152147401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 msgr2=0x56361ab46aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab46aa0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f450800fc80 tx=0x7f4508005df0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2152147401 shutdown_connections 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36ec0 0x56361ab3f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab356a0 0x56361ab3ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/2152147401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab36570 0x56361ab46aa0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2152147401 >> 172.21.15.150:0/2152147401 conn(0x56361aa97a60 msgr2=0x56361ab35d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2152147401 shutdown_connections 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 -- 172.21.15.150:0/2152147401 wait complete. 2024-05-10T04:41:59.419 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 -- start start 2024-05-10T04:41:59.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.416+0000 7f4529ffcb80 1 Processor -- start 2024-05-10T04:41:59.423 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab44a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab44f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab45ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ab463e0 con 0x56361ab356a0 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ab46550 con 0x56361ab36ec0 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ab46710 con 0x56361ab36570 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 10 client.0 _ll_register_callbacks cb 0x56361a9fab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab44a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab44a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:44058/0 (socket says 172.21.15.150:44058) 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 -- 172.21.15.150:0/3791025617 learned_addr learned my addr 172.21.15.150:0/3791025617 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4513fff700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab45ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4512ffd700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab44f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.424 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 -- 172.21.15.150:0/3791025617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 msgr2=0x56361ab44f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab44f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 -- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 msgr2=0x56361ab45ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab45ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 -- 172.21.15.150:0/3791025617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450802c040 con 0x56361ab356a0 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4512ffd700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab44f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4513fff700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab45ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f45137fe700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab44a30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4504009520 tx=0x7f450400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1182f43116cff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f44f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f44f2ffd700 1 -- 172.21.15.150:0/3791025617 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4504063070 con 0x56361ab356a0 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f44f2ffd700 1 -- 172.21.15.150:0/3791025617 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f45040b2070 con 0x56361ab356a0 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.417+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f450409d6a0 con 0x56361ab356a0 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f44f2ffd700 1 -- 172.21.15.150:0/3791025617 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45040b2230 con 0x56361ab356a0 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 10 client.16485 fetch_fsmap learned FSMap version 29 2024-05-10T04:41:59.425 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56361aa1b560 con 0x56361ab356a0 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f44f2ffd700 1 -- 172.21.15.150:0/3791025617 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f450409d6a0 con 0x56361ab356a0 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f44f2ffd700 1 -- 172.21.15.150:0/3791025617 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f45040b9120 con 0x56361ab356a0 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 10 client.16485 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 20 client.16485 populate_metadata read hostname 'smithi150' 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361aba2430 0x56361aba4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 4 client.16485 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56361aa4dec0 con 0x56361aba2430 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.418+0000 7f4512ffd700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361aba2430 0x56361aba4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f44f2ffd700 10 client.16485 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f4512ffd700 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361aba2430 0x56361aba4850 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52bd0321233e2601 server_cookie=b0bf436576c92e54 in_seq=0 out_seq=0 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f44f2ffd700 1 -- 172.21.15.150:0/3791025617 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56361aa4dec0 con 0x56361aba2430 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f44f2ffd700 10 client.16485 handle_command_reply: tid=0 2024-05-10T04:41:59.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f4529ffcb80 1 client.16485 shutdown 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 msgr2=0x56361ab44a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab44a30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4504009520 tx=0x7f450400df90 comp rx=0 tx=0).stop 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 shutdown_connections 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56361aba2430 0x56361aba4850 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.419+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56361ab36ec0 0x56361ab45ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56361ab356a0 0x56361ab44a30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4529ffcb80 1 --2- 172.21.15.150:0/3791025617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56361ab36570 0x56361ab44f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f44f2ffd700 0 client.16485 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 >> 172.21.15.150:0/3791025617 conn(0x56361aa97a60 msgr2=0x56361ab3a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 shutdown_connections 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4529ffcb80 1 -- 172.21.15.150:0/3791025617 wait complete. 2024-05-10T04:41:59.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4529ffcb80 20 client.16485 trim_cache size 0 max 16384 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: "f09282df-00fa-4769-ac5b-e40368694587": { 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "f09282df-00fa-4769-ac5b-e40368694587", 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:41:59.428 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:41:59.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44fc0a11c0 msgr2=0x7f44fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44fc0a11c0 0x7f44fc0a36d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f45140687d0 tx=0x7f450800a380 comp rx=0 tx=0).stop 2024-05-10T04:41:59.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514100ac0 msgr2=0x7f4514068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:41:59.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514100ac0 0x7f4514068e30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f450400d950 tx=0x7f450400dc60 comp rx=0 tx=0).stop 2024-05-10T04:41:59.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 shutdown_connections 2024-05-10T04:41:59.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4514102580 0x7f451419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514100ac0 0x7f4514068e30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f44fc0a11c0 0x7f44fc0a36d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 --2- 172.21.15.150:0/2999055328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4514101be0 0x7f451419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:41:59.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 >> 172.21.15.150:0/2999055328 conn(0x7f45140fc0d0 msgr2=0x7f4514104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:41:59.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 shutdown_connections 2024-05-10T04:41:59.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:41:59.420+0000 7f4519cb8700 1 -- 172.21.15.150:0/2999055328 wait complete. 2024-05-10T04:41:59.435 INFO:tasks.cephfs.filesystem:scrub status for tag:f09282df-00fa-4769-ac5b-e40368694587 - {'path': '/', 'tag': 'f09282df-00fa-4769-ac5b-e40368694587', 'options': 'recursive,force'} 2024-05-10T04:41:59.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:41:59 smithi161 ceph-mon[46912]: pgmap v344: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 775 KiB/s wr, 13 op/s 2024-05-10T04:41:59.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:41:59 smithi181 ceph-mon[47314]: pgmap v344: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 775 KiB/s wr, 13 op/s 2024-05-10T04:41:59.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:41:59 smithi150 ceph-mon[42404]: pgmap v344: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 775 KiB/s wr, 13 op/s 2024-05-10T04:42:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:00 smithi161 ceph-mon[46912]: scrub queued for path: / 2024-05-10T04:42:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:00 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:00 smithi161 ceph-mon[46912]: scrub summary: active paths [/] 2024-05-10T04:42:00.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:00 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:00.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:00 smithi181 ceph-mon[47314]: scrub queued for path: / 2024-05-10T04:42:00.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:00 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:00.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:00 smithi181 ceph-mon[47314]: scrub summary: active paths [/] 2024-05-10T04:42:00.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:00 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:00.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:00 smithi150 ceph-mon[42404]: scrub queued for path: / 2024-05-10T04:42:00.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:00 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:00.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:00 smithi150 ceph-mon[42404]: scrub summary: active paths [/] 2024-05-10T04:42:00.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:00 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:01 smithi161 ceph-mon[46912]: pgmap v345: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:42:01.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:01 smithi181 ceph-mon[47314]: pgmap v345: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:42:01.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:01 smithi150 ceph-mon[42404]: pgmap v345: 129 pgs: 129 active+clean; 1.9 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 15 op/s 2024-05-10T04:42:03.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:02 smithi181 ceph-mon[47314]: scrub summary: idle 2024-05-10T04:42:03.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:02 smithi150 ceph-mon[42404]: scrub summary: idle 2024-05-10T04:42:03.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:02 smithi161 ceph-mon[46912]: scrub summary: idle 2024-05-10T04:42:04.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:03 smithi181 ceph-mon[47314]: pgmap v346: 129 pgs: 129 active+clean; 22 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.5 MiB/s wr, 17 op/s 2024-05-10T04:42:04.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:03 smithi150 ceph-mon[42404]: pgmap v346: 129 pgs: 129 active+clean; 22 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.5 MiB/s wr, 17 op/s 2024-05-10T04:42:04.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:03 smithi161 ceph-mon[46912]: pgmap v346: 129 pgs: 129 active+clean; 22 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.5 MiB/s wr, 17 op/s 2024-05-10T04:42:06.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:06 smithi161 ceph-mon[46912]: pgmap v347: 129 pgs: 129 active+clean; 198 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 21 op/s 2024-05-10T04:42:06.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:06 smithi181 ceph-mon[47314]: pgmap v347: 129 pgs: 129 active+clean; 198 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 21 op/s 2024-05-10T04:42:06.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:06 smithi150 ceph-mon[42404]: pgmap v347: 129 pgs: 129 active+clean; 198 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 MiB/s wr, 21 op/s 2024-05-10T04:42:08.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:08 smithi161 ceph-mon[46912]: pgmap v348: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-05-10T04:42:08.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:08 smithi181 ceph-mon[47314]: pgmap v348: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-05-10T04:42:08.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:08 smithi150 ceph-mon[42404]: pgmap v348: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 23 op/s 2024-05-10T04:42:10.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:10 smithi161 ceph-mon[46912]: pgmap v349: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:10.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:10 smithi181 ceph-mon[47314]: pgmap v349: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:10.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:10 smithi150 ceph-mon[42404]: pgmap v349: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 -- 172.21.15.150:0/394880738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 msgr2=0x7fa9041071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 --2- 172.21.15.150:0/394880738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa9041071f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009990 tx=0x7fa8f8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 -- 172.21.15.150:0/394880738 shutdown_connections 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 --2- 172.21.15.150:0/394880738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa904107880 0x7fa904109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 --2- 172.21.15.150:0/394880738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa904069320 0x7fa904104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 --2- 172.21.15.150:0/394880738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa9041071f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 -- 172.21.15.150:0/394880738 >> 172.21.15.150:0/394880738 conn(0x7fa9040fc0f0 msgr2=0x7fa9040fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 -- 172.21.15.150:0/394880738 shutdown_connections 2024-05-10T04:42:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.758+0000 7fa909189700 1 -- 172.21.15.150:0/394880738 wait complete. 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 Processor -- start 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 -- start start 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa904069320 0x7fa904197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa904198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa904107880 0x7fa90419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90419fa60 con 0x7fa904104d10 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90419fba0 con 0x7fa904107880 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa909189700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90419fd30 con 0x7fa904069320 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa90259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa904198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa90259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa904198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49930/0 (socket says 172.21.15.150:49930) 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa90259c700 1 -- 172.21.15.150:0/3718304744 learned_addr learned my addr 172.21.15.150:0/3718304744 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa902d9d700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa904069320 0x7fa904197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.759+0000 7fa90359e700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa904107880 0x7fa90419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa90259c700 1 -- 172.21.15.150:0/3718304744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa904069320 msgr2=0x7fa904197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa90259c700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa904069320 0x7fa904197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa90259c700 1 -- 172.21.15.150:0/3718304744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa904107880 msgr2=0x7fa90419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa90259c700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa904107880 0x7fa90419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa90259c700 1 -- 172.21.15.150:0/3718304744 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80095f0 con 0x7fa904104d10 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa90259c700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa904198500 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8005770 tx=0x7fa8f80057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa8ebfff700 1 -- 172.21.15.150:0/3718304744 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f801d070 con 0x7fa904104d10 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa8ebfff700 1 -- 172.21.15.150:0/3718304744 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8f8021410 con 0x7fa904104d10 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90419ffb0 con 0x7fa904104d10 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa8ebfff700 1 -- 172.21.15.150:0/3718304744 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8017650 con 0x7fa904104d10 2024-05-10T04:42:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.760+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9041a0320 con 0x7fa904104d10 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.763+0000 7fa8ebfff700 1 -- 172.21.15.150:0/3718304744 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa8f80177b0 con 0x7fa904104d10 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa8ebfff700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa8ec0a1120 0x7fa8ec0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa902d9d700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa8ec0a1120 0x7fa8ec0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa8ebfff700 1 -- 172.21.15.150:0/3718304744 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fa8f80ce0e0 con 0x7fa904104d10 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 Processor -- start 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 -- start start 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa902d9d700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa8ec0a1120 0x7fa8ec0a3630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa904069730 tx=0x7fa8f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f83821460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f838282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f837470f0 con 0x561f83821050 2024-05-10T04:42:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f83744ab0 con 0x561f838219a0 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.764+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f837403a0 con 0x561f83820180 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f83821460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f838282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f83821460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49932/0 (socket says 172.21.15.150:49932) 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90259c700 1 -- 172.21.15.150:0/2859007767 learned_addr learned my addr 172.21.15.150:0/2859007767 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa902d9d700 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90359e700 1 -- 172.21.15.150:0/2859007767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 msgr2=0x561f838205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90359e700 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90359e700 1 -- 172.21.15.150:0/2859007767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 msgr2=0x561f83821460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90359e700 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f83821460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90359e700 1 -- 172.21.15.150:0/2859007767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f835c1350 con 0x561f838219a0 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90259c700 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f83821460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa90359e700 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f838282a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc011580 tx=0x7fa8fc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ea2a8247711a5e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa8e97fa700 1 -- 172.21.15.150:0/2859007767 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00e040 con 0x561f838219a0 2024-05-10T04:42:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa8e97fa700 1 -- 172.21.15.150:0/2859007767 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8fc00f040 con 0x561f838219a0 2024-05-10T04:42:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.765+0000 7fa8e97fa700 1 -- 172.21.15.150:0/2859007767 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc004450 con 0x561f838219a0 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2859007767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 msgr2=0x561f838282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f838282a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc011580 tx=0x7fa8fc009920 comp rx=0 tx=0).stop 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2859007767 shutdown_connections 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f838282a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2859007767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f83821460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2859007767 >> 172.21.15.150:0/2859007767 conn(0x561f83782a00 msgr2=0x561f8381aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2859007767 shutdown_connections 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.766+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2859007767 wait complete. 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 -- start start 2024-05-10T04:42:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 Processor -- start 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83820180 0x561f8382a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83821050 0x561f8382a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f83831610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f836dc500 con 0x561f83820180 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f836dc1a0 con 0x561f838219a0 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f835c3640 con 0x561f83821050 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa9194cdb80 10 client.0 _ll_register_callbacks cb 0x561f836e5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa902d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83820180 0x561f8382a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa902d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83820180 0x561f8382a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49946/0 (socket says 172.21.15.150:49946) 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa902d9d700 1 -- 172.21.15.150:0/1353943218 learned_addr learned my addr 172.21.15.150:0/1353943218 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.767+0000 7fa90259c700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83821050 0x561f8382a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa90359e700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f83831610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa902d9d700 1 -- 172.21.15.150:0/1353943218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83821050 msgr2=0x561f8382a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa902d9d700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83821050 0x561f8382a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa902d9d700 1 -- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 msgr2=0x561f83831610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa902d9d700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f83831610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa902d9d700 1 -- 172.21.15.150:0/1353943218 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9041a0320 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa90359e700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f83831610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa902d9d700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83820180 0x561f8382a170 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa8f401d870 tx=0x7fa8f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5c5d08544fa65f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa8e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1353943218 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4022070 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f83699210 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1353943218 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fa8f4004020 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1353943218 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8f40041e0 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa9194cdb80 10 client.16497 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fa8fc011230 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.768+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1353943218 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f83699210 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1353943218 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa8f400e6d0 con 0x561f83820180 2024-05-10T04:42:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa9194cdb80 10 client.16497 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa9194cdb80 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa9194cdb80 20 client.16497 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f83835ea0 0x561f838382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa9194cdb80 4 client.16497 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f837ef720 con 0x561f83835ea0 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa90259c700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f83835ea0 0x561f838382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa8e6ff5700 10 client.16497 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.769+0000 7fa90259c700 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f83835ea0 0x561f838382f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24fdffb6ab7354b server_cookie=1962991bcad9f1c9 in_seq=0 out_seq=0 2024-05-10T04:42:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1353943218 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f837ef720 con 0x561f83835ea0 2024-05-10T04:42:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa8e6ff5700 10 client.16497 handle_command_reply: tid=0 2024-05-10T04:42:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 client.16497 shutdown 2024-05-10T04:42:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83820180 msgr2=0x561f8382a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83820180 0x561f8382a170 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa8f401d870 tx=0x7fa8f40047c0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 shutdown_connections 2024-05-10T04:42:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f83835ea0 0x561f838382f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f83831610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83820180 0x561f8382a170 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa8e6ff5700 0 client.16497 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1353943218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83821050 0x561f8382a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 >> 172.21.15.150:0/1353943218 conn(0x561f83782a00 msgr2=0x561f8381aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 shutdown_connections 2024-05-10T04:42:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.770+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1353943218 wait complete. 2024-05-10T04:42:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.771+0000 7fa9194cdb80 20 client.16497 trim_cache size 0 max 16384 2024-05-10T04:42:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.785+0000 7fa9194cdb80 1 Processor -- start 2024-05-10T04:42:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.785+0000 7fa9194cdb80 1 -- start start 2024-05-10T04:42:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f83821050 0x561f838284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f838219a0 0x561f83828a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f8381c240 con 0x561f838219a0 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f8383d4b0 con 0x561f83821050 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f838319c0 con 0x561f83820180 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f838219a0 0x561f83828a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f838219a0 0x561f83828a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49962/0 (socket says 172.21.15.150:49962) 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 -- 172.21.15.150:0/2060423517 learned_addr learned my addr 172.21.15.150:0/2060423517 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa902d9d700 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90259c700 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f83821050 0x561f838284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 -- 172.21.15.150:0/2060423517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 msgr2=0x561f838382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 -- 172.21.15.150:0/2060423517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f83821050 msgr2=0x561f838284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f83821050 0x561f838284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 -- 172.21.15.150:0/2060423517 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f8356db00 con 0x561f838219a0 2024-05-10T04:42:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.786+0000 7fa90359e700 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f838219a0 0x561f83828a00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00b010 tx=0x7fa8fc011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6270a7719a412887 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.787+0000 7fa8e9ffb700 1 -- 172.21.15.150:0/2060423517 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00e040 con 0x561f838219a0 2024-05-10T04:42:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.787+0000 7fa8e9ffb700 1 -- 172.21.15.150:0/2060423517 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8fc022070 con 0x561f838219a0 2024-05-10T04:42:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.787+0000 7fa8e9ffb700 1 -- 172.21.15.150:0/2060423517 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc007e60 con 0x561f838219a0 2024-05-10T04:42:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2060423517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f838219a0 msgr2=0x561f83828a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f838219a0 0x561f83828a00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00b010 tx=0x7fa8fc011ef0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2060423517 shutdown_connections 2024-05-10T04:42:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f838219a0 0x561f83828a00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f838382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/2060423517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f83821050 0x561f838284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2060423517 >> 172.21.15.150:0/2060423517 conn(0x561f83782a00 msgr2=0x561f83824280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2060423517 shutdown_connections 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 -- 172.21.15.150:0/2060423517 wait complete. 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 -- start start 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.788+0000 7fa9194cdb80 1 Processor -- start 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f83887490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f838879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa9194cdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f8388e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f8388ee70 con 0x561f83821050 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f8388efb0 con 0x561f838219a0 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa9194cdb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f8388f140 con 0x561f83820180 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa9194cdb80 10 client.0 _ll_register_callbacks cb 0x561f836e5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f838879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f8388e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa902d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f83887490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f838879d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49970/0 (socket says 172.21.15.150:49970) 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f8388e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:34436/0 (socket says 172.21.15.150:34436) 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 -- 172.21.15.150:0/1953776075 learned_addr learned my addr 172.21.15.150:0/1953776075 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 -- 172.21.15.150:0/1953776075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 msgr2=0x561f83887490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f83887490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 -- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 msgr2=0x561f8388e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f8388e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 -- 172.21.15.150:0/1953776075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f835c1350 con 0x561f83821050 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa902d9d700 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f83887490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90359e700 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f8388e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa90259c700 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f838879d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8097c30 tx=0x7fa8f8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ae78528e045788 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.789+0000 7fa8e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1953776075 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a35b0 con 0x561f83821050 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1953776075 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fa8f80cb8f0 con 0x561f83821050 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8fc00f040 con 0x561f83821050 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1953776075 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8f8099560 con 0x561f83821050 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 10 client.16503 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561f83706500 con 0x561f83821050 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1953776075 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8fc00f040 con 0x561f83821050 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1953776075 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa8f80a3120 con 0x561f83821050 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 10 client.16503 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 20 client.16503 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f838907e0 0x561f83892c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 4 client.16503 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f83738e60 con 0x561f838907e0 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.790+0000 7fa902d9d700 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f838907e0 0x561f83892c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.791+0000 7fa8e6ff5700 10 client.16503 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.791+0000 7fa902d9d700 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f838907e0 0x561f83892c00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1bf2b759486bfa server_cookie=3a00cf3aae47d6da in_seq=0 out_seq=0 2024-05-10T04:42:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.800+0000 7fa8e6ff5700 1 -- 172.21.15.150:0/1953776075 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f83738e60 con 0x561f838907e0 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.800+0000 7fa8e6ff5700 10 client.16503 handle_command_reply: tid=0 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.800+0000 7fa9194cdb80 1 client.16503 shutdown 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 msgr2=0x561f838879d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f838879d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8097c30 tx=0x7fa8f8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 shutdown_connections 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x561f838907e0 0x561f83892c00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f838219a0 0x561f8388e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561f83820180 0x561f83887490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa8e6ff5700 0 client.16503 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 --2- 172.21.15.150:0/1953776075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561f83821050 0x561f838879d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 >> 172.21.15.150:0/1953776075 conn(0x561f83782a00 msgr2=0x561f83828450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 shutdown_connections 2024-05-10T04:42:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 1 -- 172.21.15.150:0/1953776075 wait complete. 2024-05-10T04:42:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa9194cdb80 20 client.16503 trim_cache size 0 max 16384 2024-05-10T04:42:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:42:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:42:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:42:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.801+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa8ec0a1120 msgr2=0x7fa8ec0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa8ec0a1120 0x7fa8ec0a3630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa904069730 tx=0x7fa8f4009260 comp rx=0 tx=0).stop 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 msgr2=0x7fa904198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa904198500 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8005770 tx=0x7fa8f80057a0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 shutdown_connections 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa904107880 0x7fa90419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa8ec0a1120 0x7fa8ec0a3630 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa904069320 0x7fa904197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 --2- 172.21.15.150:0/3718304744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa904104d10 0x7fa904198500 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 >> 172.21.15.150:0/3718304744 conn(0x7fa9040fc0f0 msgr2=0x7fa904104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 shutdown_connections 2024-05-10T04:42:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:11.802+0000 7fa909189700 1 -- 172.21.15.150:0/3718304744 wait complete. 2024-05-10T04:42:11.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:42:11.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:42:11.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:42:11.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:42:11.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:11 smithi161 ceph-mon[46912]: pgmap v350: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-10T04:42:11.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:42:11.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:42:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:42:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:42:11.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:11 smithi181 ceph-mon[47314]: pgmap v350: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-10T04:42:11.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:42:11.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:42:11.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:42:11.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:42:11.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:11 smithi150 ceph-mon[42404]: pgmap v350: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 21 op/s 2024-05-10T04:42:14.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:14 smithi161 ceph-mon[46912]: pgmap v351: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:14.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:14 smithi181 ceph-mon[47314]: pgmap v351: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:14.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:14 smithi150 ceph-mon[42404]: pgmap v351: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:16.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:16 smithi161 ceph-mon[46912]: pgmap v352: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 21 op/s 2024-05-10T04:42:16.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:16 smithi181 ceph-mon[47314]: pgmap v352: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 21 op/s 2024-05-10T04:42:16.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:16 smithi150 ceph-mon[42404]: pgmap v352: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 21 op/s 2024-05-10T04:42:17.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:17 smithi161 ceph-mon[46912]: pgmap v353: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 17 op/s 2024-05-10T04:42:17.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:17 smithi181 ceph-mon[47314]: pgmap v353: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 17 op/s 2024-05-10T04:42:17.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:17 smithi150 ceph-mon[42404]: pgmap v353: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 17 op/s 2024-05-10T04:42:20.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:20 smithi161 ceph-mon[46912]: pgmap v354: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 860 KiB/s wr, 14 op/s 2024-05-10T04:42:20.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:20 smithi181 ceph-mon[47314]: pgmap v354: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 860 KiB/s wr, 14 op/s 2024-05-10T04:42:20.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:20 smithi150 ceph-mon[42404]: pgmap v354: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 860 KiB/s wr, 14 op/s 2024-05-10T04:42:22.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:22 smithi161 ceph-mon[46912]: pgmap v355: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-05-10T04:42:22.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:22 smithi181 ceph-mon[47314]: pgmap v355: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-05-10T04:42:22.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:22 smithi150 ceph-mon[42404]: pgmap v355: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 14 op/s 2024-05-10T04:42:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:23 smithi161 ceph-mon[46912]: pgmap v356: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 16 op/s 2024-05-10T04:42:23.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:23 smithi181 ceph-mon[47314]: pgmap v356: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 16 op/s 2024-05-10T04:42:23.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:23 smithi150 ceph-mon[42404]: pgmap v356: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 16 op/s 2024-05-10T04:42:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 -- 172.21.15.150:0/3034899022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 msgr2=0x7f112c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 --2- 172.21.15.150:0/3034899022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c102010 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1120009990 tx=0x7f1120009ca0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 -- 172.21.15.150:0/3034899022 shutdown_connections 2024-05-10T04:42:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 --2- 172.21.15.150:0/3034899022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f112c1025a0 0x7f112c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 --2- 172.21.15.150:0/3034899022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c100ae0 0x7f112c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 --2- 172.21.15.150:0/3034899022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c102010 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 -- 172.21.15.150:0/3034899022 >> 172.21.15.150:0/3034899022 conn(0x7f112c0fc0d0 msgr2=0x7f112c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 -- 172.21.15.150:0/3034899022 shutdown_connections 2024-05-10T04:42:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.934+0000 7f1130809700 1 -- 172.21.15.150:0/3034899022 wait complete. 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 Processor -- start 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 -- start start 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f112c100ae0 0x7f112c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c1025a0 0x7f112c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c19fa50 con 0x7f112c101c00 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c19fb90 con 0x7f112c1025a0 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1130809700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c19fd20 con 0x7f112c100ae0 2024-05-10T04:42:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1129d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1129d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39316/0 (socket says 172.21.15.150:39316) 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f1129d9b700 1 -- 172.21.15.150:0/1796349865 learned_addr learned my addr 172.21.15.150:0/1796349865 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f112a59c700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f112c100ae0 0x7f112c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.935+0000 7f112ad9d700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c1025a0 0x7f112c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1129d9b700 1 -- 172.21.15.150:0/1796349865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f112c100ae0 msgr2=0x7f112c197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1129d9b700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f112c100ae0 0x7f112c197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1129d9b700 1 -- 172.21.15.150:0/1796349865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c1025a0 msgr2=0x7f112c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1129d9b700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c1025a0 0x7f112c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1129d9b700 1 -- 172.21.15.150:0/1796349865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11200095f0 con 0x7f112c101c00 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1129d9b700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c1984f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1120005910 tx=0x7f1120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f111b7fe700 1 -- 172.21.15.150:0/1796349865 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112001d070 con 0x7f112c101c00 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f111b7fe700 1 -- 172.21.15.150:0/1796349865 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1120021410 con 0x7f112c101c00 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f111b7fe700 1 -- 172.21.15.150:0/1796349865 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120017650 con 0x7f112c101c00 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c19ffa0 con 0x7f112c101c00 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.936+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c1a0310 con 0x7f112c101c00 2024-05-10T04:42:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.938+0000 7f111b7fe700 1 -- 172.21.15.150:0/1796349865 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1120021580 con 0x7f112c101c00 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.939+0000 7f111b7fe700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11140a1120 0x7f11140a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.939+0000 7f111b7fe700 1 -- 172.21.15.150:0/1796349865 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f11200cdf90 con 0x7f112c101c00 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.939+0000 7f112a59c700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11140a1120 0x7f11140a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f112a59c700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11140a1120 0x7f11140a3630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f112c101b00 tx=0x7f111c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 Processor -- start 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 -- start start 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72bfef0f0 con 0x55b72c0c99a0 2024-05-10T04:42:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72bfecab0 con 0x55b72c0c8180 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.940+0000 7f1140b4db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72bfe83a0 con 0x55b72c0c9050 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39326/0 (socket says 172.21.15.150:39326) 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 -- 172.21.15.150:0/1553333411 learned_addr learned my addr 172.21.15.150:0/1553333411 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f1129d9b700 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112a59c700 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 -- 172.21.15.150:0/1553333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 msgr2=0x55b72c0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 -- 172.21.15.150:0/1553333411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 msgr2=0x55b72c0c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 -- 172.21.15.150:0/1553333411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b72be69350 con 0x55b72c0c99a0 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f112ad9d700 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d02a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1124011680 tx=0x7f1124009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1232d71d46f8de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f1118ff9700 1 -- 172.21.15.150:0/1553333411 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400e040 con 0x55b72c0c99a0 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f1118ff9700 1 -- 172.21.15.150:0/1553333411 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f112400f040 con 0x55b72c0c99a0 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.941+0000 7f1118ff9700 1 -- 172.21.15.150:0/1553333411 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240048c0 con 0x55b72c0c99a0 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 -- 172.21.15.150:0/1553333411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 msgr2=0x55b72c0d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d02a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1124011680 tx=0x7f1124009920 comp rx=0 tx=0).stop 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 -- 172.21.15.150:0/1553333411 shutdown_connections 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d02a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 --2- 172.21.15.150:0/1553333411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 -- 172.21.15.150:0/1553333411 >> 172.21.15.150:0/1553333411 conn(0x55b72c02aa00 msgr2=0x55b72c0c2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 -- 172.21.15.150:0/1553333411 shutdown_connections 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.942+0000 7f1140b4db80 1 -- 172.21.15.150:0/1553333411 wait complete. 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 -- start start 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 Processor -- start 2024-05-10T04:42:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c9050 0x55b72c0d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72bf84500 con 0x55b72c0c9050 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72bf841a0 con 0x55b72c0c8180 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72be6b640 con 0x55b72c0c99a0 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1140b4db80 10 client.0 _ll_register_callbacks cb 0x55b72bf8dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1129d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c9050 0x55b72c0d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1129d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c9050 0x55b72c0d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39336/0 (socket says 172.21.15.150:39336) 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1129d9b700 1 -- 172.21.15.150:0/2324536524 learned_addr learned my addr 172.21.15.150:0/2324536524 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f112ad9d700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f112a59c700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1129d9b700 1 -- 172.21.15.150:0/2324536524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 msgr2=0x55b72c0d9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1129d9b700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.943+0000 7f1129d9b700 1 -- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 msgr2=0x55b72c0d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1129d9b700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1129d9b700 1 -- 172.21.15.150:0/2324536524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1124011330 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f112a59c700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1129d9b700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c9050 0x55b72c0d2710 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f112009add0 tx=0x7f112009ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9c6a9dd9c774c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1111ffb700 1 -- 172.21.15.150:0/2324536524 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112009b0d0 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b72bf41210 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1111ffb700 1 -- 172.21.15.150:0/2324536524 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f112009ef00 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1111ffb700 1 -- 172.21.15.150:0/2324536524 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11200a34c0 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 10 client.16515 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b72be15b00 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1111ffb700 1 -- 172.21.15.150:0/2324536524 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11200b9030 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1111ffb700 1 -- 172.21.15.150:0/2324536524 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11200a3030 con 0x55b72c0c9050 2024-05-10T04:42:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 10 client.16515 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 20 client.16515 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c0ddaa0 0x55b72c0dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 4 client.16515 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.944+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b72c097720 con 0x55b72c0ddaa0 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.945+0000 7f112a59c700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c0ddaa0 0x55b72c0dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.945+0000 7f1111ffb700 10 client.16515 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.945+0000 7f112a59c700 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c0ddaa0 0x55b72c0dfef0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b641ea7a6a238e8 server_cookie=45a3d0c2752f2dc1 in_seq=0 out_seq=0 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1111ffb700 1 -- 172.21.15.150:0/2324536524 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b72c097720 con 0x55b72c0ddaa0 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1111ffb700 10 client.16515 handle_command_reply: tid=0 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 client.16515 shutdown 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c9050 msgr2=0x55b72c0d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c9050 0x55b72c0d2710 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f112009add0 tx=0x7f112009ad40 comp rx=0 tx=0).stop 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 shutdown_connections 2024-05-10T04:42:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c0ddaa0 0x55b72c0dfef0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1111ffb700 0 client.16515 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 --2- 172.21.15.150:0/2324536524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c9050 0x55b72c0d2710 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 >> 172.21.15.150:0/2324536524 conn(0x55b72c02aa00 msgr2=0x55b72c0c2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 shutdown_connections 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 1 -- 172.21.15.150:0/2324536524 wait complete. 2024-05-10T04:42:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.946+0000 7f1140b4db80 20 client.16515 trim_cache size 0 max 16384 2024-05-10T04:42:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.962+0000 7f1140b4db80 1 Processor -- start 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1140b4db80 1 -- start start 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c8180 0x55b72c0d9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c9050 0x55b72c0d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1140b4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72c0c4230 con 0x55b72c0c8180 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1140b4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72c0bd360 con 0x55b72c0c9050 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1140b4db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72c0ba9b0 con 0x55b72c0c99a0 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c8180 0x55b72c0d9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c8180 0x55b72c0d9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39346/0 (socket says 172.21.15.150:39346) 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 -- 172.21.15.150:0/3945062240 learned_addr learned my addr 172.21.15.150:0/3945062240 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f1129d9b700 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c9050 0x55b72c0d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112ad9d700 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 -- 172.21.15.150:0/3945062240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 msgr2=0x55b72c0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 -- 172.21.15.150:0/3945062240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c9050 msgr2=0x55b72c0d9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c9050 0x55b72c0d9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.963+0000 7f112a59c700 1 -- 172.21.15.150:0/3945062240 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b72bfae500 con 0x55b72c0c8180 2024-05-10T04:42:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.964+0000 7f112a59c700 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c8180 0x55b72c0d9c80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f112c1007d0 tx=0x7f111c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc538465cce797ea server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.964+0000 7f11197fa700 1 -- 172.21.15.150:0/3945062240 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c025070 con 0x55b72c0c8180 2024-05-10T04:42:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.964+0000 7f11197fa700 1 -- 172.21.15.150:0/3945062240 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f111c0201d0 con 0x55b72c0c8180 2024-05-10T04:42:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.964+0000 7f11197fa700 1 -- 172.21.15.150:0/3945062240 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c004380 con 0x55b72c0c8180 2024-05-10T04:42:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 -- 172.21.15.150:0/3945062240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c8180 msgr2=0x55b72c0d9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c8180 0x55b72c0d9c80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f112c1007d0 tx=0x7f111c014000 comp rx=0 tx=0).stop 2024-05-10T04:42:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 -- 172.21.15.150:0/3945062240 shutdown_connections 2024-05-10T04:42:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c99a0 0x55b72c0d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c8180 0x55b72c0d9c80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3945062240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c9050 0x55b72c0d9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 -- 172.21.15.150:0/3945062240 >> 172.21.15.150:0/3945062240 conn(0x55b72c02aa00 msgr2=0x55b72c0c2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 -- 172.21.15.150:0/3945062240 shutdown_connections 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 -- 172.21.15.150:0/3945062240 wait complete. 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 -- start start 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 Processor -- start 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0d75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.965+0000 7f1140b4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1140b4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72c0d89d0 con 0x55b72c0c99a0 2024-05-10T04:42:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1140b4db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72c0d8b40 con 0x55b72c0c8180 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1140b4db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b72c0d8d00 con 0x55b72c0c9050 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1140b4db80 10 client.0 _ll_register_callbacks cb 0x55b72bf8dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f112ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f112ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39356/0 (socket says 172.21.15.150:39356) 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f112ad9d700 1 -- 172.21.15.150:0/3437218669 learned_addr learned my addr 172.21.15.150:0/3437218669 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1129d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0d75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f112a59c700 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1129d9b700 1 -- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 msgr2=0x55b72c0d7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1129d9b700 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1129d9b700 1 -- 172.21.15.150:0/3437218669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 msgr2=0x55b72c0d8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1129d9b700 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1129d9b700 1 -- 172.21.15.150:0/3437218669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c017070 con 0x55b72c0c9050 2024-05-10T04:42:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f112ad9d700 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1129d9b700 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0d75b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f11200975d0 tx=0x7f1120000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e309c78fa1f762f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.966+0000 7f1111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:42:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1111ffb700 1 -- 172.21.15.150:0/3437218669 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112000fd00 con 0x55b72c0c9050 2024-05-10T04:42:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1111ffb700 1 -- 172.21.15.150:0/3437218669 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f112009a5c0 con 0x55b72c0c9050 2024-05-10T04:42:24.979 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f112400f040 con 0x55b72c0c9050 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1111ffb700 1 -- 172.21.15.150:0/3437218669 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f112000fe60 con 0x55b72c0c9050 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 10 client.26047 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b72be6dc70 con 0x55b72c0c9050 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1111ffb700 1 -- 172.21.15.150:0/3437218669 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11200a7080 con 0x55b72c0c9050 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1111ffb700 1 -- 172.21.15.150:0/3437218669 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f112009a030 con 0x55b72c0c9050 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 10 client.26047 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 20 client.26047 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c1344c0 0x55b72c1368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 4 client.26047 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-10T04:42:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.967+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b72bfe0e60 con 0x55b72c1344c0 2024-05-10T04:42:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.968+0000 7f112a59c700 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c1344c0 0x55b72c1368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.968+0000 7f1111ffb700 10 client.26047 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.968+0000 7f112a59c700 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c1344c0 0x55b72c1368e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7f5a7e66209eae server_cookie=39364761503844ab in_seq=0 out_seq=0 2024-05-10T04:42:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.979+0000 7f1111ffb700 1 -- 172.21.15.150:0/3437218669 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b72bfe0e60 con 0x55b72c1344c0 2024-05-10T04:42:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.979+0000 7f1111ffb700 10 client.26047 handle_command_reply: tid=0 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 client.26047 shutdown 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 msgr2=0x55b72c0d75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0d75b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f11200975d0 tx=0x7f1120000bc0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 shutdown_connections 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55b72c1344c0 0x55b72c1368e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b72c0c99a0 0x55b72c0d8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72c0c8180 0x55b72c0d7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 --2- 172.21.15.150:0/3437218669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b72c0c9050 0x55b72c0d75b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1111ffb700 0 client.26047 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 >> 172.21.15.150:0/3437218669 conn(0x55b72c02aa00 msgr2=0x55b72c0c2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 shutdown_connections 2024-05-10T04:42:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 1 -- 172.21.15.150:0/3437218669 wait complete. 2024-05-10T04:42:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1140b4db80 20 client.26047 trim_cache size 0 max 16384 2024-05-10T04:42:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:{ 2024-05-10T04:42:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "message": "", 2024-05-10T04:42:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout: "return_code": 0 2024-05-10T04:42:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stdout:} 2024-05-10T04:42:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11140a1120 msgr2=0x7f11140a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1130809700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11140a1120 0x7f11140a3630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f112c101b00 tx=0x7f111c009260 comp rx=0 tx=0).stop 2024-05-10T04:42:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 msgr2=0x7f112c1984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.980+0000 7f1130809700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c1984f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1120005910 tx=0x7f1120004730 comp rx=0 tx=0).stop 2024-05-10T04:42:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 shutdown_connections 2024-05-10T04:42:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f112c1025a0 0x7f112c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f11140a1120 0x7f11140a3630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f112c100ae0 0x7f112c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 --2- 172.21.15.150:0/1796349865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f112c101c00 0x7f112c1984f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 >> 172.21.15.150:0/1796349865 conn(0x7f112c0fc0d0 msgr2=0x7f112c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 shutdown_connections 2024-05-10T04:42:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi150.front.sepia.ceph.com.smithi150.stderr:2024-05-10T04:42:24.981+0000 7f1130809700 1 -- 172.21.15.150:0/1796349865 wait complete. 2024-05-10T04:42:26.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:26 smithi161 ceph-mon[46912]: pgmap v357: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-10T04:42:26.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:26 smithi181 ceph-mon[47314]: pgmap v357: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-10T04:42:26.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:26 smithi150 ceph-mon[42404]: pgmap v357: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-10T04:42:27.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:27 smithi161 ceph-mon[46912]: pgmap v358: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 16 op/s 2024-05-10T04:42:27.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:27 smithi181 ceph-mon[47314]: pgmap v358: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 16 op/s 2024-05-10T04:42:27.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:27 smithi150 ceph-mon[42404]: pgmap v358: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 863 KiB/s wr, 16 op/s 2024-05-10T04:42:29.436 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 -- 172.21.15.150:0/1974761725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 msgr2=0x7f4400108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 --2- 172.21.15.150:0/1974761725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400108f90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f43f800b0a0 tx=0x7f43f800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 -- 172.21.15.150:0/1974761725 shutdown_connections 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 --2- 172.21.15.150:0/1974761725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400108f90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 --2- 172.21.15.150:0/1974761725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400102030 0x7f4400102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 --2- 172.21.15.150:0/1974761725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4400100d30 0x7f4400101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 -- 172.21.15.150:0/1974761725 >> 172.21.15.150:0/1974761725 conn(0x7f44000fb8f0 msgr2=0x7f44000fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 -- 172.21.15.150:0/1974761725 shutdown_connections 2024-05-10T04:42:29.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.584+0000 7f44072b2700 1 -- 172.21.15.150:0/1974761725 wait complete. 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 Processor -- start 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 -- start start 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400100d30 0x7f4400196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4400102030 0x7f440019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440019e9c0 con 0x7f4400101680 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440019eb30 con 0x7f4400100d30 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.585+0000 7f44072b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440019ecf0 con 0x7f4400102030 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37622/0 (socket says 172.21.15.150:37622) 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 -- 172.21.15.150:0/4077148235 learned_addr learned my addr 172.21.15.150:0/4077148235 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:29.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440504e700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400100d30 0x7f4400196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440584f700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4400102030 0x7f440019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 -- 172.21.15.150:0/4077148235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4400102030 msgr2=0x7f440019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4400102030 0x7f440019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 -- 172.21.15.150:0/4077148235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400100d30 msgr2=0x7f4400196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400100d30 0x7f4400196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 -- 172.21.15.150:0/4077148235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8009d00 con 0x7f4400101680 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440504e700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400100d30 0x7f4400196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f440484d700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400197420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f43f000e9a0 tx=0x7f43f000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f43f67fc700 1 -- 172.21.15.150:0/4077148235 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0009790 con 0x7f4400101680 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440019efd0 con 0x7f4400101680 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f43f67fc700 1 -- 172.21.15.150:0/4077148235 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f000cbd0 con 0x7f4400101680 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.586+0000 7f43f67fc700 1 -- 172.21.15.150:0/4077148235 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0018680 con 0x7f4400101680 2024-05-10T04:42:29.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.587+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440019f400 con 0x7f4400101680 2024-05-10T04:42:29.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.589+0000 7f43f67fc700 1 -- 172.21.15.150:0/4077148235 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f43f000cd40 con 0x7f4400101680 2024-05-10T04:42:29.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.590+0000 7f43f67fc700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43ec0a11c0 0x7f43ec0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.590+0000 7f43f67fc700 1 -- 172.21.15.150:0/4077148235 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f43f0014070 con 0x7f4400101680 2024-05-10T04:42:29.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.590+0000 7f440504e700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43ec0a11c0 0x7f43ec0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.591 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.590+0000 7f44175f6b80 1 Processor -- start 2024-05-10T04:42:29.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f44175f6b80 1 -- start start 2024-05-10T04:42:29.592 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440504e700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43ec0a11c0 0x7f43ec0a36d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f44000681e0 tx=0x7f43fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce8daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce8e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f44175f6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bcdb4150 con 0x5638bce8eec0 2024-05-10T04:42:29.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f44175f6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bcdb1b10 con 0x5638bce8d6a0 2024-05-10T04:42:29.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f44175f6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bcdad400 con 0x5638bce8e570 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440504e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce8daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37626/0 (socket says 172.21.15.150:37626) 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 -- 172.21.15.150:0/3199135094 learned_addr learned my addr 172.21.15.150:0/3199135094 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440484d700 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce8e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 -- 172.21.15.150:0/3199135094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 msgr2=0x5638bce8e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce8e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 -- 172.21.15.150:0/3199135094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 msgr2=0x5638bce8daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce8daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440584f700 1 -- 172.21.15.150:0/3199135094 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638bcc2e3b0 con 0x5638bce8eec0 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.591+0000 7f440504e700 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce8daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.592+0000 7f440584f700 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce957c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f43f8015000 tx=0x7f43f8009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd60721ab847ee50 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.592+0000 7f43eb7fe700 1 -- 172.21.15.150:0/3199135094 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f800e050 con 0x5638bce8eec0 2024-05-10T04:42:29.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.592+0000 7f43eb7fe700 1 -- 172.21.15.150:0/3199135094 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f80075c0 con 0x5638bce8eec0 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.592+0000 7f43eb7fe700 1 -- 172.21.15.150:0/3199135094 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f80052a0 con 0x5638bce8eec0 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 -- 172.21.15.150:0/3199135094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 msgr2=0x5638bce957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce957c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f43f8015000 tx=0x7f43f8009c60 comp rx=0 tx=0).stop 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 -- 172.21.15.150:0/3199135094 shutdown_connections 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce957c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce8daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 --2- 172.21.15.150:0/3199135094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce8e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 -- 172.21.15.150:0/3199135094 >> 172.21.15.150:0/3199135094 conn(0x5638bcdefa60 msgr2=0x5638bce87fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 -- 172.21.15.150:0/3199135094 shutdown_connections 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 -- 172.21.15.150:0/3199135094 wait complete. 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 -- start start 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.593+0000 7f44175f6b80 1 Processor -- start 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce97640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8e570 0x5638bce97b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.600 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8eec0 0x5638bce9eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f44175f6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bcd49560 con 0x5638bce8e570 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f44175f6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bcd49200 con 0x5638bce8d6a0 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f44175f6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bcc306a0 con 0x5638bce8eec0 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f44175f6b80 10 client.0 _ll_register_callbacks cb 0x5638bcd52b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f440484d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8e570 0x5638bce97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f440484d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8e570 0x5638bce97b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37634/0 (socket says 172.21.15.150:37634) 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f440484d700 1 -- 172.21.15.150:0/2163871846 learned_addr learned my addr 172.21.15.150:0/2163871846 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f440504e700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce97640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.594+0000 7f440584f700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8eec0 0x5638bce9eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440484d700 1 -- 172.21.15.150:0/2163871846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8eec0 msgr2=0x5638bce9eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440484d700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8eec0 0x5638bce9eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440484d700 1 -- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 msgr2=0x5638bce97640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.601 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440484d700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce97640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440484d700 1 -- 172.21.15.150:0/2163871846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc011ad0 con 0x5638bce8e570 2024-05-10T04:42:29.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440584f700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8eec0 0x5638bce9eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:42:29.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440504e700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce97640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:29.602 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f440484d700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8e570 0x5638bce97b80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f43f0096ad0 tx=0x7f43f009a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af352a46da034e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f43e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f43e8ff9700 1 -- 172.21.15.150:0/2163871846 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00cadd0 con 0x5638bce8e570 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f43e8ff9700 1 -- 172.21.15.150:0/2163871846 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f43f00a24e0 con 0x5638bce8e570 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638bcd06270 con 0x5638bce8e570 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f43e8ff9700 1 -- 172.21.15.150:0/2163871846 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43f00a2050 con 0x5638bce8e570 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f44175f6b80 10 client.16533 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4400109940 con 0x5638bce8e570 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f43e8ff9700 1 -- 172.21.15.150:0/2163871846 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638bcd06270 con 0x5638bce8e570 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.595+0000 7f43e8ff9700 1 -- 172.21.15.150:0/2163871846 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43f0098fc0 con 0x5638bce8e570 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f44175f6b80 10 client.16533 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:29.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f44175f6b80 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f44175f6b80 20 client.16533 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f44175f6b80 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcea3290 0x5638bcea56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f44175f6b80 4 client.16533 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638bce5c780 con 0x5638bcea3290 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f440504e700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcea3290 0x5638bcea56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f43e8ff9700 10 client.16533 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.596+0000 7f440504e700 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcea3290 0x5638bcea56e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e24a1f1758f69c server_cookie=199cfee3bdd0b5d5 in_seq=0 out_seq=0 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f43e8ff9700 1 -- 172.21.15.150:0/2163871846 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638bce5c780 con 0x5638bcea3290 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f43e8ff9700 10 client.16533 handle_command_reply: tid=0 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 client.16533 shutdown 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8e570 msgr2=0x5638bce97b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8e570 0x5638bce97b80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f43f0096ad0 tx=0x7f43f009a5a0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 shutdown_connections 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcea3290 0x5638bcea56e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8eec0 0x5638bce9eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f43e8ff9700 0 client.16533 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:29.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce97640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 --2- 172.21.15.150:0/2163871846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8e570 0x5638bce97b80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 >> 172.21.15.150:0/2163871846 conn(0x5638bcdefa60 msgr2=0x5638bce87fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:29.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 shutdown_connections 2024-05-10T04:42:29.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.597+0000 7f44175f6b80 1 -- 172.21.15.150:0/2163871846 wait complete. 2024-05-10T04:42:29.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.598+0000 7f44175f6b80 20 client.16533 trim_cache size 0 max 16384 2024-05-10T04:42:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.615+0000 7f44175f6b80 1 Processor -- start 2024-05-10T04:42:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.615+0000 7f44175f6b80 1 -- start start 2024-05-10T04:42:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.615+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce84180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce96bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f44175f6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bcea3090 con 0x5638bce8eec0 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f44175f6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bce89770 con 0x5638bce8d6a0 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f44175f6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bce8e0f0 con 0x5638bce8e570 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440584f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce96bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440584f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce96bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37644/0 (socket says 172.21.15.150:37644) 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440504e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce84180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440584f700 1 -- 172.21.15.150:0/206918783 learned_addr learned my addr 172.21.15.150:0/206918783 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440484d700 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440504e700 1 -- 172.21.15.150:0/206918783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 msgr2=0x5638bce9eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440504e700 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440504e700 1 -- 172.21.15.150:0/206918783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 msgr2=0x5638bce96bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440504e700 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce96bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440504e700 1 -- 172.21.15.150:0/206918783 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638bcbdab60 con 0x5638bce8d6a0 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440584f700 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce96bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:29.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440484d700 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.616+0000 7f440504e700 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce84180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5638bcea5f90 tx=0x7f43fc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28222f4c18163c6f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.617+0000 7f43ebfff700 1 -- 172.21.15.150:0/206918783 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc028070 con 0x5638bce8d6a0 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.617+0000 7f43ebfff700 1 -- 172.21.15.150:0/206918783 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43fc028e60 con 0x5638bce8d6a0 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.617+0000 7f43ebfff700 1 -- 172.21.15.150:0/206918783 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc003e10 con 0x5638bce8d6a0 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 -- 172.21.15.150:0/206918783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 msgr2=0x5638bce84180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce84180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5638bcea5f90 tx=0x7f43fc014000 comp rx=0 tx=0).stop 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 -- 172.21.15.150:0/206918783 shutdown_connections 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8eec0 0x5638bce96bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8d6a0 0x5638bce84180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 --2- 172.21.15.150:0/206918783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 -- 172.21.15.150:0/206918783 >> 172.21.15.150:0/206918783 conn(0x5638bcdefa60 msgr2=0x5638bce87fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 -- 172.21.15.150:0/206918783 shutdown_connections 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 -- 172.21.15.150:0/206918783 wait complete. 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.618+0000 7f44175f6b80 1 -- start start 2024-05-10T04:42:29.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 1 Processor -- start 2024-05-10T04:42:29.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8d6a0 0x5638bce9c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8eec0 0x5638bce9d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bce9df30 con 0x5638bce8d6a0 2024-05-10T04:42:29.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bce9e0a0 con 0x5638bce8eec0 2024-05-10T04:42:29.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638bce9e260 con 0x5638bce8e570 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f44175f6b80 10 client.0 _ll_register_callbacks cb 0x5638bcd52b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f440504e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8d6a0 0x5638bce9c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f440504e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8d6a0 0x5638bce9c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37648/0 (socket says 172.21.15.150:37648) 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f440504e700 1 -- 172.21.15.150:0/691626427 learned_addr learned my addr 172.21.15.150:0/691626427 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f440484d700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.619+0000 7f440584f700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8eec0 0x5638bce9d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440504e700 1 -- 172.21.15.150:0/691626427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 msgr2=0x5638bce9cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440504e700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440504e700 1 -- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8eec0 msgr2=0x5638bce9d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440504e700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8eec0 0x5638bce9d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440504e700 1 -- 172.21.15.150:0/691626427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4400109940 con 0x5638bce8d6a0 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440484d700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:29.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440584f700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8eec0 0x5638bce9d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f440504e700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8d6a0 0x5638bce9c620 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5638bcea5f60 tx=0x7f43fc004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79426ad80a153dc server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f43e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f43e8ff9700 1 -- 172.21.15.150:0/691626427 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc028050 con 0x5638bce8d6a0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f43e8ff9700 1 -- 172.21.15.150:0/691626427 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f43fc018030 con 0x5638bce8d6a0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43f80075c0 con 0x5638bce8d6a0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f43e8ff9700 1 -- 172.21.15.150:0/691626427 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43fc0281b0 con 0x5638bce8d6a0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f44175f6b80 10 client.16539 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5638bcc2e3b0 con 0x5638bce8d6a0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.620+0000 7f43e8ff9700 1 -- 172.21.15.150:0/691626427 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f80075c0 con 0x5638bce8d6a0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f43e8ff9700 1 -- 172.21.15.150:0/691626427 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43fc03a070 con 0x5638bce8d6a0 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f44175f6b80 10 client.16539 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f44175f6b80 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f44175f6b80 20 client.16539 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:29.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f44175f6b80 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcefa280 0x5638bcefc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f44175f6b80 4 client.16539 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638bcda5ec0 con 0x5638bcefa280 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f440484d700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcefa280 0x5638bcefc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f43e8ff9700 10 client.16539 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f440484d700 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcefa280 0x5638bcefc6a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9950ecb6ba887ab5 server_cookie=6a0795e387e67344 in_seq=0 out_seq=0 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f43e8ff9700 1 -- 172.21.15.150:0/691626427 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5638bcda5ec0 con 0x5638bcefa280 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f43e8ff9700 10 client.16539 handle_command_reply: tid=0 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.621+0000 7f44175f6b80 1 client.16539 shutdown 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8d6a0 msgr2=0x5638bce9c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8d6a0 0x5638bce9c620 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5638bcea5f60 tx=0x7f43fc004650 comp rx=0 tx=0).stop 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 shutdown_connections 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5638bcefa280 0x5638bcefc6a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638bce8eec0 0x5638bce9d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5638bce8d6a0 0x5638bce9c620 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f43e8ff9700 0 client.16539 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:29.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 --2- 172.21.15.150:0/691626427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5638bce8e570 0x5638bce9cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 >> 172.21.15.150:0/691626427 conn(0x5638bcdefa60 msgr2=0x5638bce95d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:29.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 shutdown_connections 2024-05-10T04:42:29.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 1 -- 172.21.15.150:0/691626427 wait complete. 2024-05-10T04:42:29.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.622+0000 7f44175f6b80 20 client.16539 trim_cache size 0 max 16384 2024-05-10T04:42:29.630 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:42:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:42:29.630 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:42:29.630 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43ec0a11c0 msgr2=0x7f43ec0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43ec0a11c0 0x7f43ec0a36d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f44000681e0 tx=0x7f43fc009260 comp rx=0 tx=0).stop 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 msgr2=0x7f4400197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400197420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f43f000e9a0 tx=0x7f43f000ed60 comp rx=0 tx=0).stop 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 shutdown_connections 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f43ec0a11c0 0x7f43ec0a36d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4400102030 0x7f440019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400100d30 0x7f4400196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 --2- 172.21.15.150:0/4077148235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4400101680 0x7f4400197420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 >> 172.21.15.150:0/4077148235 conn(0x7f44000fb8f0 msgr2=0x7f4400104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 shutdown_connections 2024-05-10T04:42:29.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.623+0000 7f44072b2700 1 -- 172.21.15.150:0/4077148235 wait complete. 2024-05-10T04:42:29.642 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:42:29.642 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:42:29.785 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 -- 172.21.15.150:0/3444149015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 msgr2=0x7f458c1071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 --2- 172.21.15.150:0/3444149015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1071f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4578009990 tx=0x7f4578009ca0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 -- 172.21.15.150:0/3444149015 shutdown_connections 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 --2- 172.21.15.150:0/3444149015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c107880 0x7f458c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 --2- 172.21.15.150:0/3444149015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f458c069030 0x7f458c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 --2- 172.21.15.150:0/3444149015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1071f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 -- 172.21.15.150:0/3444149015 >> 172.21.15.150:0/3444149015 conn(0x7f458c0fc0d0 msgr2=0x7f458c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 -- 172.21.15.150:0/3444149015 shutdown_connections 2024-05-10T04:42:29.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.785+0000 7f4592aa4700 1 -- 172.21.15.150:0/3444149015 wait complete. 2024-05-10T04:42:29.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 Processor -- start 2024-05-10T04:42:29.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 -- start start 2024-05-10T04:42:29.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c069030 0x7f458c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f458c107880 0x7f458c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.789 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19fa70 con 0x7f458c104d50 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19fbe0 con 0x7f458c107880 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4592aa4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19fda0 con 0x7f458c069030 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f458bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4590840700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c069030 0x7f458c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f458bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37662/0 (socket says 172.21.15.150:37662) 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4590840700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c069030 0x7f458c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:52566/0 (socket says 172.21.15.150:52566) 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f458bfff700 1 -- 172.21.15.150:0/1788212852 learned_addr learned my addr 172.21.15.150:0/1788212852 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.786+0000 7f4591041700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f458c107880 0x7f458c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4590840700 1 -- 172.21.15.150:0/1788212852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f458c107880 msgr2=0x7f458c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4590840700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f458c107880 0x7f458c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4590840700 1 -- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 msgr2=0x7f458c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:29.790 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4590840700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4590840700 1 -- 172.21.15.150:0/1788212852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780095f0 con 0x7f458c069030 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f458bfff700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4590840700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c069030 0x7f458c197f90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f458000e9c0 tx=0x7f458000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4589ffb700 1 -- 172.21.15.150:0/1788212852 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458000c9c0 con 0x7f458c069030 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c100680 con 0x7f458c069030 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4589ffb700 1 -- 172.21.15.150:0/1788212852 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4580010430 con 0x7f458c069030 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4589ffb700 1 -- 172.21.15.150:0/1788212852 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580018680 con 0x7f458c069030 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.787+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c100be0 con 0x7f458c069030 2024-05-10T04:42:29.791 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.788+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c0fdc80 con 0x7f458c069030 2024-05-10T04:42:29.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.791+0000 7f4589ffb700 1 -- 172.21.15.150:0/1788212852 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f458000cb20 con 0x7f458c069030 2024-05-10T04:42:29.792 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.791+0000 7f4589ffb700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f457c0a1120 0x7f457c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:29.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.791+0000 7f4589ffb700 1 -- 172.21.15.150:0/1788212852 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4580014070 con 0x7f458c069030 2024-05-10T04:42:29.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.792+0000 7f458bfff700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f457c0a1120 0x7f457c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:29.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.792+0000 7f458bfff700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f457c0a1120 0x7f457c0a3630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4578006010 tx=0x7f457801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:29.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:29.795+0000 7f4589ffb700 1 -- 172.21.15.150:0/1788212852 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f45800963e0 con 0x7f458c069030 2024-05-10T04:42:30.051 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.051+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f458c0694d0 con 0x7f458c069030 2024-05-10T04:42:30.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.052+0000 7f4589ffb700 1 -- 172.21.15.150:0/1788212852 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f4580099a00 con 0x7f458c069030 2024-05-10T04:42:30.053 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:42:30.053 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:42:30.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f457c0a1120 msgr2=0x7f457c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f457c0a1120 0x7f457c0a3630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4578006010 tx=0x7f457801a040 comp rx=0 tx=0).stop 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c069030 msgr2=0x7f458c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c069030 0x7f458c197f90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f458000e9c0 tx=0x7f458000ecd0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 shutdown_connections 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f458c107880 0x7f458c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f457c0a1120 0x7f457c0a3630 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f458c069030 0x7f458c197f90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 --2- 172.21.15.150:0/1788212852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458c104d50 0x7f458c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 >> 172.21.15.150:0/1788212852 conn(0x7f458c0fc0d0 msgr2=0x7f458c1032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 shutdown_connections 2024-05-10T04:42:30.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.055+0000 7f4592aa4700 1 -- 172.21.15.150:0/1788212852 wait complete. 2024-05-10T04:42:30.057 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:42:30.071 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T04:42:30.216 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 -- 172.21.15.150:0/3347708487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0103690 msgr2=0x7fe5f0109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 --2- 172.21.15.150:0/3347708487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0103690 0x7fe5f0109f40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b0a0 tx=0x7fe5e800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 -- 172.21.15.150:0/3347708487 shutdown_connections 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 --2- 172.21.15.150:0/3347708487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0103690 0x7fe5f0109f40 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 --2- 172.21.15.150:0/3347708487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f0101bd0 0x7fe5f0101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 --2- 172.21.15.150:0/3347708487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5f0102cf0 0x7fe5f0103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 -- 172.21.15.150:0/3347708487 >> 172.21.15.150:0/3347708487 conn(0x7fe5f00fd1e0 msgr2=0x7fe5f00ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 -- 172.21.15.150:0/3347708487 shutdown_connections 2024-05-10T04:42:30.217 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.216+0000 7fe5f8080700 1 -- 172.21.15.150:0/3347708487 wait complete. 2024-05-10T04:42:30.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 Processor -- start 2024-05-10T04:42:30.219 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 -- start start 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0101bd0 0x7fe5f0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f0102cf0 0x7fe5f0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5f0103690 0x7fe5f019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f019faf0 con 0x7fe5f0101bd0 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f019fc60 con 0x7fe5f0103690 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f8080700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f019fe20 con 0x7fe5f0102cf0 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f5e1c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0101bd0 0x7fe5f0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f5e1c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0101bd0 0x7fe5f0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37692/0 (socket says 172.21.15.150:37692) 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f5e1c700 1 -- 172.21.15.150:0/1862766705 learned_addr learned my addr 172.21.15.150:0/1862766705 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f561b700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f0102cf0 0x7fe5f0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f661d700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5f0103690 0x7fe5f019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f5e1c700 1 -- 172.21.15.150:0/1862766705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f0102cf0 msgr2=0x7fe5f0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f5e1c700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f0102cf0 0x7fe5f0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.220 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f5e1c700 1 -- 172.21.15.150:0/1862766705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5f0103690 msgr2=0x7fe5f019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.217+0000 7fe5f5e1c700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5f0103690 0x7fe5f019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.218+0000 7fe5f5e1c700 1 -- 172.21.15.150:0/1862766705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8009d00 con 0x7fe5f0101bd0 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.218+0000 7fe5f5e1c700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0101bd0 0x7fe5f0198010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00d950 tx=0x7fe5ec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.218+0000 7fe5e6ffd700 1 -- 172.21.15.150:0/1862766705 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec004220 con 0x7fe5f0101bd0 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.218+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f01a0100 con 0x7fe5f0101bd0 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.218+0000 7fe5e6ffd700 1 -- 172.21.15.150:0/1862766705 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5ec00de30 con 0x7fe5f0101bd0 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.218+0000 7fe5e6ffd700 1 -- 172.21.15.150:0/1862766705 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec003d00 con 0x7fe5f0101bd0 2024-05-10T04:42:30.221 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.218+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f01a0610 con 0x7fe5f0101bd0 2024-05-10T04:42:30.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 Processor -- start 2024-05-10T04:42:30.222 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 -- start start 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988429af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a98842a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a988350150 con 0x55a9884296a0 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98834db10 con 0x55a98842a570 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a988349400 con 0x55a98842aec0 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988429af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988429af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37706/0 (socket says 172.21.15.150:37706) 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe5f661d700 1 -- 172.21.15.150:0/2803317494 learned_addr learned my addr 172.21.15.150:0/2803317494 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.222+0000 7fe5f561b700 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.225 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5f661d700 1 -- 172.21.15.150:0/2803317494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 msgr2=0x55a9884317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5f661d700 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5f661d700 1 -- 172.21.15.150:0/2803317494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 msgr2=0x55a98842a980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5f661d700 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a98842a980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5f661d700 1 -- 172.21.15.150:0/2803317494 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9881ca3b0 con 0x55a9884296a0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5f661d700 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988429af0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8015000 tx=0x7fe5e8009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27a6367b841b854 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5e6ffd700 1 -- 172.21.15.150:0/1862766705 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fe5ec004380 con 0x7fe5f0101bd0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5dbfff700 1 -- 172.21.15.150:0/2803317494 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800e040 con 0x55a9884296a0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5dbfff700 1 -- 172.21.15.150:0/2803317494 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e80077c0 con 0x55a9884296a0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5dbfff700 1 -- 172.21.15.150:0/2803317494 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801b830 con 0x55a9884296a0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.223+0000 7fe5e6ffd700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe5dc0a11c0 0x7fe5dc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe5f5e1c700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe5dc0a11c0 0x7fe5dc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe5e6ffd700 1 -- 172.21.15.150:0/1862766705 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe5ec0cc830 con 0x7fe5f0101bd0 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2803317494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 msgr2=0x55a988429af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.226 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988429af0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8015000 tx=0x7fe5e8009c40 comp rx=0 tx=0).stop 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe5f5e1c700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe5dc0a11c0 0x7fe5dc0a36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009520 tx=0x7fe5ec064dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2803317494 shutdown_connections 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988429af0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2803317494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a98842a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2803317494 >> 172.21.15.150:0/2803317494 conn(0x55a98838ba60 msgr2=0x55a988423fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2803317494 shutdown_connections 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.224+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2803317494 wait complete. 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 -- start start 2024-05-10T04:42:30.227 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 Processor -- start 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988433660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988433ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a98843ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9882e5560 con 0x55a9884296a0 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9882e5200 con 0x55a98842a570 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9881cc6a0 con 0x55a98842aec0 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe6083c4b80 10 client.0 _ll_register_callbacks cb 0x55a9882eeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988433660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.231 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988433660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37718/0 (socket says 172.21.15.150:37718) 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.225+0000 7fe5f661d700 1 -- 172.21.15.150:0/3726337672 learned_addr learned my addr 172.21.15.150:0/3726337672 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f561b700 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988433ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a98843ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f661d700 1 -- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 msgr2=0x55a98843ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a98843ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f661d700 1 -- 172.21.15.150:0/3726337672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 msgr2=0x55a988433ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988433ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f661d700 1 -- 172.21.15.150:0/3726337672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f010a9d0 con 0x55a9884296a0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988433660 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009a20 tx=0x7fe5e800b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646012b72d5abd1a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3726337672 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800e070 con 0x55a9884296a0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3726337672 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe5e801ba20 con 0x55a9884296a0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9882a2270 con 0x55a9884296a0 2024-05-10T04:42:30.232 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3726337672 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e801bbe0 con 0x55a9884296a0 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe6083c4b80 10 client.16557 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a988176b60 con 0x55a9884296a0 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.226+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3726337672 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9882a2270 con 0x55a9884296a0 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3726337672 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe5e8019040 con 0x55a9884296a0 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe6083c4b80 10 client.16557 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe6083c4b80 10 client.16557 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe6083c4b80 20 client.16557 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a98843ed90 0x55a9884411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe6083c4b80 4 client.16557 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9883f8780 con 0x55a98843ed90 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe5f561b700 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a98843ed90 0x55a9884411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe5d97fa700 10 client.16557 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.227+0000 7fe5f561b700 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a98843ed90 0x55a9884411e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5eca14fc4f28ae51 server_cookie=d50617fbb20c4dd8 in_seq=0 out_seq=0 2024-05-10T04:42:30.233 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.228+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3726337672 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9883f8780 con 0x55a98843ed90 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.228+0000 7fe5d97fa700 10 client.16557 handle_command_reply: tid=0 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.228+0000 7fe6083c4b80 1 client.16557 shutdown 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.228+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 msgr2=0x55a988433660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.228+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988433660 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009a20 tx=0x7fe5e800b020 comp rx=0 tx=0).stop 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 shutdown_connections 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a98843ed90 0x55a9884411e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a98843ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988433660 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3726337672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988433ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe5d97fa700 0 client.16557 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 >> 172.21.15.150:0/3726337672 conn(0x55a98838ba60 msgr2=0x55a988423fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 shutdown_connections 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3726337672 wait complete. 2024-05-10T04:42:30.234 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.229+0000 7fe6083c4b80 20 client.16557 trim_cache size 0 max 16384 2024-05-10T04:42:30.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.242+0000 7fe6083c4b80 1 Processor -- start 2024-05-10T04:42:30.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.242+0000 7fe6083c4b80 1 -- start start 2024-05-10T04:42:30.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.242+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a9884411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842a570 0x55a98843aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842aec0 0x55a98843b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a988429f50 con 0x55a9884296a0 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98843bc00 con 0x55a98842aec0 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a988425770 con 0x55a98842a570 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a9884411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a9884411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37730/0 (socket says 172.21.15.150:37730) 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 -- 172.21.15.150:0/2362846814 learned_addr learned my addr 172.21.15.150:0/2362846814 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f561b700 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842a570 0x55a98843aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 -- 172.21.15.150:0/2362846814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842a570 msgr2=0x55a98843aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842a570 0x55a98843aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 -- 172.21.15.150:0/2362846814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842aec0 msgr2=0x55a98843b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842aec0 0x55a98843b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 -- 172.21.15.150:0/2362846814 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a98830f560 con 0x55a9884296a0 2024-05-10T04:42:30.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5f661d700 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a9884411e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8005bb0 tx=0x7fe5e8004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85877abaecfa449 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.243+0000 7fe5e4ff9700 1 -- 172.21.15.150:0/2362846814 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800e040 con 0x55a9884296a0 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe5e4ff9700 1 -- 172.21.15.150:0/2362846814 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e801b440 con 0x55a9884296a0 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe5e4ff9700 1 -- 172.21.15.150:0/2362846814 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8012b40 con 0x55a9884296a0 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2362846814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 msgr2=0x55a9884411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a9884411e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8005bb0 tx=0x7fe5e8004dd0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2362846814 shutdown_connections 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842aec0 0x55a98843b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a9884411e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/2362846814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842a570 0x55a98843aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2362846814 >> 172.21.15.150:0/2362846814 conn(0x55a98838ba60 msgr2=0x55a988420180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2362846814 shutdown_connections 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.244+0000 7fe6083c4b80 1 -- 172.21.15.150:0/2362846814 wait complete. 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 -- start start 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 Processor -- start 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988437740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988437c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98843abc0 con 0x55a9884296a0 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98843ad30 con 0x55a98842a570 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98843aef0 con 0x55a98842aec0 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.245+0000 7fe6083c4b80 10 client.0 _ll_register_callbacks cb 0x55a9882eeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.255 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988437740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988437740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37746/0 (socket says 172.21.15.150:37746) 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 -- 172.21.15.150:0/3605124917 learned_addr learned my addr 172.21.15.150:0/3605124917 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f561b700 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988437c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 -- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 msgr2=0x55a9884931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884931f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 -- 172.21.15.150:0/3605124917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 msgr2=0x55a988437c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988437c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.256 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 -- 172.21.15.150:0/3605124917 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a988176b60 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5f661d700 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988437740 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8005b00 tx=0x7fe5e8004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae1c2f38c06bb2d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3605124917 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800e070 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5e80077c0 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3605124917 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fe5e801ce70 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3605124917 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e80124c0 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe6083c4b80 10 client.16563 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.246+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9881cecd0 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3605124917 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e80077c0 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3605124917 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe5e8046070 con 0x55a9884296a0 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe6083c4b80 10 client.16563 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe6083c4b80 10 client.16563 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe6083c4b80 20 client.16563 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a988494a50 0x55a988496e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe6083c4b80 4 client.16563 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a988341ec0 con 0x55a988494a50 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.247+0000 7fe5f561b700 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a988494a50 0x55a988496e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe5d97fa700 10 client.16563 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe5f561b700 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a988494a50 0x55a988496e70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d5b594d3b1a8ecfc server_cookie=d3f84533d13106f3 in_seq=0 out_seq=0 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe5d97fa700 1 -- 172.21.15.150:0/3605124917 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a988341ec0 con 0x55a988494a50 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe5d97fa700 10 client.16563 handle_command_reply: tid=0 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 client.16563 shutdown 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 msgr2=0x55a988437740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988437740 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8005b00 tx=0x7fe5e8004d60 comp rx=0 tx=0).stop 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 shutdown_connections 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x55a988494a50 0x55a988496e70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a98842aec0 0x55a9884931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9884296a0 0x55a988437740 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe5d97fa700 0 client.16563 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 --2- 172.21.15.150:0/3605124917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a98842a570 0x55a988437c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 >> 172.21.15.150:0/3605124917 conn(0x55a98838ba60 msgr2=0x55a988420180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 shutdown_connections 2024-05-10T04:42:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 1 -- 172.21.15.150:0/3605124917 wait complete. 2024-05-10T04:42:30.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.248+0000 7fe6083c4b80 20 client.16563 trim_cache size 0 max 16384 2024-05-10T04:42:30.260 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:42:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe5dc0a11c0 msgr2=0x7fe5dc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe5dc0a11c0 0x7fe5dc0a36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009520 tx=0x7fe5ec064dc0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0101bd0 msgr2=0x7fe5f0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0101bd0 0x7fe5f0198010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00d950 tx=0x7fe5ec00dd10 comp rx=0 tx=0).stop 2024-05-10T04:42:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 shutdown_connections 2024-05-10T04:42:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5f0103690 0x7fe5f019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5f0101bd0 0x7fe5f0198010 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fe5dc0a11c0 0x7fe5dc0a36d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 --2- 172.21.15.150:0/1862766705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f0102cf0 0x7fe5f0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 >> 172.21.15.150:0/1862766705 conn(0x7fe5f00fd1e0 msgr2=0x7fe5f0105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 shutdown_connections 2024-05-10T04:42:30.262 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.249+0000 7fe5f8080700 1 -- 172.21.15.150:0/1862766705 wait complete. 2024-05-10T04:42:30.264 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T04:42:30.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 -- 172.21.15.150:0/171105552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 msgr2=0x7f9364109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 --2- 172.21.15.150:0/171105552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 0x7f9364109250 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b0a0 tx=0x7f935c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 -- 172.21.15.150:0/171105552 shutdown_connections 2024-05-10T04:42:30.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 --2- 172.21.15.150:0/171105552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 0x7f9364109250 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.411 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 --2- 172.21.15.150:0/171105552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9364101110 0x7f9364101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 --2- 172.21.15.150:0/171105552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364102000 0x7f9364102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 -- 172.21.15.150:0/171105552 >> 172.21.15.150:0/171105552 conn(0x7f93640fb8f0 msgr2=0x7f93640fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 -- 172.21.15.150:0/171105552 shutdown_connections 2024-05-10T04:42:30.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.410+0000 7f936c625700 1 -- 172.21.15.150:0/171105552 wait complete. 2024-05-10T04:42:30.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 Processor -- start 2024-05-10T04:42:30.412 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 -- start start 2024-05-10T04:42:30.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364101110 0x7f936419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9364102000 0x7f936419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 0x7f93641a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93641a1f10 con 0x7f9364102950 2024-05-10T04:42:30.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93641a2080 con 0x7f9364101110 2024-05-10T04:42:30.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.411+0000 7f936c625700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93641a2240 con 0x7f9364102000 2024-05-10T04:42:30.414 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364101110 0x7f936419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364101110 0x7f936419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:50078/0 (socket says 172.21.15.150:50078) 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 -- 172.21.15.150:0/4134872025 learned_addr learned my addr 172.21.15.150:0/4134872025 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936abc2700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 0x7f93641a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f9369bc0700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9364102000 0x7f936419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 -- 172.21.15.150:0/4134872025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9364102000 msgr2=0x7f936419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9364102000 0x7f936419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 -- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 msgr2=0x7f93641a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 0x7f93641a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 -- 172.21.15.150:0/4134872025 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c009d00 con 0x7f9364101110 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936abc2700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 0x7f93641a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936a3c1700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364101110 0x7f936419a430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f936000e970 tx=0x7f936000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f935b7fe700 1 -- 172.21.15.150:0/4134872025 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360009790 con 0x7f9364101110 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.412+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93641a2520 con 0x7f9364101110 2024-05-10T04:42:30.415 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.413+0000 7f935b7fe700 1 -- 172.21.15.150:0/4134872025 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f936000c7e0 con 0x7f9364101110 2024-05-10T04:42:30.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.413+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93641a2a80 con 0x7f9364101110 2024-05-10T04:42:30.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.413+0000 7f935b7fe700 1 -- 172.21.15.150:0/4134872025 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600053f0 con 0x7f9364101110 2024-05-10T04:42:30.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.415+0000 7f935b7fe700 1 -- 172.21.15.150:0/4134872025 <== mon.2 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f936000cba0 con 0x7f9364101110 2024-05-10T04:42:30.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.416+0000 7f935b7fe700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93500a11c0 0x7f93500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.416+0000 7f935b7fe700 1 -- 172.21.15.150:0/4134872025 <== mon.2 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9360014070 con 0x7f9364101110 2024-05-10T04:42:30.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.416+0000 7f9369bc0700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93500a11c0 0x7f93500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.416+0000 7f9369bc0700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93500a11c0 0x7f93500a36d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f936419b4d0 tx=0x7f935400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.417+0000 7f937c969b80 1 Processor -- start 2024-05-10T04:42:30.417 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.417+0000 7f937c969b80 1 -- start start 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f937c969b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f937c969b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ac4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f937c969b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5acb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f937c969b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d59ea150 con 0x5605d5ac4ec0 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f937c969b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d59e7b10 con 0x5605d5ac36a0 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f937c969b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d59e3400 con 0x5605d5ac4570 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5acb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5acb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37758/0 (socket says 172.21.15.150:37758) 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 -- 172.21.15.150:0/2816111172 learned_addr learned my addr 172.21.15.150:0/2816111172 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.420 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936a3c1700 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f9369bc0700 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ac4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 -- 172.21.15.150:0/2816111172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 msgr2=0x5605d5ac4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ac4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 -- 172.21.15.150:0/2816111172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 msgr2=0x5605d5ac3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 -- 172.21.15.150:0/2816111172 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605d58643b0 con 0x5605d5ac4ec0 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f9369bc0700 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ac4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f936abc2700 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5acb7c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b0a0 tx=0x7f935c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3080bc63e91c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.418+0000 7f9358ff9700 1 -- 172.21.15.150:0/2816111172 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00e040 con 0x5605d5ac4ec0 2024-05-10T04:42:30.421 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f9358ff9700 1 -- 172.21.15.150:0/2816111172 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f935c009460 con 0x5605d5ac4ec0 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f9358ff9700 1 -- 172.21.15.150:0/2816111172 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c005040 con 0x5605d5ac4ec0 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f937c969b80 1 -- 172.21.15.150:0/2816111172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 msgr2=0x5605d5acb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f937c969b80 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5acb7c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b0a0 tx=0x7f935c0098b0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f937c969b80 1 -- 172.21.15.150:0/2816111172 shutdown_connections 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f937c969b80 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5acb7c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f937c969b80 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f937c969b80 1 --2- 172.21.15.150:0/2816111172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ac4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.419+0000 7f937c969b80 1 -- 172.21.15.150:0/2816111172 >> 172.21.15.150:0/2816111172 conn(0x5605d5a25a60 msgr2=0x5605d5abdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 -- 172.21.15.150:0/2816111172 shutdown_connections 2024-05-10T04:42:30.422 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 -- 172.21.15.150:0/2816111172 wait complete. 2024-05-10T04:42:30.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 -- start start 2024-05-10T04:42:30.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 Processor -- start 2024-05-10T04:42:30.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac36a0 0x5605d5acd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.426 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5acdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac4ec0 0x5605d5ad4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d597f560 con 0x5605d5ac36a0 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d597f200 con 0x5605d5ac4ec0 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d58666a0 con 0x5605d5ac4570 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.420+0000 7f937c969b80 10 client.0 _ll_register_callbacks cb 0x5605d5988b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac36a0 0x5605d5acd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac36a0 0x5605d5acd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37762/0 (socket says 172.21.15.150:37762) 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 -- 172.21.15.150:0/1884670582 learned_addr learned my addr 172.21.15.150:0/1884670582 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f9369bc0700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5acdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936abc2700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac4ec0 0x5605d5ad4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 -- 172.21.15.150:0/1884670582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 msgr2=0x5605d5acdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5acdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 -- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac4ec0 msgr2=0x5605d5ad4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.427 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac4ec0 0x5605d5ad4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 -- 172.21.15.150:0/1884670582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9364109b80 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f9369bc0700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5acdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936abc2700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac4ec0 0x5605d5ad4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f936a3c1700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac36a0 0x5605d5acd620 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f93600ca5c0 tx=0x7f9360099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f57b3c37064fa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f934dffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f934dffb700 1 -- 172.21.15.150:0/1884670582 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600a2390 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f934dffb700 1 -- 172.21.15.150:0/1884670582 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f93600982b0 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605d593c270 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f934dffb700 1 -- 172.21.15.150:0/1884670582 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9360005630 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f937c969b80 10 client.16575 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.421+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5605d5810b60 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f934dffb700 1 -- 172.21.15.150:0/1884670582 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605d593c270 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f934dffb700 1 -- 172.21.15.150:0/1884670582 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9360094b20 con 0x5605d5ac36a0 2024-05-10T04:42:30.428 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f937c969b80 10 client.16575 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f937c969b80 10 client.16575 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f937c969b80 20 client.16575 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f937c969b80 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5adb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f937c969b80 4 client.16575 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605d5a92780 con 0x5605d5ad92f0 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.422+0000 7f9369bc0700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5adb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.423+0000 7f934dffb700 10 client.16575 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.423+0000 7f9369bc0700 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5adb740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=243399e1dea9881d server_cookie=40fb60866b0fc1fc in_seq=0 out_seq=0 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.423+0000 7f934dffb700 1 -- 172.21.15.150:0/1884670582 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605d5a92780 con 0x5605d5ad92f0 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.423+0000 7f934dffb700 10 client.16575 handle_command_reply: tid=0 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.423+0000 7f937c969b80 1 client.16575 shutdown 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.423+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac36a0 msgr2=0x5605d5acd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.423+0000 7f937c969b80 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac36a0 0x5605d5acd620 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f93600ca5c0 tx=0x7f9360099fc0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 shutdown_connections 2024-05-10T04:42:30.429 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5adb740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac4ec0 0x5605d5ad4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f934dffb700 0 client.16575 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac36a0 0x5605d5acd620 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 --2- 172.21.15.150:0/1884670582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5acdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 >> 172.21.15.150:0/1884670582 conn(0x5605d5a25a60 msgr2=0x5605d5abdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 shutdown_connections 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 1 -- 172.21.15.150:0/1884670582 wait complete. 2024-05-10T04:42:30.430 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.424+0000 7f937c969b80 20 client.16575 trim_cache size 0 max 16384 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.437+0000 7f937c969b80 1 Processor -- start 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.437+0000 7f937c969b80 1 -- start start 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f937c969b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f937c969b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4570 0x5605d5ad4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f937c969b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4ec0 0x5605d5ac77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f937c969b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d5ac8c80 con 0x5605d5ac4570 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f937c969b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d5acb5e0 con 0x5605d5ac36a0 2024-05-10T04:42:30.438 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f937c969b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d5ac7ce0 con 0x5605d5ac4ec0 2024-05-10T04:42:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f9369bc0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4570 0x5605d5ad4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f9369bc0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4570 0x5605d5ad4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37770/0 (socket says 172.21.15.150:37770) 2024-05-10T04:42:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f9369bc0700 1 -- 172.21.15.150:0/4177679937 learned_addr learned my addr 172.21.15.150:0/4177679937 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936a3c1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.439 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936abc2700 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4ec0 0x5605d5ac77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936a3c1700 1 -- 172.21.15.150:0/4177679937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4ec0 msgr2=0x5605d5ac77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936a3c1700 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4ec0 0x5605d5ac77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936a3c1700 1 -- 172.21.15.150:0/4177679937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4570 msgr2=0x5605d5ad4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936a3c1700 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4570 0x5605d5ad4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936a3c1700 1 -- 172.21.15.150:0/4177679937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605d59a9560 con 0x5605d5ac36a0 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f9369bc0700 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4570 0x5605d5ad4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936abc2700 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4ec0 0x5605d5ac77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:30.440 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.438+0000 7f936a3c1700 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac4980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9360099b00 tx=0x7f9360005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbfc3db6aa81ee0a server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.442 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.442+0000 7f93597fa700 1 -- 172.21.15.150:0/4177679937 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936007c030 con 0x5605d5ac36a0 2024-05-10T04:42:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.442+0000 7f93597fa700 1 -- 172.21.15.150:0/4177679937 <== mon.2 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93600a2390 con 0x5605d5ac36a0 2024-05-10T04:42:30.443 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.442+0000 7f93597fa700 1 -- 172.21.15.150:0/4177679937 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360094b20 con 0x5605d5ac36a0 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 -- 172.21.15.150:0/4177679937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 msgr2=0x5605d5ac4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac4980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9360099b00 tx=0x7f9360005910 comp rx=0 tx=0).stop 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 -- 172.21.15.150:0/4177679937 shutdown_connections 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4ec0 0x5605d5ac77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ac4980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 --2- 172.21.15.150:0/4177679937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4570 0x5605d5ad4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 -- 172.21.15.150:0/4177679937 >> 172.21.15.150:0/4177679937 conn(0x5605d5a25a60 msgr2=0x5605d5aba180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 -- 172.21.15.150:0/4177679937 shutdown_connections 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.443+0000 7f937c969b80 1 -- 172.21.15.150:0/4177679937 wait complete. 2024-05-10T04:42:30.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 -- start start 2024-05-10T04:42:30.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 Processor -- start 2024-05-10T04:42:30.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ad1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ad1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5b2d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d5ad26b0 con 0x5605d5ac4ec0 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d5b2dd40 con 0x5605d5ac36a0 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d5b2df00 con 0x5605d5ac4570 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f937c969b80 10 client.0 _ll_register_callbacks cb 0x5605d5988b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f936abc2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5b2d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f936abc2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5b2d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37774/0 (socket says 172.21.15.150:37774) 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f936abc2700 1 -- 172.21.15.150:0/3297190257 learned_addr learned my addr 172.21.15.150:0/3297190257 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.444+0000 7f9369bc0700 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ad1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f936a3c1700 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ad1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f936abc2700 1 -- 172.21.15.150:0/3297190257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 msgr2=0x5605d5ad1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f936abc2700 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ad1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f936abc2700 1 -- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 msgr2=0x5605d5ad1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f936abc2700 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ad1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f936abc2700 1 -- 172.21.15.150:0/3297190257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9354020440 con 0x5605d5ac4ec0 2024-05-10T04:42:30.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f936abc2700 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5b2d7d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f935c009cd0 tx=0x7f935c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870d6c9ce8781275 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f934dffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f934dffb700 1 -- 172.21.15.150:0/3297190257 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00e040 con 0x5605d5ac4ec0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f934dffb700 1 -- 172.21.15.150:0/3297190257 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f935c00bdb0 con 0x5605d5ac4ec0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f935c009460 con 0x5605d5ac4ec0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f934dffb700 1 -- 172.21.15.150:0/3297190257 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f935c024460 con 0x5605d5ac4ec0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f937c969b80 10 client.16581 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5605d58643b0 con 0x5605d5ac4ec0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.445+0000 7f934dffb700 1 -- 172.21.15.150:0/3297190257 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f935c009460 con 0x5605d5ac4ec0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f934dffb700 1 -- 172.21.15.150:0/3297190257 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f935c033080 con 0x5605d5ac4ec0 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f937c969b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f937c969b80 10 client.16581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f937c969b80 20 client.16581 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f937c969b80 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5b31670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.450 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f937c969b80 4 client.16581 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5605d59dbec0 con 0x5605d5ad92f0 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f936a3c1700 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5b31670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f934dffb700 10 client.16581 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.446+0000 7f936a3c1700 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5b31670 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c683032c6c9cf99 server_cookie=ac41facd0e6c2424 in_seq=0 out_seq=0 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f934dffb700 1 -- 172.21.15.150:0/3297190257 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5605d59dbec0 con 0x5605d5ad92f0 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f934dffb700 10 client.16581 handle_command_reply: tid=0 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 client.16581 shutdown 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 msgr2=0x5605d5b2d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5b2d7d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f935c009cd0 tx=0x7f935c0099e0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 shutdown_connections 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x5605d5ad92f0 0x5605d5b31670 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5605d5ac4ec0 0x5605d5b2d7d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605d5ac36a0 0x5605d5ad1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f934dffb700 0 client.16581 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:42:30.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 --2- 172.21.15.150:0/3297190257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5605d5ac4570 0x5605d5ad1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 >> 172.21.15.150:0/3297190257 conn(0x5605d5a25a60 msgr2=0x5605d5aba180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 shutdown_connections 2024-05-10T04:42:30.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 1 -- 172.21.15.150:0/3297190257 wait complete. 2024-05-10T04:42:30.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f937c969b80 20 client.16581 trim_cache size 0 max 16384 2024-05-10T04:42:30.452 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.447+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93500a11c0 msgr2=0x7f93500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93500a11c0 0x7f93500a36d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f936419b4d0 tx=0x7f935400b410 comp rx=0 tx=0).stop 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364101110 msgr2=0x7f936419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364101110 0x7f936419a430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f936000e970 tx=0x7f936000ec80 comp rx=0 tx=0).stop 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 shutdown_connections 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9364102950 0x7f93641a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9364101110 0x7f936419a430 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f93500a11c0 0x7f93500a36d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 --2- 172.21.15.150:0/4134872025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9364102000 0x7f936419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 >> 172.21.15.150:0/4134872025 conn(0x7f93640fb8f0 msgr2=0x7f9364105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 shutdown_connections 2024-05-10T04:42:30.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.448+0000 7f936c625700 1 -- 172.21.15.150:0/4134872025 wait complete. 2024-05-10T04:42:30.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 -- 172.21.15.150:0/424960465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89000ffbd0 msgr2=0x7f89000fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 --2- 172.21.15.150:0/424960465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89000ffbd0 0x7f89000fffe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009990 tx=0x7f88f4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 -- 172.21.15.150:0/424960465 shutdown_connections 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 --2- 172.21.15.150:0/424960465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8900100520 0x7f890010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 --2- 172.21.15.150:0/424960465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8900103190 0x7f89001035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 --2- 172.21.15.150:0/424960465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89000ffbd0 0x7f89000fffe0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 -- 172.21.15.150:0/424960465 >> 172.21.15.150:0/424960465 conn(0x7f89000fb880 msgr2=0x7f89000fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 -- 172.21.15.150:0/424960465 shutdown_connections 2024-05-10T04:42:30.609 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.608+0000 7f8905ce0700 1 -- 172.21.15.150:0/424960465 wait complete. 2024-05-10T04:42:30.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 Processor -- start 2024-05-10T04:42:30.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 -- start start 2024-05-10T04:42:30.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89000ffbd0 0x7f8900102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8900100520 0x7f8900100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8900103190 0x7f8900101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8900101a20 con 0x7f8900103190 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8900101b90 con 0x7f8900100520 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f8905ce0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89001a1ff0 con 0x7f89000ffbd0 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f88fffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8900103190 0x7f8900101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f88fffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8900103190 0x7f8900101340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37794/0 (socket says 172.21.15.150:37794) 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f88fffff700 1 -- 172.21.15.150:0/3759894698 learned_addr learned my addr 172.21.15.150:0/3759894698 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89000ffbd0 0x7f8900102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.609+0000 7f88feffd700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8900100520 0x7f8900100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fffff700 1 -- 172.21.15.150:0/3759894698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89000ffbd0 msgr2=0x7f8900102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fffff700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89000ffbd0 0x7f8900102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fffff700 1 -- 172.21.15.150:0/3759894698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8900100520 msgr2=0x7f8900100e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fffff700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8900100520 0x7f8900100e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fffff700 1 -- 172.21.15.150:0/3759894698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40095f0 con 0x7f8900103190 2024-05-10T04:42:30.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fffff700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8900103190 0x7f8900101340 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f88f8007ee0 tx=0x7f88f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fcff9700 1 -- 172.21.15.150:0/3759894698 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8017070 con 0x7f8900103190 2024-05-10T04:42:30.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89001a2250 con 0x7f8900103190 2024-05-10T04:42:30.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fcff9700 1 -- 172.21.15.150:0/3759894698 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f800f040 con 0x7f8900103190 2024-05-10T04:42:30.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f88fcff9700 1 -- 172.21.15.150:0/3759894698 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8005580 con 0x7f8900103190 2024-05-10T04:42:30.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.610+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89001a26e0 con 0x7f8900103190 2024-05-10T04:42:30.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.613+0000 7f88fcff9700 1 -- 172.21.15.150:0/3759894698 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f88f800db20 con 0x7f8900103190 2024-05-10T04:42:30.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.613+0000 7f88fcff9700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88e80a11c0 0x7f88e80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 Processor -- start 2024-05-10T04:42:30.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88fcff9700 1 -- 172.21.15.150:0/3759894698 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f88f80cc430 con 0x7f8900103190 2024-05-10T04:42:30.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88e80a11c0 0x7f88e80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 -- start start 2024-05-10T04:42:30.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e3caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4d63150 con 0x560bf4e3d570 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4d60b10 con 0x560bf4e3dec0 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f8916024b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4d5c400 con 0x560bf4e3c6a0 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88feffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88feffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e3d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37796/0 (socket says 172.21.15.150:37796) 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88feffd700 1 -- 172.21.15.150:0/997719154 learned_addr learned my addr 172.21.15.150:0/997719154 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88e80a11c0 0x7f88e80a36d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8900069840 tx=0x7f88f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88fffff700 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.614+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e3caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88feffd700 1 -- 172.21.15.150:0/997719154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 msgr2=0x560bf4e3caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88feffd700 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e3caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88feffd700 1 -- 172.21.15.150:0/997719154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 msgr2=0x560bf4e447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88feffd700 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88feffd700 1 -- 172.21.15.150:0/997719154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bf4bdd3b0 con 0x560bf4e3d570 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e3caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88fffff700 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88feffd700 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e3d980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f88f4015000 tx=0x7f88f400b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfe1d20ca5e690f server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88e5ffb700 1 -- 172.21.15.150:0/997719154 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400bb50 con 0x560bf4e3d570 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88e5ffb700 1 -- 172.21.15.150:0/997719154 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f401e040 con 0x560bf4e3d570 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.615+0000 7f88e5ffb700 1 -- 172.21.15.150:0/997719154 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40223f0 con 0x560bf4e3d570 2024-05-10T04:42:30.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 -- 172.21.15.150:0/997719154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 msgr2=0x560bf4e3d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e3d980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f88f4015000 tx=0x7f88f400b750 comp rx=0 tx=0).stop 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 -- 172.21.15.150:0/997719154 shutdown_connections 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e3caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 --2- 172.21.15.150:0/997719154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e3d980 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 -- 172.21.15.150:0/997719154 >> 172.21.15.150:0/997719154 conn(0x560bf4d9ea60 msgr2=0x560bf4e36fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 -- 172.21.15.150:0/997719154 shutdown_connections 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 -- 172.21.15.150:0/997719154 wait complete. 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.616+0000 7f8916024b80 1 -- start start 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 1 Processor -- start 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e46650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3d570 0x560bf4e46b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3dec0 0x560bf4e4daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4cf8560 con 0x560bf4e3c6a0 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4cf8200 con 0x560bf4e3d570 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4bdf6a0 con 0x560bf4e3dec0 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f8916024b80 10 client.0 _ll_register_callbacks cb 0x560bf4d01b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f88ff7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e46650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f88ff7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e46650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37798/0 (socket says 172.21.15.150:37798) 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.617+0000 7f88ff7fe700 1 -- 172.21.15.150:0/563226454 learned_addr learned my addr 172.21.15.150:0/563226454 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88feffd700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3d570 0x560bf4e46b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88fffff700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3dec0 0x560bf4e4daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88ff7fe700 1 -- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3dec0 msgr2=0x560bf4e4daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3dec0 0x560bf4e4daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88ff7fe700 1 -- 172.21.15.150:0/563226454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3d570 msgr2=0x560bf4e46b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3d570 0x560bf4e46b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88ff7fe700 1 -- 172.21.15.150:0/563226454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890010aa00 con 0x560bf4e3c6a0 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88feffd700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3d570 0x560bf4e46b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88fffff700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3dec0 0x560bf4e4daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e46650 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f88f001d790 tx=0x7f88f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbd631d1879dfe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88deffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88deffd700 1 -- 172.21.15.150:0/563226454 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0004220 con 0x560bf4e3c6a0 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88deffd700 1 -- 172.21.15.150:0/563226454 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f88f0004380 con 0x560bf4e3c6a0 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560bf4cb5270 con 0x560bf4e3c6a0 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f88deffd700 1 -- 172.21.15.150:0/563226454 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f0004540 con 0x560bf4e3c6a0 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f8916024b80 10 client.16593 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.618+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560bf4b89b60 con 0x560bf4e3c6a0 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f88deffd700 1 -- 172.21.15.150:0/563226454 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560bf4cb5270 con 0x560bf4e3c6a0 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f88deffd700 1 -- 172.21.15.150:0/563226454 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88f000e6c0 con 0x560bf4e3c6a0 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f8916024b80 10 client.16593 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f8916024b80 10 client.16593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f8916024b80 20 client.16593 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f8916024b80 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4e522b0 0x560bf4e54700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f8916024b80 4 client.16593 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bf4e0b780 con 0x560bf4e522b0 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.619+0000 7f88feffd700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4e522b0 0x560bf4e54700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.620+0000 7f88deffd700 10 client.16593 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:42:30.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.620+0000 7f88feffd700 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4e522b0 0x560bf4e54700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ee14b29d697f21f server_cookie=94f8297d0cf9597b in_seq=0 out_seq=0 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f88deffd700 1 -- 172.21.15.150:0/563226454 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560bf4e0b780 con 0x560bf4e522b0 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f88deffd700 10 client.16593 handle_command_reply: tid=0 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 client.16593 shutdown 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 msgr2=0x560bf4e46650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e46650 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f88f001d790 tx=0x7f88f00047c0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 shutdown_connections 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4e522b0 0x560bf4e54700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3dec0 0x560bf4e4daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e46650 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f88deffd700 0 client.16593 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:42:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 --2- 172.21.15.150:0/563226454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3d570 0x560bf4e46b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 >> 172.21.15.150:0/563226454 conn(0x560bf4d9ea60 msgr2=0x560bf4e36fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 shutdown_connections 2024-05-10T04:42:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 1 -- 172.21.15.150:0/563226454 wait complete. 2024-05-10T04:42:30.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.621+0000 7f8916024b80 20 client.16593 trim_cache size 0 max 16384 2024-05-10T04:42:30.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.634+0000 7f8916024b80 1 Processor -- start 2024-05-10T04:42:30.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f8916024b80 1 -- start start 2024-05-10T04:42:30.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f8916024b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f8916024b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f8916024b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f8916024b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4e38b70 con 0x560bf4e3d570 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f8916024b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4e317c0 con 0x560bf4e3dec0 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f8916024b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4e40ce0 con 0x560bf4e3c6a0 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88feffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88feffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37800/0 (socket says 172.21.15.150:37800) 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88feffd700 1 -- 172.21.15.150:0/4196609986 learned_addr learned my addr 172.21.15.150:0/4196609986 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88fffff700 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88ff7fe700 1 -- 172.21.15.150:0/4196609986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 msgr2=0x560bf4e412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88ff7fe700 1 -- 172.21.15.150:0/4196609986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 msgr2=0x560bf4e407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88ff7fe700 1 -- 172.21.15.150:0/4196609986 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bf4d22560 con 0x560bf4e3c6a0 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88feffd700 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:30.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.635+0000 7f88fffff700 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.636+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4e850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f88f000efd0 tx=0x7f88f00099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3200cb115c255e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.636+0000 7f88e67fc700 1 -- 172.21.15.150:0/4196609986 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f000ec40 con 0x560bf4e3c6a0 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.636+0000 7f88e67fc700 1 -- 172.21.15.150:0/4196609986 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f000eda0 con 0x560bf4e3c6a0 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.636+0000 7f88e67fc700 1 -- 172.21.15.150:0/4196609986 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0003d90 con 0x560bf4e3c6a0 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 -- 172.21.15.150:0/4196609986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 msgr2=0x560bf4e4e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4e850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f88f000efd0 tx=0x7f88f00099f0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 -- 172.21.15.150:0/4196609986 shutdown_connections 2024-05-10T04:42:30.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4e412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4e850 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 --2- 172.21.15.150:0/4196609986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3d570 0x560bf4e407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 -- 172.21.15.150:0/4196609986 >> 172.21.15.150:0/4196609986 conn(0x560bf4d9ea60 msgr2=0x560bf4e36f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 -- 172.21.15.150:0/4196609986 shutdown_connections 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 -- 172.21.15.150:0/4196609986 wait complete. 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 -- start start 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.637+0000 7f8916024b80 1 Processor -- start 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f8916024b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f8916024b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3d570 0x560bf4e4e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f8916024b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4ea6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f8916024b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4ea70b0 con 0x560bf4e3c6a0 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f8916024b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4ea71f0 con 0x560bf4e3dec0 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f8916024b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bf4ea73b0 con 0x560bf4e3d570 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f8916024b80 10 client.0 _ll_register_callbacks cb 0x560bf4d01b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88ff7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88feffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3d570 0x560bf4e4e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88ff7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37802/0 (socket says 172.21.15.150:37802) 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88ff7fe700 1 -- 172.21.15.150:0/3168406514 learned_addr learned my addr 172.21.15.150:0/3168406514 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88fffff700 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4ea6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88feffd700 1 -- 172.21.15.150:0/3168406514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 msgr2=0x560bf4ea6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88feffd700 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4ea6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88feffd700 1 -- 172.21.15.150:0/3168406514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 msgr2=0x560bf4e4dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88feffd700 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88feffd700 1 -- 172.21.15.150:0/3168406514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bf4bdd3b0 con 0x560bf4e3d570 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88feffd700 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3d570 0x560bf4e4e220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f88f40095c0 tx=0x7f88f400b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb535f45cfde302b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.638+0000 7f88deffd700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f88deffd700 1 -- 172.21.15.150:0/3168406514 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4032040 con 0x560bf4e3d570 2024-05-10T04:42:30.649 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88f401e040 con 0x560bf4e3d570 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f88deffd700 1 -- 172.21.15.150:0/3168406514 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f88f40342b0 con 0x560bf4e3d570 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f88deffd700 1 -- 172.21.15.150:0/3168406514 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f4034470 con 0x560bf4e3d570 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 10 client.26116 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560bf4b89b60 con 0x560bf4e3d570 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f88deffd700 1 -- 172.21.15.150:0/3168406514 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f401e040 con 0x560bf4e3d570 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f88deffd700 1 -- 172.21.15.150:0/3168406514 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88f4021070 con 0x560bf4e3d570 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 10 client.26116 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 20 client.26116 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.650 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4ea8810 0x560bf4eaac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 4 client.26116 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.639+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560bf4d54ec0 con 0x560bf4ea8810 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4ea8810 0x560bf4eaac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f88deffd700 10 client.26116 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f88ff7fe700 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4ea8810 0x560bf4eaac30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50ee415e50539a6e server_cookie=da8e70261095de3 in_seq=0 out_seq=0 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f88deffd700 1 -- 172.21.15.150:0/3168406514 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560bf4d54ec0 con 0x560bf4ea8810 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f88deffd700 10 client.26116 handle_command_reply: tid=0 2024-05-10T04:42:30.651 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f8916024b80 1 client.26116 shutdown 2024-05-10T04:42:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3d570 msgr2=0x560bf4e4e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.640+0000 7f8916024b80 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3d570 0x560bf4e4e220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f88f40095c0 tx=0x7f88f400b850 comp rx=0 tx=0).stop 2024-05-10T04:42:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 shutdown_connections 2024-05-10T04:42:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x560bf4ea8810 0x560bf4eaac30 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560bf4e3dec0 0x560bf4ea6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.652 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560bf4e3c6a0 0x560bf4e4dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 --2- 172.21.15.150:0/3168406514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560bf4e3d570 0x560bf4e4e220 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f88deffd700 0 client.26116 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:42:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 >> 172.21.15.150:0/3168406514 conn(0x560bf4d9ea60 msgr2=0x560bf4e3b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 shutdown_connections 2024-05-10T04:42:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 1 -- 172.21.15.150:0/3168406514 wait complete. 2024-05-10T04:42:30.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8916024b80 20 client.26116 trim_cache size 0 max 16384 2024-05-10T04:42:30.654 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:42:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88e80a11c0 msgr2=0x7f88e80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88e80a11c0 0x7f88e80a36d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8900069840 tx=0x7f88f0009260 comp rx=0 tx=0).stop 2024-05-10T04:42:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8900103190 msgr2=0x7f8900101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8900103190 0x7f8900101340 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f88f8007ee0 tx=0x7f88f800d3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 shutdown_connections 2024-05-10T04:42:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8900103190 0x7f8900101340 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.655 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f88e80a11c0 0x7f88e80a36d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89000ffbd0 0x7f8900102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 --2- 172.21.15.150:0/3759894698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8900100520 0x7f8900100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 >> 172.21.15.150:0/3759894698 conn(0x7f89000fb880 msgr2=0x7f8900106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 shutdown_connections 2024-05-10T04:42:30.656 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.641+0000 7f8905ce0700 1 -- 172.21.15.150:0/3759894698 wait complete. 2024-05-10T04:42:30.657 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:42:30.694 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:30 smithi150 ceph-mon[42404]: pgmap v359: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 861 KiB/s wr, 14 op/s 2024-05-10T04:42:30.694 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:30 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1788212852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:42:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/946252977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 msgr2=0x7f8fe010a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/946252977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe010a140 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800b0a0 tx=0x7f8fd800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/946252977 shutdown_connections 2024-05-10T04:42:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/946252977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe010a140 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/946252977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fe0103190 0x7f8fe01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/946252977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe00ffbd0 0x7f8fe00fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.813 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/946252977 >> 172.21.15.150:0/946252977 conn(0x7f8fe00fb880 msgr2=0x7f8fe00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/946252977 shutdown_connections 2024-05-10T04:42:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.811+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/946252977 wait complete. 2024-05-10T04:42:30.814 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 Processor -- start 2024-05-10T04:42:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 -- start start 2024-05-10T04:42:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe00ffbd0 0x7f8fe0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fe0103190 0x7f8fe019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe019fb20 con 0x7f8fe0100520 2024-05-10T04:42:30.830 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe019fc90 con 0x7f8fe00ffbd0 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fe5c4b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe019fe50 con 0x7f8fe0103190 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe01985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37828/0 (socket says 172.21.15.150:37828) 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.812+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1686837055 learned_addr learned my addr 172.21.15.150:0/1686837055 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdffff700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fe0103190 0x7f8fe019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe00ffbd0 0x7f8fe0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1686837055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fe0103190 msgr2=0x7f8fe019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fe0103190 0x7f8fe019f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1686837055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe00ffbd0 msgr2=0x7f8fe0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe00ffbd0 0x7f8fe0198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.831 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1686837055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009d00 con 0x7f8fe0100520 2024-05-10T04:42:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe01985d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400d950 tx=0x7f8fd400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdcff9700 1 -- 172.21.15.150:0/1686837055 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4009b40 con 0x7f8fe0100520 2024-05-10T04:42:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe01a0130 con 0x7f8fe0100520 2024-05-10T04:42:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdcff9700 1 -- 172.21.15.150:0/1686837055 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd4009ca0 con 0x7f8fe0100520 2024-05-10T04:42:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fdcff9700 1 -- 172.21.15.150:0/1686837055 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd400f620 con 0x7f8fe0100520 2024-05-10T04:42:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.813+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe01a0690 con 0x7f8fe0100520 2024-05-10T04:42:30.832 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.816+0000 7f8fdcff9700 1 -- 172.21.15.150:0/1686837055 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8fd400f780 con 0x7f8fe0100520 2024-05-10T04:42:30.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.817+0000 7f8fdcff9700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8fc80a11c0 0x7f8fc80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.817+0000 7f8fdcff9700 1 -- 172.21.15.150:0/1686837055 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f8fd40cd050 con 0x7f8fe0100520 2024-05-10T04:42:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.817+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8fc80a11c0 0x7f8fc80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.817+0000 7f8ff5f8fb80 1 Processor -- start 2024-05-10T04:42:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.817+0000 7f8ff5f8fb80 1 -- start start 2024-05-10T04:42:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf60af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf61980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61ec0 0x55c7faf687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fae87150 con 0x55c7faf61570 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fae84b10 con 0x55c7faf606a0 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fae80400 con 0x55c7faf61ec0 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8fc80a11c0 0x7f8fc80a36d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0069840 tx=0x7f8fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf61980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf61980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37844/0 (socket says 172.21.15.150:37844) 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 -- 172.21.15.150:0/2538074726 learned_addr learned my addr 172.21.15.150:0/2538074726 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf60af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdffff700 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61ec0 0x55c7faf687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 -- 172.21.15.150:0/2538074726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61ec0 msgr2=0x55c7faf687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61ec0 0x55c7faf687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 -- 172.21.15.150:0/2538074726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 msgr2=0x55c7faf60af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf60af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 -- 172.21.15.150:0/2538074726 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7fad013b0 con 0x55c7faf61570 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf61980 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8fd409a090 tx=0x7f8fd4099d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259129e024646495 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fc5ffb700 1 -- 172.21.15.150:0/2538074726 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd409a100 con 0x55c7faf61570 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fc5ffb700 1 -- 172.21.15.150:0/2538074726 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd409df30 con 0x55c7faf61570 2024-05-10T04:42:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.818+0000 7f8fc5ffb700 1 -- 172.21.15.150:0/2538074726 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4099850 con 0x55c7faf61570 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/2538074726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 msgr2=0x55c7faf61980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf61980 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8fd409a090 tx=0x7f8fd4099d10 comp rx=0 tx=0).stop 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/2538074726 shutdown_connections 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61ec0 0x55c7faf687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf60af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/2538074726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf61980 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/2538074726 >> 172.21.15.150:0/2538074726 conn(0x55c7faec2a60 msgr2=0x55c7faf5afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/2538074726 shutdown_connections 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.819+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/2538074726 wait complete. 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 -- start start 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 Processor -- start 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf6a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.837 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61570 0x55c7faf6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7faf719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fae1c560 con 0x55c7faf61ec0 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fae1c200 con 0x55c7faf61570 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fad036a0 con 0x55c7faf606a0 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.820+0000 7f8ff5f8fb80 10 client.0 _ll_register_callbacks cb 0x55c7fae25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7faf719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7faf719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37860/0 (socket says 172.21.15.150:37860) 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 -- 172.21.15.150:0/1204026959 learned_addr learned my addr 172.21.15.150:0/1204026959 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf6a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61570 0x55c7faf6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 -- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 msgr2=0x55c7faf6a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf6a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 -- 172.21.15.150:0/1204026959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61570 msgr2=0x55c7faf6aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61570 0x55c7faf6aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 -- 172.21.15.150:0/1204026959 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe01a0690 con 0x55c7faf61ec0 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fdffff700 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7faf719c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8015000 tx=0x7f8fd80078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f189e7f56393775c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fbeffd700 1 -- 172.21.15.150:0/1204026959 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800e040 con 0x55c7faf61ec0 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fbeffd700 1 -- 172.21.15.150:0/1204026959 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f8fd8022070 con 0x55c7faf61ec0 2024-05-10T04:42:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7fadd9270 con 0x55c7faf61ec0 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8fbeffd700 1 -- 172.21.15.150:0/1204026959 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd8004a50 con 0x55c7faf61ec0 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.821+0000 7f8ff5f8fb80 10 client.16611 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c7facadb60 con 0x55c7faf61ec0 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8fbeffd700 1 -- 172.21.15.150:0/1204026959 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7fadd9270 con 0x55c7faf61ec0 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8fbeffd700 1 -- 172.21.15.150:0/1204026959 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fd800b0d0 con 0x55c7faf61ec0 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8ff5f8fb80 10 client.16611 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8ff5f8fb80 10 client.16611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8ff5f8fb80 20 client.16611 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7faf76140 0x55c7faf78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8ff5f8fb80 4 client.16611 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7faf2f780 con 0x55c7faf76140 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.822+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7faf76140 0x55c7faf78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.823+0000 7f8fbeffd700 10 client.16611 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:42:30.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.823+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7faf76140 0x55c7faf78590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdcd83bf07d6a8af server_cookie=d01c54b3a78bbc58 in_seq=0 out_seq=0 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.823+0000 7f8fbeffd700 1 -- 172.21.15.150:0/1204026959 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7faf2f780 con 0x55c7faf76140 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.823+0000 7f8fbeffd700 10 client.16611 handle_command_reply: tid=0 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.823+0000 7f8ff5f8fb80 1 client.16611 shutdown 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 msgr2=0x55c7faf719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7faf719c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8015000 tx=0x7f8fd80078c0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 shutdown_connections 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7faf76140 0x55c7faf78590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7faf719c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf6a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1204026959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61570 0x55c7faf6aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8fbeffd700 0 client.16611 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:42:30.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 >> 172.21.15.150:0/1204026959 conn(0x55c7faec2a60 msgr2=0x55c7faf5afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 shutdown_connections 2024-05-10T04:42:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1204026959 wait complete. 2024-05-10T04:42:30.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.824+0000 7f8ff5f8fb80 20 client.16611 trim_cache size 0 max 16384 2024-05-10T04:42:30.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:30 smithi161 ceph-mon[46912]: pgmap v359: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 861 KiB/s wr, 14 op/s 2024-05-10T04:42:30.843 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:30 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1788212852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:42:30.849 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.837+0000 7f8ff5f8fb80 1 Processor -- start 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.837+0000 7f8ff5f8fb80 1 -- start start 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf61980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf57180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61ec0 0x55c7faf715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7faf576f0 con 0x55c7faf61570 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7faf60f50 con 0x55c7faf61ec0 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7faf75e90 con 0x55c7faf606a0 2024-05-10T04:42:30.850 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf57180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61ec0 0x55c7faf715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf57180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37868/0 (socket says 172.21.15.150:37868) 2024-05-10T04:42:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1140519426 learned_addr learned my addr 172.21.15.150:0/1140519426 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf61980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.851 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1140519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 msgr2=0x55c7faf61980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf61980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1140519426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61ec0 msgr2=0x55c7faf715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61ec0 0x55c7faf715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 -- 172.21.15.150:0/1140519426 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7fae46560 con 0x55c7faf61570 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.838+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf57180 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4096b70 tx=0x7f8fd4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46463702be737ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.839+0000 7f8fc67fc700 1 -- 172.21.15.150:0/1140519426 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd409a100 con 0x55c7faf61570 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.839+0000 7f8fc67fc700 1 -- 172.21.15.150:0/1140519426 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd40a2580 con 0x55c7faf61570 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.839+0000 7f8fc67fc700 1 -- 172.21.15.150:0/1140519426 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd409a360 con 0x55c7faf61570 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.839+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1140519426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 msgr2=0x55c7faf57180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.852 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf57180 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4096b70 tx=0x7f8fd4009f00 comp rx=0 tx=0).stop 2024-05-10T04:42:30.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1140519426 shutdown_connections 2024-05-10T04:42:30.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf61ec0 0x55c7faf715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf606a0 0x55c7faf61980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/1140519426 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61570 0x55c7faf57180 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1140519426 >> 172.21.15.150:0/1140519426 conn(0x55c7faec2a60 msgr2=0x55c7faf68d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.853 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1140519426 shutdown_connections 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/1140519426 wait complete. 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 -- start start 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 Processor -- start 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.840+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61570 0x55c7faf6ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8ff5f8fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7fafca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7faf6f3d0 con 0x55c7faf61ec0 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7faf6f540 con 0x55c7faf606a0 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8ff5f8fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fafcabc0 con 0x55c7faf61570 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8ff5f8fb80 10 client.0 _ll_register_callbacks cb 0x55c7fae25b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7fafca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.854 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7fafca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37878/0 (socket says 172.21.15.150:37878) 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 -- 172.21.15.150:0/87543921 learned_addr learned my addr 172.21.15.150:0/87543921 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdeffd700 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61570 0x55c7faf6ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 -- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61570 msgr2=0x55c7faf6ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61570 0x55c7faf6ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 -- 172.21.15.150:0/87543921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 msgr2=0x55c7faf6e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf6e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 -- 172.21.15.150:0/87543921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd40b1040 con 0x55c7faf61ec0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fdffff700 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7fafca600 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8002230 tx=0x7f8fd801c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469f5890f4631bf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fbeffd700 1 -- 172.21.15.150:0/87543921 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd802a030 con 0x55c7faf61ec0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8fbeffd700 1 -- 172.21.15.150:0/87543921 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f8fd802b070 con 0x55c7faf61ec0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.841+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fd409df30 con 0x55c7faf61ec0 2024-05-10T04:42:30.855 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8fbeffd700 1 -- 172.21.15.150:0/87543921 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd800e070 con 0x55c7faf61ec0 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 10 client.16617 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c7fad05cd0 con 0x55c7faf61ec0 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8fbeffd700 1 -- 172.21.15.150:0/87543921 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd409df30 con 0x55c7faf61ec0 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8fbeffd700 1 -- 172.21.15.150:0/87543921 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fd8026790 con 0x55c7faf61ec0 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 10 client.16617 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 10 client.16617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 20 client.16617 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7fafcc240 0x55c7fafce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 4 client.16617 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c7fae78ec0 con 0x55c7fafcc240 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.842+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7fafcc240 0x55c7fafce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8fbeffd700 10 client.16617 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:42:30.856 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8fdf7fe700 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7fafcc240 0x55c7fafce660 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=999d5f796f5b2ebc server_cookie=e1327775ebbdf22 in_seq=0 out_seq=0 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8fbeffd700 1 -- 172.21.15.150:0/87543921 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c7fae78ec0 con 0x55c7fafcc240 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8fbeffd700 10 client.16617 handle_command_reply: tid=0 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 client.16617 shutdown 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 msgr2=0x55c7fafca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7fafca600 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8002230 tx=0x7f8fd801c590 comp rx=0 tx=0).stop 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 shutdown_connections 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55c7fafcc240 0x55c7fafce660 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c7faf61ec0 0x55c7fafca600 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7faf606a0 0x55c7faf6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 --2- 172.21.15.150:0/87543921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c7faf61570 0x55c7faf6ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8fbeffd700 0 client.16617 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:42:30.857 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 >> 172.21.15.150:0/87543921 conn(0x55c7faec2a60 msgr2=0x55c7faf68ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 shutdown_connections 2024-05-10T04:42:30.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.843+0000 7f8ff5f8fb80 1 -- 172.21.15.150:0/87543921 wait complete. 2024-05-10T04:42:30.858 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8ff5f8fb80 20 client.16617 trim_cache size 0 max 16384 2024-05-10T04:42:30.858 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:42:30.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8fc80a11c0 msgr2=0x7f8fc80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8fc80a11c0 0x7f8fc80a36d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0069840 tx=0x7f8fd0009260 comp rx=0 tx=0).stop 2024-05-10T04:42:30.860 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 msgr2=0x7f8fe01985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe01985d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400d950 tx=0x7f8fd400dd10 comp rx=0 tx=0).stop 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 shutdown_connections 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8fc80a11c0 0x7f8fc80a36d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fe0103190 0x7f8fe019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe00ffbd0 0x7f8fe0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 --2- 172.21.15.150:0/1686837055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fe0100520 0x7f8fe01985d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 >> 172.21.15.150:0/1686837055 conn(0x7f8fe00fb880 msgr2=0x7f8fe0106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 shutdown_connections 2024-05-10T04:42:30.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:30.844+0000 7f8fe5c4b700 1 -- 172.21.15.150:0/1686837055 wait complete. 2024-05-10T04:42:30.862 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:42:30.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:30 smithi181 ceph-mon[47314]: pgmap v359: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 861 KiB/s wr, 14 op/s 2024-05-10T04:42:30.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:30 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1788212852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:42:31.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 -- 172.21.15.150:0/3016043377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218107950 msgr2=0x7f6218109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 --2- 172.21.15.150:0/3016043377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218107950 0x7f6218109e30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f621000b0a0 tx=0x7f621000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.053 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 -- 172.21.15.150:0/3016043377 shutdown_connections 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 --2- 172.21.15.150:0/3016043377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218107950 0x7f6218109e30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 --2- 172.21.15.150:0/3016043377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621806a430 0x7f6218104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 --2- 172.21.15.150:0/3016043377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6218104de0 0x7f62181072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 -- 172.21.15.150:0/3016043377 >> 172.21.15.150:0/3016043377 conn(0x7f62180fc180 msgr2=0x7f62180fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 -- 172.21.15.150:0/3016043377 shutdown_connections 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 -- 172.21.15.150:0/3016043377 wait complete. 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.051+0000 7f621ee17700 1 Processor -- start 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621ee17700 1 -- start start 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621ee17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621806a430 0x7f6218102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621ee17700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6218104de0 0x7f6218100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621ee17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6218107950 0x7f6218100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621ee17700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6218101330 con 0x7f621806a430 2024-05-10T04:42:31.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621ee17700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62181014a0 con 0x7f6218104de0 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621ee17700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62181a20c0 con 0x7f6218107950 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621806a430 0x7f6218102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621806a430 0x7f6218102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37896/0 (socket says 172.21.15.150:37896) 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 -- 172.21.15.150:0/2407480406 learned_addr learned my addr 172.21.15.150:0/2407480406 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f6217fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6218104de0 0x7f6218100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621d3b4700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6218107950 0x7f6218100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 -- 172.21.15.150:0/2407480406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6218107950 msgr2=0x7f6218100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6218107950 0x7f6218100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 -- 172.21.15.150:0/2407480406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6218104de0 msgr2=0x7f6218100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6218104de0 0x7f6218100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 -- 172.21.15.150:0/2407480406 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210009d00 con 0x7f621806a430 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.052+0000 7f621cbb3700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621806a430 0x7f6218102070 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f620c00d980 tx=0x7f620c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.053+0000 7f6215ffb700 1 -- 172.21.15.150:0/2407480406 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f890 con 0x7f621806a430 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.053+0000 7f6215ffb700 1 -- 172.21.15.150:0/2407480406 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620c010460 con 0x7f621806a430 2024-05-10T04:42:31.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.053+0000 7f6215ffb700 1 -- 172.21.15.150:0/2407480406 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00e610 con 0x7f621806a430 2024-05-10T04:42:31.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.053+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62181a2320 con 0x7f621806a430 2024-05-10T04:42:31.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.053+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62181a27b0 con 0x7f621806a430 2024-05-10T04:42:31.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.055+0000 7f6215ffb700 1 -- 172.21.15.150:0/2407480406 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f620c00e770 con 0x7f621806a430 2024-05-10T04:42:31.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.056+0000 7f6215ffb700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62080a11c0 0x7f62080a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.056+0000 7f6215ffb700 1 -- 172.21.15.150:0/2407480406 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f620c0cc6b0 con 0x7f621806a430 2024-05-10T04:42:31.056 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.056+0000 7f6217fff700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62080a11c0 0x7f62080a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f6217fff700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62080a11c0 0x7f62080a36d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6218101a10 tx=0x7f62040073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 Processor -- start 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 -- start start 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f76a0 0x56324b5f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8570 0x56324b5f8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b51e150 con 0x56324b5f8ec0 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b51bb10 con 0x56324b5f76a0 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f622f15bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b517400 con 0x56324b5f8570 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f621d3b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f621d3b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5ff7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37900/0 (socket says 172.21.15.150:37900) 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.057+0000 7f621d3b4700 1 -- 172.21.15.150:0/4144777176 learned_addr learned my addr 172.21.15.150:0/4144777176 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f6217fff700 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8570 0x56324b5f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621cbb3700 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f76a0 0x56324b5f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621d3b4700 1 -- 172.21.15.150:0/4144777176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8570 msgr2=0x56324b5f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621d3b4700 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8570 0x56324b5f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621d3b4700 1 -- 172.21.15.150:0/4144777176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f76a0 msgr2=0x56324b5f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621d3b4700 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f76a0 0x56324b5f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621d3b4700 1 -- 172.21.15.150:0/4144777176 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324b3983b0 con 0x56324b5f8ec0 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621cbb3700 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f76a0 0x56324b5f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f621d3b4700 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5ff7c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6210015000 tx=0x7f6210009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465e83feb4deb1d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f6202ffd700 1 -- 172.21.15.150:0/4144777176 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000e040 con 0x56324b5f8ec0 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f6202ffd700 1 -- 172.21.15.150:0/4144777176 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6210009b80 con 0x56324b5f8ec0 2024-05-10T04:42:31.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.058+0000 7f6202ffd700 1 -- 172.21.15.150:0/4144777176 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100054f0 con 0x56324b5f8ec0 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 -- 172.21.15.150:0/4144777176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 msgr2=0x56324b5ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5ff7c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6210015000 tx=0x7f6210009850 comp rx=0 tx=0).stop 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 -- 172.21.15.150:0/4144777176 shutdown_connections 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5ff7c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f76a0 0x56324b5f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 --2- 172.21.15.150:0/4144777176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8570 0x56324b5f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 -- 172.21.15.150:0/4144777176 >> 172.21.15.150:0/4144777176 conn(0x56324b559a60 msgr2=0x56324b5f1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 -- 172.21.15.150:0/4144777176 shutdown_connections 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 -- 172.21.15.150:0/4144777176 wait complete. 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.059+0000 7f622f15bb80 1 -- start start 2024-05-10T04:42:31.065 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 1 Processor -- start 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b601640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b601b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b608ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b4b3560 con 0x56324b5f8ec0 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b4b3200 con 0x56324b5f8570 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b39a6a0 con 0x56324b5f76a0 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f622f15bb80 10 client.0 _ll_register_callbacks cb 0x56324b4bcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f621cbb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b601640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f621cbb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b601640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:52830/0 (socket says 172.21.15.150:52830) 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f621cbb3700 1 -- 172.21.15.150:0/1021512139 learned_addr learned my addr 172.21.15.150:0/1021512139 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.060+0000 7f6217fff700 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b601b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f621cbb3700 1 -- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 msgr2=0x56324b601b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.066 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f621cbb3700 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b601b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f621cbb3700 1 -- 172.21.15.150:0/1021512139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 msgr2=0x56324b608ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f621cbb3700 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b608ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f621cbb3700 1 -- 172.21.15.150:0/1021512139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621810aa50 con 0x56324b5f76a0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f621cbb3700 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b601640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f620c09a670 tx=0x7f620c09a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d488956353f669b server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f61f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f61f7fff700 1 -- 172.21.15.150:0/1021512139 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0cc3c0 con 0x56324b5f76a0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f61f7fff700 1 -- 172.21.15.150:0/1021512139 <== mon.1 v2:172.21.15.181:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f620c0b90c0 con 0x56324b5f76a0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56324b5c6780 con 0x56324b5f76a0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f61f7fff700 1 -- 172.21.15.150:0/1021512139 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620c0cc520 con 0x56324b5f76a0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f622f15bb80 10 client.26146 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56324b344b60 con 0x56324b5f76a0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.061+0000 7f61f7fff700 1 -- 172.21.15.150:0/1021512139 <== mon.1 v2:172.21.15.181:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56324b5c6780 con 0x56324b5f76a0 2024-05-10T04:42:31.067 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f61f7fff700 1 -- 172.21.15.150:0/1021512139 <== mon.1 v2:172.21.15.181:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f620c099d20 con 0x56324b5f76a0 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f622f15bb80 10 client.26146 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f622f15bb80 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f622f15bb80 20 client.26146 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f622f15bb80 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b60d220 0x56324b60f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f622f15bb80 4 client.26146 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56324b50fec0 con 0x56324b60d220 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f6217fff700 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b60d220 0x56324b60f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f61f7fff700 10 client.26146 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.062+0000 7f6217fff700 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b60d220 0x56324b60f670 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11584b54b212575e server_cookie=30ac6e67ce57d74a in_seq=0 out_seq=0 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f61f7fff700 1 -- 172.21.15.150:0/1021512139 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56324b50fec0 con 0x56324b60d220 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f61f7fff700 10 client.26146 handle_command_reply: tid=0 2024-05-10T04:42:31.068 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 client.26146 shutdown 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 msgr2=0x56324b601640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b601640 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f620c09a670 tx=0x7f620c09a130 comp rx=0 tx=0).stop 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 shutdown_connections 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b60d220 0x56324b60f670 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b608ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b601640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f61f7fff700 0 client.26146 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 --2- 172.21.15.150:0/1021512139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b601b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 >> 172.21.15.150:0/1021512139 conn(0x56324b559a60 msgr2=0x56324b5f1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 shutdown_connections 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.063+0000 7f622f15bb80 1 -- 172.21.15.150:0/1021512139 wait complete. 2024-05-10T04:42:31.069 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.064+0000 7f622f15bb80 20 client.26146 trim_cache size 0 max 16384 2024-05-10T04:42:31.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 Processor -- start 2024-05-10T04:42:31.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 -- start start 2024-05-10T04:42:31.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b6090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b608aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b5f37f0 con 0x56324b5f8ec0 2024-05-10T04:42:31.084 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b60f8f0 con 0x56324b5f8570 2024-05-10T04:42:31.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f622f15bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b5f3010 con 0x56324b5f76a0 2024-05-10T04:42:31.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f621d3b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f621d3b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37918/0 (socket says 172.21.15.150:37918) 2024-05-10T04:42:31.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.077+0000 7f621d3b4700 1 -- 172.21.15.150:0/756025319 learned_addr learned my addr 172.21.15.150:0/756025319 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:31.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f621cbb3700 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b6090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f6217fff700 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b608aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.085 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f621d3b4700 1 -- 172.21.15.150:0/756025319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 msgr2=0x56324b6090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f621d3b4700 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b6090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f621d3b4700 1 -- 172.21.15.150:0/756025319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 msgr2=0x56324b608aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f621d3b4700 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b608aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f621d3b4700 1 -- 172.21.15.150:0/756025319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324b4dd560 con 0x56324b5f8ec0 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f621d3b4700 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5f2ad0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6210000f80 tx=0x7f62100099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baafef61fe74bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f62037fe700 1 -- 172.21.15.150:0/756025319 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000e040 con 0x56324b5f8ec0 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f62037fe700 1 -- 172.21.15.150:0/756025319 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62100092d0 con 0x56324b5f8ec0 2024-05-10T04:42:31.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.078+0000 7f62037fe700 1 -- 172.21.15.150:0/756025319 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100057f0 con 0x56324b5f8ec0 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 -- 172.21.15.150:0/756025319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 msgr2=0x56324b5f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5f2ad0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6210000f80 tx=0x7f62100099e0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 -- 172.21.15.150:0/756025319 shutdown_connections 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f8ec0 0x56324b5f2ad0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f76a0 0x56324b6090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 --2- 172.21.15.150:0/756025319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b608aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 -- 172.21.15.150:0/756025319 >> 172.21.15.150:0/756025319 conn(0x56324b559a60 msgr2=0x56324b5f1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 -- 172.21.15.150:0/756025319 shutdown_connections 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 -- 172.21.15.150:0/756025319 wait complete. 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.079+0000 7f622f15bb80 1 -- start start 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 1 Processor -- start 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f76a0 0x56324b606030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b606570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8ec0 0x56324b6074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b6079e0 con 0x56324b5f76a0 2024-05-10T04:42:31.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b607b50 con 0x56324b5f8570 2024-05-10T04:42:31.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324b607d10 con 0x56324b5f8ec0 2024-05-10T04:42:31.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f622f15bb80 10 client.0 _ll_register_callbacks cb 0x56324b4bcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:31.091 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f621cbb3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f76a0 0x56324b606030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f621cbb3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f76a0 0x56324b606030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37926/0 (socket says 172.21.15.150:37926) 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f621cbb3700 1 -- 172.21.15.150:0/2565264910 learned_addr learned my addr 172.21.15.150:0/2565264910 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.080+0000 7f621d3b4700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8ec0 0x56324b6074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f6217fff700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b606570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f621cbb3700 1 -- 172.21.15.150:0/2565264910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8ec0 msgr2=0x56324b6074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f621cbb3700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8ec0 0x56324b6074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f621cbb3700 1 -- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 msgr2=0x56324b606570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f621cbb3700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b606570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f621cbb3700 1 -- 172.21.15.150:0/2565264910 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56324b3983b0 con 0x56324b5f76a0 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f621d3b4700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8ec0 0x56324b6074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f6217fff700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b606570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f621cbb3700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f76a0 0x56324b606030 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f620c098cd0 tx=0x7f620c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18aaf71a9e184cb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f61f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:31.092 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f61f7fff700 1 -- 172.21.15.150:0/2565264910 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0b6070 con 0x56324b5f76a0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f61f7fff700 1 -- 172.21.15.150:0/2565264910 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f620c0cc3c0 con 0x56324b5f76a0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56324b470270 con 0x56324b5f76a0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f61f7fff700 1 -- 172.21.15.150:0/2565264910 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620c096a20 con 0x56324b5f76a0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f622f15bb80 10 client.16632 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.081+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56324b39ccd0 con 0x56324b5f76a0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f61f7fff700 1 -- 172.21.15.150:0/2565264910 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56324b470270 con 0x56324b5f76a0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f61f7fff700 1 -- 172.21.15.150:0/2565264910 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f620c0965c0 con 0x56324b5f76a0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f622f15bb80 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f622f15bb80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f622f15bb80 20 client.16632 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f622f15bb80 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b6638b0 0x56324b665cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f622f15bb80 4 client.16632 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56324b365cf0 con 0x56324b6638b0 2024-05-10T04:42:31.093 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f6217fff700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b6638b0 0x56324b665cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f61f7fff700 10 client.16632 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.082+0000 7f6217fff700 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b6638b0 0x56324b665cd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d53132fdcd77862 server_cookie=e2e4e0f4b54cf2e in_seq=0 out_seq=0 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f61f7fff700 1 -- 172.21.15.150:0/2565264910 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56324b365cf0 con 0x56324b6638b0 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f61f7fff700 10 client.16632 handle_command_reply: tid=0 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 client.16632 shutdown 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f76a0 msgr2=0x56324b606030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f76a0 0x56324b606030 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f620c098cd0 tx=0x7f620c0106b0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 shutdown_connections 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x56324b6638b0 0x56324b665cd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56324b5f8ec0 0x56324b6074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56324b5f76a0 0x56324b606030 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.094 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f61f7fff700 0 client.16632 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:31.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 --2- 172.21.15.150:0/2565264910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56324b5f8570 0x56324b606570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 >> 172.21.15.150:0/2565264910 conn(0x56324b559a60 msgr2=0x56324b5fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:31.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 shutdown_connections 2024-05-10T04:42:31.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 1 -- 172.21.15.150:0/2565264910 wait complete. 2024-05-10T04:42:31.095 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.083+0000 7f622f15bb80 20 client.16632 trim_cache size 0 max 16384 2024-05-10T04:42:31.095 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:42:31.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62080a11c0 msgr2=0x7f62080a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.096 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62080a11c0 0x7f62080a36d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6218101a10 tx=0x7f62040073d0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621806a430 msgr2=0x7f6218102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621806a430 0x7f6218102070 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f620c00d980 tx=0x7f620c00dc90 comp rx=0 tx=0).stop 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 shutdown_connections 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f62080a11c0 0x7f62080a36d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6218107950 0x7f6218100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f621806a430 0x7f6218102070 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 --2- 172.21.15.150:0/2407480406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6218104de0 0x7f6218100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 >> 172.21.15.150:0/2407480406 conn(0x7f62180fc180 msgr2=0x7f6218103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 shutdown_connections 2024-05-10T04:42:31.097 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:31.084+0000 7f621ee17700 1 -- 172.21.15.150:0/2407480406 wait complete. 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:setting up random write file 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:done setting up random write file 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:starting fsync run 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0096s 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0064s 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0063s 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.459 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0069s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0063s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0063s 2024-05-10T04:42:31.460 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0060s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0061s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0052s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.461 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0063s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0055s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0060s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0061s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.462 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0061s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0068s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0058s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0057s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0062s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0055s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0060s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0066s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0059s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:write time: 0.0003s fsync time: 0.0056s 2024-05-10T04:42:31.463 INFO:tasks.workunit.client.0.smithi150.stdout:run done 61 fsyncs total, killing random writer 2024-05-10T04:42:31.464 INFO:tasks.workunit.client.0.smithi150.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-10T04:42:31.464 INFO:tasks.workunit.client.0.smithi150.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-10T04:42:31.464 INFO:tasks.workunit.client.0.smithi150.stderr:+ whereis lsof 2024-05-10T04:42:31.467 INFO:tasks.workunit.client.0.smithi150.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-05-10T04:42:31.467 INFO:tasks.workunit.client.0.smithi150.stderr:+ lsof 2024-05-10T04:42:31.491 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/35ebb9cbecc3b9873f6300bf79c1c9438bff141fdd4ebe7df09f00e5034a54cb/merged 2024-05-10T04:42:31.491 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/bfb1ce1beb03b362c5e8eccbb0e33bf748f7af21333ee93139cb620322ad1459/merged 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/42d299ee9da8cfaf688c6287a3e5b1280b9620c399daa590bef80d77539ef8f4/merged 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/37903e9d825711d8f2fba91d1141458d0895ae7290dba737b8033c09dba01eb4/merged 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fcff9d7a100d60306069606572ec4d4242ff8b40c09ae376f715800acc1d93c3/merged 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/77f14ad13309a4429feeafbdb8f1e2e12e12f01e2d996033e489f9bf469c2b96/merged 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/459982c004f6fa8fbb513b2482cd6f639400697589bfe77785af6e02fd66b68b/merged 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/090b59f85072f03e19d230945860894eca8ba73e1c4c7f7288b301e82871d3da/merged 2024-05-10T04:42:31.492 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.493 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/5780d22fe672c88c19b4ce65b3ee14db28b73da9e339a2f0b839861d5d7c6063/merged 2024-05-10T04:42:31.493 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.493 INFO:tasks.workunit.client.0.smithi150.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/8faad8cabc75fc55312147e626ac67643f5d2596f59fb38e185655daceb253c6/merged 2024-05-10T04:42:31.493 INFO:tasks.workunit.client.0.smithi150.stderr: Output information may be incomplete. 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-05-10T04:42:31.554 INFO:tasks.workunit.client.0.smithi150.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-05-10T04:42:31.555 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 7 root cwd unknown /proc/7/cwd (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 7 root rtd unknown /proc/7/root (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 7 root txt unknown /proc/7/exe (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 7 root NOFD /proc/7/fd (opendir: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-05-10T04:42:31.556 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_sched 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_sched 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_sched 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:rcu_sched 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:migration 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:migration 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:migration 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:migration 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-05-10T04:42:31.557 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/0 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/0 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/0 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/0 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/1 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/1 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-05-10T04:42:31.558 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/1 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/1 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:migration 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:migration 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:migration 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:migration 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-05-10T04:42:31.559 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/2 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/2 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/2 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/2 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:migration 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:migration 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-05-10T04:42:31.560 INFO:tasks.workunit.client.0.smithi150.stdout:migration 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:migration 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/3 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/3 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-05-10T04:42:31.561 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/3 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/3 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:migration 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:migration 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:migration 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:migration 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-05-10T04:42:31.562 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/4 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/4 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/4 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/4 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:migration 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-05-10T04:42:31.563 INFO:tasks.workunit.client.0.smithi150.stdout:migration 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:migration 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:migration 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/5 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/5 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/5 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/5 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-05-10T04:42:31.564 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:migration 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:migration 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:migration 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:migration 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-05-10T04:42:31.565 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/6 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/6 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/6 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/6 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:migration 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:migration 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:migration 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:migration 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-05-10T04:42:31.566 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/7 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/7 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/7 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:cpuhp/7 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:watchdog/ 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-05-10T04:42:31.567 INFO:tasks.workunit.client.0.smithi150.stdout:migration 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:migration 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:migration 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:migration 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:ksoftirqd 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-05-10T04:42:31.568 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kdevtmpfs 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kdevtmpfs 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kdevtmpfs 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kdevtmpfs 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:netns 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:netns 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:netns 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:netns 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kauditd 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kauditd 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-05-10T04:42:31.569 INFO:tasks.workunit.client.0.smithi150.stdout:kauditd 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:kauditd 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:khungtask 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:khungtask 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:khungtask 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:khungtask 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:oom_reape 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:oom_reape 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:oom_reape 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:oom_reape 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:writeback 74 root cwd unknown /proc/74/cwd (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:writeback 74 root rtd unknown /proc/74/root (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:writeback 74 root txt unknown /proc/74/exe (readlink: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:writeback 74 root NOFD /proc/74/fd (opendir: Permission denied) 2024-05-10T04:42:31.570 INFO:tasks.workunit.client.0.smithi150.stdout:kcompactd 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:kcompactd 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:kcompactd 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:kcompactd 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:ksmd 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:ksmd 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:ksmd 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:ksmd 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:khugepage 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:khugepage 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:khugepage 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:khugepage 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:crypto 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:crypto 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-05-10T04:42:31.571 INFO:tasks.workunit.client.0.smithi150.stdout:crypto 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:crypto 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kintegrit 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kintegrit 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kintegrit 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kintegrit 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kblockd 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kblockd 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kblockd 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kblockd 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:blkcg_pun 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:blkcg_pun 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:blkcg_pun 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:blkcg_pun 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-05-10T04:42:31.572 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:tpm_dev_w 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:tpm_dev_w 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:tpm_dev_w 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:tpm_dev_w 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:md 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:md 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:md 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:md 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:edac-poll 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-05-10T04:42:31.573 INFO:tasks.workunit.client.0.smithi150.stdout:edac-poll 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:edac-poll 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:edac-poll 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:watchdogd 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:watchdogd 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:watchdogd 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:watchdogd 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:kswapd0 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:kswapd0 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-05-10T04:42:31.574 INFO:tasks.workunit.client.0.smithi150.stdout:kswapd0 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kswapd0 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kthrotld 170 root cwd unknown /proc/170/cwd (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kthrotld 170 root rtd unknown /proc/170/root (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kthrotld 170 root txt unknown /proc/170/exe (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kthrotld 170 root NOFD /proc/170/fd (opendir: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:acpi_ther 171 root cwd unknown /proc/171/cwd (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:acpi_ther 171 root rtd unknown /proc/171/root (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:acpi_ther 171 root txt unknown /proc/171/exe (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:acpi_ther 171 root NOFD /proc/171/fd (opendir: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kmpath_rd 172 root cwd unknown /proc/172/cwd (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kmpath_rd 172 root rtd unknown /proc/172/root (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kmpath_rd 172 root txt unknown /proc/172/exe (readlink: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kmpath_rd 172 root NOFD /proc/172/fd (opendir: Permission denied) 2024-05-10T04:42:31.575 INFO:tasks.workunit.client.0.smithi150.stdout:kaluad 173 root cwd unknown /proc/173/cwd (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kaluad 173 root rtd unknown /proc/173/root (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kaluad 173 root txt unknown /proc/173/exe (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kaluad 173 root NOFD /proc/173/fd (opendir: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 175 root cwd unknown /proc/175/cwd (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 175 root rtd unknown /proc/175/root (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 175 root txt unknown /proc/175/exe (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 175 root NOFD /proc/175/fd (opendir: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:ipv6_addr 176 root cwd unknown /proc/176/cwd (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:ipv6_addr 176 root rtd unknown /proc/176/root (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:ipv6_addr 176 root txt unknown /proc/176/exe (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:ipv6_addr 176 root NOFD /proc/176/fd (opendir: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kstrp 177 root cwd unknown /proc/177/cwd (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kstrp 177 root rtd unknown /proc/177/root (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kstrp 177 root txt unknown /proc/177/exe (readlink: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:kstrp 177 root NOFD /proc/177/fd (opendir: Permission denied) 2024-05-10T04:42:31.576 INFO:tasks.workunit.client.0.smithi150.stdout:zswap-shr 217 root cwd unknown /proc/217/cwd (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:zswap-shr 217 root rtd unknown /proc/217/root (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:zswap-shr 217 root txt unknown /proc/217/exe (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:zswap-shr 217 root NOFD /proc/217/fd (opendir: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 231 root cwd unknown /proc/231/cwd (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 231 root rtd unknown /proc/231/root (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 231 root txt unknown /proc/231/exe (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 231 root NOFD /proc/231/fd (opendir: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 263 root cwd unknown /proc/263/cwd (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 263 root rtd unknown /proc/263/root (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 263 root txt unknown /proc/263/exe (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 263 root NOFD /proc/263/fd (opendir: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 264 root cwd unknown /proc/264/cwd (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 264 root rtd unknown /proc/264/root (readlink: Permission denied) 2024-05-10T04:42:31.577 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 264 root txt unknown /proc/264/exe (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 264 root NOFD /proc/264/fd (opendir: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 292 root cwd unknown /proc/292/cwd (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 292 root rtd unknown /proc/292/root (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 292 root txt unknown /proc/292/exe (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 292 root NOFD /proc/292/fd (opendir: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 368 root cwd unknown /proc/368/cwd (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 368 root rtd unknown /proc/368/root (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 368 root txt unknown /proc/368/exe (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 368 root NOFD /proc/368/fd (opendir: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 373 root cwd unknown /proc/373/cwd (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 373 root rtd unknown /proc/373/root (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 373 root txt unknown /proc/373/exe (readlink: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 373 root NOFD /proc/373/fd (opendir: Permission denied) 2024-05-10T04:42:31.578 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-wq 412 root cwd unknown /proc/412/cwd (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-wq 412 root rtd unknown /proc/412/root (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-wq 412 root txt unknown /proc/412/exe (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-wq 412 root NOFD /proc/412/fd (opendir: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-rese 413 root cwd unknown /proc/413/cwd (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-rese 413 root rtd unknown /proc/413/root (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-rese 413 root txt unknown /proc/413/exe (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-rese 413 root NOFD /proc/413/fd (opendir: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-dele 420 root cwd unknown /proc/420/cwd (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-dele 420 root rtd unknown /proc/420/root (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-dele 420 root txt unknown /proc/420/exe (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:nvme-dele 420 root NOFD /proc/420/fd (opendir: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:ata_sff 423 root cwd unknown /proc/423/cwd (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:ata_sff 423 root rtd unknown /proc/423/root (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:ata_sff 423 root txt unknown /proc/423/exe (readlink: Permission denied) 2024-05-10T04:42:31.579 INFO:tasks.workunit.client.0.smithi150.stdout:ata_sff 423 root NOFD /proc/423/fd (opendir: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:ixgbe 424 root cwd unknown /proc/424/cwd (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:ixgbe 424 root rtd unknown /proc/424/root (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:ixgbe 424 root txt unknown /proc/424/exe (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:ixgbe 424 root NOFD /proc/424/fd (opendir: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_0 428 root cwd unknown /proc/428/cwd (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_0 428 root rtd unknown /proc/428/root (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_0 428 root txt unknown /proc/428/exe (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_0 428 root NOFD /proc/428/fd (opendir: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 429 root cwd unknown /proc/429/cwd (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 429 root rtd unknown /proc/429/root (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 429 root txt unknown /proc/429/exe (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 429 root NOFD /proc/429/fd (opendir: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_1 430 root cwd unknown /proc/430/cwd (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_1 430 root rtd unknown /proc/430/root (readlink: Permission denied) 2024-05-10T04:42:31.580 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_1 430 root txt unknown /proc/430/exe (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_1 430 root NOFD /proc/430/fd (opendir: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 431 root cwd unknown /proc/431/cwd (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 431 root rtd unknown /proc/431/root (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 431 root txt unknown /proc/431/exe (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 431 root NOFD /proc/431/fd (opendir: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_2 432 root cwd unknown /proc/432/cwd (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_2 432 root rtd unknown /proc/432/root (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_2 432 root txt unknown /proc/432/exe (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_2 432 root NOFD /proc/432/fd (opendir: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 433 root cwd unknown /proc/433/cwd (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 433 root rtd unknown /proc/433/root (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 433 root txt unknown /proc/433/exe (readlink: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 433 root NOFD /proc/433/fd (opendir: Permission denied) 2024-05-10T04:42:31.581 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_3 434 root cwd unknown /proc/434/cwd (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_3 434 root rtd unknown /proc/434/root (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_3 434 root txt unknown /proc/434/exe (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_3 434 root NOFD /proc/434/fd (opendir: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 435 root cwd unknown /proc/435/cwd (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 435 root rtd unknown /proc/435/root (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 435 root txt unknown /proc/435/exe (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 435 root NOFD /proc/435/fd (opendir: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_4 436 root cwd unknown /proc/436/cwd (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_4 436 root rtd unknown /proc/436/root (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_4 436 root txt unknown /proc/436/exe (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_4 436 root NOFD /proc/436/fd (opendir: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 437 root cwd unknown /proc/437/cwd (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 437 root rtd unknown /proc/437/root (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 437 root txt unknown /proc/437/exe (readlink: Permission denied) 2024-05-10T04:42:31.582 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 437 root NOFD /proc/437/fd (opendir: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_5 438 root cwd unknown /proc/438/cwd (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_5 438 root rtd unknown /proc/438/root (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_5 438 root txt unknown /proc/438/exe (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_5 438 root NOFD /proc/438/fd (opendir: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 439 root cwd unknown /proc/439/cwd (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 439 root rtd unknown /proc/439/root (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 439 root txt unknown /proc/439/exe (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 439 root NOFD /proc/439/fd (opendir: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_6 440 root cwd unknown /proc/440/cwd (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_6 440 root rtd unknown /proc/440/root (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_6 440 root txt unknown /proc/440/exe (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_6 440 root NOFD /proc/440/fd (opendir: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 441 root cwd unknown /proc/441/cwd (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 441 root rtd unknown /proc/441/root (readlink: Permission denied) 2024-05-10T04:42:31.583 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 441 root txt unknown /proc/441/exe (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 441 root NOFD /proc/441/fd (opendir: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_7 442 root cwd unknown /proc/442/cwd (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_7 442 root rtd unknown /proc/442/root (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_7 442 root txt unknown /proc/442/exe (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_7 442 root NOFD /proc/442/fd (opendir: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 443 root cwd unknown /proc/443/cwd (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 443 root rtd unknown /proc/443/root (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 443 root txt unknown /proc/443/exe (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 443 root NOFD /proc/443/fd (opendir: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_8 444 root cwd unknown /proc/444/cwd (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_8 444 root rtd unknown /proc/444/root (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_8 444 root txt unknown /proc/444/exe (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_8 444 root NOFD /proc/444/fd (opendir: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-05-10T04:42:31.584 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_9 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_9 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_9 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_eh_9 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:scsi_tmf_ 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-05-10T04:42:31.585 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:jbd2/sda1 479 root cwd unknown /proc/479/cwd (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:jbd2/sda1 479 root rtd unknown /proc/479/root (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:jbd2/sda1 479 root txt unknown /proc/479/exe (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:jbd2/sda1 479 root NOFD /proc/479/fd (opendir: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:ext4-rsv- 480 root cwd unknown /proc/480/cwd (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:ext4-rsv- 480 root rtd unknown /proc/480/root (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:ext4-rsv- 480 root txt unknown /proc/480/exe (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:ext4-rsv- 480 root NOFD /proc/480/fd (opendir: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-j 581 root cwd unknown /proc/581/cwd (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-j 581 root rtd unknown /proc/581/root (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-j 581 root txt unknown /proc/581/exe (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-j 581 root NOFD /proc/581/fd (opendir: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:ext4lazyi 589 root cwd unknown /proc/589/cwd (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:ext4lazyi 589 root rtd unknown /proc/589/root (readlink: Permission denied) 2024-05-10T04:42:31.586 INFO:tasks.workunit.client.0.smithi150.stdout:ext4lazyi 589 root txt unknown /proc/589/exe (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:ext4lazyi 589 root NOFD /proc/589/fd (opendir: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-u 634 root cwd unknown /proc/634/cwd (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-u 634 root rtd unknown /proc/634/root (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-u 634 root txt unknown /proc/634/exe (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-u 634 root NOFD /proc/634/fd (opendir: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpcbind 656 rpc cwd unknown /proc/656/cwd (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpcbind 656 rpc rtd unknown /proc/656/root (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpcbind 656 rpc txt unknown /proc/656/exe (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpcbind 656 rpc NOFD /proc/656/fd (opendir: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpciod 681 root cwd unknown /proc/681/cwd (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpciod 681 root rtd unknown /proc/681/root (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpciod 681 root txt unknown /proc/681/exe (readlink: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:rpciod 681 root NOFD /proc/681/fd (opendir: Permission denied) 2024-05-10T04:42:31.587 INFO:tasks.workunit.client.0.smithi150.stdout:xprtiod 682 root cwd unknown /proc/682/cwd (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:xprtiod 682 root rtd unknown /proc/682/root (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:xprtiod 682 root txt unknown /proc/682/exe (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:xprtiod 682 root NOFD /proc/682/fd (opendir: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:ipmi-msgh 688 root cwd unknown /proc/688/cwd (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:ipmi-msgh 688 root rtd unknown /proc/688/root (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:ipmi-msgh 688 root txt unknown /proc/688/exe (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:ipmi-msgh 688 root NOFD /proc/688/fd (opendir: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 root cwd unknown /proc/693/cwd (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 root rtd unknown /proc/693/root (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 root txt unknown /proc/693/exe (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 root NOFD /proc/693/fd (opendir: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 694 auditd root cwd unknown /proc/693/task/694/cwd (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 694 auditd root rtd unknown /proc/693/task/694/root (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 694 auditd root txt unknown /proc/693/task/694/exe (readlink: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 694 auditd root NOFD /proc/693/task/694/fd (opendir: Permission denied) 2024-05-10T04:42:31.588 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 696 auditd root cwd unknown /proc/693/task/696/cwd (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 696 auditd root rtd unknown /proc/693/task/696/root (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 696 auditd root txt unknown /proc/693/task/696/exe (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:auditd 693 696 auditd root NOFD /proc/693/task/696/fd (opendir: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:sedispatc 695 root cwd unknown /proc/695/cwd (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:sedispatc 695 root rtd unknown /proc/695/root (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:sedispatc 695 root txt unknown /proc/695/exe (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:sedispatc 695 root NOFD /proc/695/fd (opendir: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:kipmi0 710 root cwd unknown /proc/710/cwd (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:kipmi0 710 root rtd unknown /proc/710/root (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:kipmi0 710 root txt unknown /proc/710/exe (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:kipmi0 710 root NOFD /proc/710/fd (opendir: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 802 avahi cwd unknown /proc/802/cwd (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 802 avahi rtd unknown /proc/802/root (readlink: Permission denied) 2024-05-10T04:42:31.589 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 802 avahi txt unknown /proc/802/exe (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 802 avahi NOFD /proc/802/fd (opendir: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:lsmd 803 libstoragemgmt cwd unknown /proc/803/cwd (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:lsmd 803 libstoragemgmt rtd unknown /proc/803/root (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:lsmd 803 libstoragemgmt txt unknown /proc/803/exe (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:lsmd 803 libstoragemgmt NOFD /proc/803/fd (opendir: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:sssd 804 root cwd unknown /proc/804/cwd (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:sssd 804 root rtd unknown /proc/804/root (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:sssd 804 root txt unknown /proc/804/exe (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:sssd 804 root NOFD /proc/804/fd (opendir: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:mcelog 805 root cwd unknown /proc/805/cwd (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:mcelog 805 root rtd unknown /proc/805/root (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:mcelog 805 root txt unknown /proc/805/exe (readlink: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:mcelog 805 root NOFD /proc/805/fd (opendir: Permission denied) 2024-05-10T04:42:31.590 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 dbus cwd unknown /proc/806/cwd (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 dbus rtd unknown /proc/806/root (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 dbus txt unknown /proc/806/exe (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 dbus NOFD /proc/806/fd (opendir: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 855 dbus-daem dbus cwd unknown /proc/806/task/855/cwd (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 855 dbus-daem dbus rtd unknown /proc/806/task/855/root (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 855 dbus-daem dbus txt unknown /proc/806/task/855/exe (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:dbus-daem 806 855 dbus-daem dbus NOFD /proc/806/task/855/fd (opendir: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 root cwd unknown /proc/807/cwd (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 root rtd unknown /proc/807/root (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 root txt unknown /proc/807/exe (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 root NOFD /proc/807/fd (opendir: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 854 gmain root cwd unknown /proc/807/task/854/cwd (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 854 gmain root rtd unknown /proc/807/task/854/root (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 854 gmain root txt unknown /proc/807/task/854/exe (readlink: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 854 gmain root NOFD /proc/807/task/854/fd (opendir: Permission denied) 2024-05-10T04:42:31.591 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 856 gdbus root cwd unknown /proc/807/task/856/cwd (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 856 gdbus root rtd unknown /proc/807/task/856/root (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 856 gdbus root txt unknown /proc/807/task/856/exe (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:NetworkMa 807 856 gdbus root NOFD /proc/807/task/856/fd (opendir: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:smartd 809 root cwd unknown /proc/809/cwd (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:smartd 809 root rtd unknown /proc/809/root (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:smartd 809 root txt unknown /proc/809/exe (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:smartd 809 root NOFD /proc/809/fd (opendir: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 root cwd unknown /proc/810/cwd (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 root rtd unknown /proc/810/root (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 root txt unknown /proc/810/exe (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 root NOFD /proc/810/fd (opendir: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 824 gmain root cwd unknown /proc/810/task/824/cwd (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 824 gmain root rtd unknown /proc/810/task/824/root (readlink: Permission denied) 2024-05-10T04:42:31.592 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 824 gmain root txt unknown /proc/810/task/824/exe (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:irqbalanc 810 824 gmain root NOFD /proc/810/task/824/fd (opendir: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 polkitd cwd unknown /proc/811/cwd (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 polkitd rtd unknown /proc/811/root (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 polkitd txt unknown /proc/811/exe (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 polkitd NOFD /proc/811/fd (opendir: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 880 gmain polkitd cwd unknown /proc/811/task/880/cwd (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 880 gmain polkitd rtd unknown /proc/811/task/880/root (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 880 gmain polkitd txt unknown /proc/811/task/880/exe (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 880 gmain polkitd NOFD /proc/811/task/880/fd (opendir: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 881 gdbus polkitd cwd unknown /proc/811/task/881/cwd (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 881 gdbus polkitd rtd unknown /proc/811/task/881/root (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 881 gdbus polkitd txt unknown /proc/811/task/881/exe (readlink: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 881 gdbus polkitd NOFD /proc/811/task/881/fd (opendir: Permission denied) 2024-05-10T04:42:31.593 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 882 JS\x20Hel polkitd cwd unknown /proc/811/task/882/cwd (readlink: Permission denied) 2024-05-10T04:42:31.594 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 882 JS\x20Hel polkitd rtd unknown /proc/811/task/882/root (readlink: Permission denied) 2024-05-10T04:42:31.594 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 882 JS\x20Hel polkitd txt unknown /proc/811/task/882/exe (readlink: Permission denied) 2024-05-10T04:42:31.595 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 882 JS\x20Hel polkitd NOFD /proc/811/task/882/fd (opendir: Permission denied) 2024-05-10T04:42:31.595 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 883 JS\x20Hel polkitd cwd unknown /proc/811/task/883/cwd (readlink: Permission denied) 2024-05-10T04:42:31.595 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 883 JS\x20Hel polkitd rtd unknown /proc/811/task/883/root (readlink: Permission denied) 2024-05-10T04:42:31.595 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 883 JS\x20Hel polkitd txt unknown /proc/811/task/883/exe (readlink: Permission denied) 2024-05-10T04:42:31.595 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 883 JS\x20Hel polkitd NOFD /proc/811/task/883/fd (opendir: Permission denied) 2024-05-10T04:42:31.595 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 884 JS\x20Hel polkitd cwd unknown /proc/811/task/884/cwd (readlink: Permission denied) 2024-05-10T04:42:31.595 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 884 JS\x20Hel polkitd rtd unknown /proc/811/task/884/root (readlink: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 884 JS\x20Hel polkitd txt unknown /proc/811/task/884/exe (readlink: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 884 JS\x20Hel polkitd NOFD /proc/811/task/884/fd (opendir: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 885 JS\x20Hel polkitd cwd unknown /proc/811/task/885/cwd (readlink: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 885 JS\x20Hel polkitd rtd unknown /proc/811/task/885/root (readlink: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 885 JS\x20Hel polkitd txt unknown /proc/811/task/885/exe (readlink: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 885 JS\x20Hel polkitd NOFD /proc/811/task/885/fd (opendir: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 886 JS\x20Hel polkitd cwd unknown /proc/811/task/886/cwd (readlink: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 886 JS\x20Hel polkitd rtd unknown /proc/811/task/886/root (readlink: Permission denied) 2024-05-10T04:42:31.596 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 886 JS\x20Hel polkitd txt unknown /proc/811/task/886/exe (readlink: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 886 JS\x20Hel polkitd NOFD /proc/811/task/886/fd (opendir: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 887 JS\x20Hel polkitd cwd unknown /proc/811/task/887/cwd (readlink: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 887 JS\x20Hel polkitd rtd unknown /proc/811/task/887/root (readlink: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 887 JS\x20Hel polkitd txt unknown /proc/811/task/887/exe (readlink: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 887 JS\x20Hel polkitd NOFD /proc/811/task/887/fd (opendir: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 888 JS\x20Hel polkitd cwd unknown /proc/811/task/888/cwd (readlink: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 888 JS\x20Hel polkitd rtd unknown /proc/811/task/888/root (readlink: Permission denied) 2024-05-10T04:42:31.597 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 888 JS\x20Hel polkitd txt unknown /proc/811/task/888/exe (readlink: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 888 JS\x20Hel polkitd NOFD /proc/811/task/888/fd (opendir: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 889 JS\x20Hel polkitd cwd unknown /proc/811/task/889/cwd (readlink: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 889 JS\x20Hel polkitd rtd unknown /proc/811/task/889/root (readlink: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 889 JS\x20Hel polkitd txt unknown /proc/811/task/889/exe (readlink: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 889 JS\x20Hel polkitd NOFD /proc/811/task/889/fd (opendir: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 896 polkitd polkitd cwd unknown /proc/811/task/896/cwd (readlink: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 896 polkitd polkitd rtd unknown /proc/811/task/896/root (readlink: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 896 polkitd polkitd txt unknown /proc/811/task/896/exe (readlink: Permission denied) 2024-05-10T04:42:31.598 INFO:tasks.workunit.client.0.smithi150.stdout:polkitd 811 896 polkitd polkitd NOFD /proc/811/task/896/fd (opendir: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 859 avahi cwd unknown /proc/859/cwd (readlink: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 859 avahi rtd unknown /proc/859/root (readlink: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 859 avahi txt unknown /proc/859/exe (readlink: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:avahi-dae 859 avahi NOFD /proc/859/fd (opendir: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 root cwd unknown /proc/862/cwd (readlink: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 root rtd unknown /proc/862/root (readlink: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 root txt unknown /proc/862/exe (readlink: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 root NOFD /proc/862/fd (opendir: Permission denied) 2024-05-10T04:42:31.599 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1088 tuned root cwd unknown /proc/862/task/1088/cwd (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1088 tuned root rtd unknown /proc/862/task/1088/root (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1088 tuned root txt unknown /proc/862/task/1088/exe (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1088 tuned root NOFD /proc/862/task/1088/fd (opendir: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1093 tuned root cwd unknown /proc/862/task/1093/cwd (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1093 tuned root rtd unknown /proc/862/task/1093/root (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1093 tuned root txt unknown /proc/862/task/1093/exe (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1093 tuned root NOFD /proc/862/task/1093/fd (opendir: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1110 tuned root cwd unknown /proc/862/task/1110/cwd (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1110 tuned root rtd unknown /proc/862/task/1110/root (readlink: Permission denied) 2024-05-10T04:42:31.600 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1110 tuned root txt unknown /proc/862/task/1110/exe (readlink: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1110 tuned root NOFD /proc/862/task/1110/fd (opendir: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1111 tuned root cwd unknown /proc/862/task/1111/cwd (readlink: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1111 tuned root rtd unknown /proc/862/task/1111/root (readlink: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1111 tuned root txt unknown /proc/862/task/1111/exe (readlink: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:tuned 862 1111 tuned root NOFD /proc/862/task/1111/fd (opendir: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 863 root cwd unknown /proc/863/cwd (readlink: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 863 root rtd unknown /proc/863/root (readlink: Permission denied) 2024-05-10T04:42:31.601 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 863 root txt unknown /proc/863/exe (readlink: Permission denied) 2024-05-10T04:42:31.602 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 863 root NOFD /proc/863/fd (opendir: Permission denied) 2024-05-10T04:42:31.602 INFO:tasks.workunit.client.0.smithi150.stdout:rhsmcertd 867 root cwd unknown /proc/867/cwd (readlink: Permission denied) 2024-05-10T04:42:31.602 INFO:tasks.workunit.client.0.smithi150.stdout:rhsmcertd 867 root rtd unknown /proc/867/root (readlink: Permission denied) 2024-05-10T04:42:31.602 INFO:tasks.workunit.client.0.smithi150.stdout:rhsmcertd 867 root txt unknown /proc/867/exe (readlink: Permission denied) 2024-05-10T04:42:31.602 INFO:tasks.workunit.client.0.smithi150.stdout:rhsmcertd 867 root NOFD /proc/867/fd (opendir: Permission denied) 2024-05-10T04:42:31.602 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 root cwd unknown /proc/872/cwd (readlink: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 root rtd unknown /proc/872/root (readlink: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 root txt unknown /proc/872/exe (readlink: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 root NOFD /proc/872/fd (opendir: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 874 gssproxy root cwd unknown /proc/872/task/874/cwd (readlink: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 874 gssproxy root rtd unknown /proc/872/task/874/root (readlink: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 874 gssproxy root txt unknown /proc/872/task/874/exe (readlink: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 874 gssproxy root NOFD /proc/872/task/874/fd (opendir: Permission denied) 2024-05-10T04:42:31.603 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 875 gssproxy root cwd unknown /proc/872/task/875/cwd (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 875 gssproxy root rtd unknown /proc/872/task/875/root (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 875 gssproxy root txt unknown /proc/872/task/875/exe (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 875 gssproxy root NOFD /proc/872/task/875/fd (opendir: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 876 gssproxy root cwd unknown /proc/872/task/876/cwd (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 876 gssproxy root rtd unknown /proc/872/task/876/root (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 876 gssproxy root txt unknown /proc/872/task/876/exe (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 876 gssproxy root NOFD /proc/872/task/876/fd (opendir: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 877 gssproxy root cwd unknown /proc/872/task/877/cwd (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 877 gssproxy root rtd unknown /proc/872/task/877/root (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 877 gssproxy root txt unknown /proc/872/task/877/exe (readlink: Permission denied) 2024-05-10T04:42:31.604 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 877 gssproxy root NOFD /proc/872/task/877/fd (opendir: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 878 gssproxy root cwd unknown /proc/872/task/878/cwd (readlink: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 878 gssproxy root rtd unknown /proc/872/task/878/root (readlink: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 878 gssproxy root txt unknown /proc/872/task/878/exe (readlink: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:gssproxy 872 878 gssproxy root NOFD /proc/872/task/878/fd (opendir: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_be 899 root cwd unknown /proc/899/cwd (readlink: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_be 899 root rtd unknown /proc/899/root (readlink: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_be 899 root txt unknown /proc/899/exe (readlink: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_be 899 root NOFD /proc/899/fd (opendir: Permission denied) 2024-05-10T04:42:31.605 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_nss 915 root cwd unknown /proc/915/cwd (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_nss 915 root rtd unknown /proc/915/root (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_nss 915 root txt unknown /proc/915/exe (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:sssd_nss 915 root NOFD /proc/915/fd (opendir: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-l 930 root cwd unknown /proc/930/cwd (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-l 930 root rtd unknown /proc/930/root (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-l 930 root txt unknown /proc/930/exe (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:systemd-l 930 root NOFD /proc/930/fd (opendir: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:cupsd 931 root cwd unknown /proc/931/cwd (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:cupsd 931 root rtd unknown /proc/931/root (readlink: Permission denied) 2024-05-10T04:42:31.606 INFO:tasks.workunit.client.0.smithi150.stdout:cupsd 931 root txt unknown /proc/931/exe (readlink: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:cupsd 931 root NOFD /proc/931/fd (opendir: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:crond 935 root cwd unknown /proc/935/cwd (readlink: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:crond 935 root rtd unknown /proc/935/root (readlink: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:crond 935 root txt unknown /proc/935/exe (readlink: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:crond 935 root NOFD /proc/935/fd (opendir: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:atd 936 root cwd unknown /proc/936/cwd (readlink: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:atd 936 root rtd unknown /proc/936/root (readlink: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:atd 936 root txt unknown /proc/936/exe (readlink: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:atd 936 root NOFD /proc/936/fd (opendir: Permission denied) 2024-05-10T04:42:31.607 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 1115 root cwd unknown /proc/1115/cwd (readlink: Permission denied) 2024-05-10T04:42:31.608 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 1115 root rtd unknown /proc/1115/root (readlink: Permission denied) 2024-05-10T04:42:31.608 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 1115 root txt unknown /proc/1115/exe (readlink: Permission denied) 2024-05-10T04:42:31.608 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 1115 root NOFD /proc/1115/fd (opendir: Permission denied) 2024-05-10T04:42:31.608 INFO:tasks.workunit.client.0.smithi150.stdout:nrpe 1134 nrpe cwd unknown /proc/1134/cwd (readlink: Permission denied) 2024-05-10T04:42:31.608 INFO:tasks.workunit.client.0.smithi150.stdout:nrpe 1134 nrpe rtd unknown /proc/1134/root (readlink: Permission denied) 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:nrpe 1134 nrpe txt unknown /proc/1134/exe (readlink: Permission denied) 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:nrpe 1134 nrpe NOFD /proc/1134/fd (opendir: Permission denied) 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu cwd DIR 8,1 4096 2 / 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu rtd DIR 8,1 4096 2 / 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu txt REG 8,1 1601312 8305 /usr/lib/systemd/systemd 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 1598808 5171 /usr/lib64/libm-2.28.so 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 636728 7430 /usr/lib64/libudev.so.1.6.11 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 735192 5316 /usr/lib64/libsepol.so.1 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 1760264 5414 /usr/lib64/libunistring.so.2.1.0 2024-05-10T04:42:31.609 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 144392 5392 /usr/lib64/libgpg-error.so.0.24.2 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 66912 5574 /usr/lib64/libjson-c.so.4.0.0 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 371224 7497 /usr/lib64/libdevmapper.so.1.02 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 26192 5481 /usr/lib64/libattr.so.1.1.2448 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu DEL REG 8,1 7507 /usr/lib64/libcrypto.so.1.1.1k 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu DEL REG 8,1 7509 /usr/lib64/libssl.so.1.1.1k 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 99600 5318 /usr/lib64/libz.so.1.2.11 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 24928 5576 /usr/lib64/libcap-ng.so.0.0.0 2024-05-10T04:42:31.610 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu DEL REG 8,1 5382 /usr/lib64/libuuid.so.1.3.0 2024-05-10T04:42:31.611 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-05-10T04:42:31.611 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 543160 4872 /usr/lib64/libpcre2-8.so.0.7.1 2024-05-10T04:42:31.611 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu DEL REG 8,1 7499 /usr/lib64/libblkid.so.1.1.0 2024-05-10T04:42:31.611 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 119760 5781 /usr/lib64/liblz4.so.1.8.3 2024-05-10T04:42:31.611 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 192016 5359 /usr/lib64/liblzma.so.5.2.4 2024-05-10T04:42:31.611 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 162224 5584 /usr/lib64/libidn2.so.0.3.6 2024-05-10T04:42:31.611 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 1187360 5616 /usr/lib64/libgcrypt.so.20.2.5 2024-05-10T04:42:31.612 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 526360 7535 /usr/lib64/libcryptsetup.so.12.6.0 2024-05-10T04:42:31.612 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 54816 5483 /usr/lib64/libacl.so.1.1.2253 2024-05-10T04:42:31.612 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 33752 5371 /usr/lib64/libcap.so.2.48 2024-05-10T04:42:31.612 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-05-10T04:42:31.612 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-05-10T04:42:31.612 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 99664 6703 /usr/lib64/libgcc_s-8-20210514.so.1 2024-05-10T04:42:31.612 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 100200 7532 /usr/lib64/libkmod.so.2.3.3 2024-05-10T04:42:31.613 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 135792 5578 /usr/lib64/libaudit.so.1.0.0 2024-05-10T04:42:31.613 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 66504 7674 /usr/lib64/libpam.so.0.84.2 2024-05-10T04:42:31.613 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu DEL REG 8,1 7306 /usr/lib64/libmount.so.1.1.0 2024-05-10T04:42:31.613 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 168536 4876 /usr/lib64/libselinux.so.1 2024-05-10T04:42:31.613 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 128784 6087 /usr/lib64/libseccomp.so.2.5.2 2024-05-10T04:42:31.613 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 42744 5185 /usr/lib64/librt-2.28.so 2024-05-10T04:42:31.613 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 2740672 8300 /usr/lib/systemd/libsystemd-shared-239.so 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 0r CHR 1,3 0t0 16631 /dev/null 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 1u unix 0x0000000000000000 0t0 35231 type=STREAM 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 2u unix 0x0000000000000000 0t0 35231 type=STREAM 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 3u unix 0x0000000000000000 0t0 35260 type=DGRAM 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 4u a_inode 0,14 0 16625 [eventpoll] 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 5u a_inode 0,14 0 16625 [signalfd] 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 6r DIR 0,26 0 813 /sys/fs/cgroup/systemd/user.slice/user-1000.slice/user@1000.service 2024-05-10T04:42:31.614 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 7u a_inode 0,14 0 16625 [timerfd] 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 8u netlink 0t0 35267 KOBJECT_UEVENT 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 9u a_inode 0,14 0 16625 [eventpoll] 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 10r a_inode 0,14 0 16625 inotify 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 11u unix 0x0000000000000000 0t0 30537 /run/user/1000/bus type=STREAM 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 12r REG 0,5 0 35272 /proc/5701/mountinfo 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 13r a_inode 0,14 0 16625 inotify 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 14r REG 0,5 0 4026532074 /proc/swaps 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 15u unix 0x0000000000000000 0t0 35273 /run/user/1000/systemd/notify type=DGRAM 2024-05-10T04:42:31.615 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 16u unix 0x0000000000000000 0t0 35275 type=DGRAM 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 17u unix 0x0000000000000000 0t0 35276 type=DGRAM 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 18u unix 0x0000000000000000 0t0 35277 /run/user/1000/systemd/private type=STREAM 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 19u unix 0x0000000000000000 0t0 35279 type=STREAM 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 20u a_inode 0,14 0 16625 [timerfd] 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 5701 ubuntu 26u netlink 0t0 30540 SELINUX 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 5712 ubuntu cwd unknown /proc/5712/cwd (readlink: Permission denied) 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 5712 ubuntu rtd unknown /proc/5712/root (readlink: Permission denied) 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 5712 ubuntu txt unknown /proc/5712/exe (readlink: Permission denied) 2024-05-10T04:42:31.616 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 5712 ubuntu NOFD /proc/5712/fd (opendir: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5892 root cwd unknown /proc/5892/cwd (readlink: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5892 root rtd unknown /proc/5892/root (readlink: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5892 root txt unknown /proc/5892/exe (readlink: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5892 root NOFD /proc/5892/fd (opendir: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5893 root cwd unknown /proc/5893/cwd (readlink: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5893 root rtd unknown /proc/5893/root (readlink: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5893 root txt unknown /proc/5893/exe (readlink: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:agetty 5893 root NOFD /proc/5893/fd (opendir: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 6300 root cwd unknown /proc/6300/cwd (readlink: Permission denied) 2024-05-10T04:42:31.617 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 6300 root rtd unknown /proc/6300/root (readlink: Permission denied) 2024-05-10T04:42:31.618 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 6300 root txt unknown /proc/6300/exe (readlink: Permission denied) 2024-05-10T04:42:31.618 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 6300 root NOFD /proc/6300/fd (opendir: Permission denied) 2024-05-10T04:42:31.618 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 root cwd unknown /proc/6611/cwd (readlink: Permission denied) 2024-05-10T04:42:31.618 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 root rtd unknown /proc/6611/root (readlink: Permission denied) 2024-05-10T04:42:31.618 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 root txt unknown /proc/6611/exe (readlink: Permission denied) 2024-05-10T04:42:31.618 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 root NOFD /proc/6611/fd (opendir: Permission denied) 2024-05-10T04:42:31.618 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6613 in:imjour root cwd unknown /proc/6611/task/6613/cwd (readlink: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6613 in:imjour root rtd unknown /proc/6611/task/6613/root (readlink: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6613 in:imjour root txt unknown /proc/6611/task/6613/exe (readlink: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6613 in:imjour root NOFD /proc/6611/task/6613/fd (opendir: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6614 rs:main root cwd unknown /proc/6611/task/6614/cwd (readlink: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6614 rs:main root rtd unknown /proc/6611/task/6614/root (readlink: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6614 rs:main root txt unknown /proc/6611/task/6614/exe (readlink: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:rsyslogd 6611 6614 rs:main root NOFD /proc/6611/task/6614/fd (opendir: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:ksmtuned 25958 root cwd unknown /proc/25958/cwd (readlink: Permission denied) 2024-05-10T04:42:31.619 INFO:tasks.workunit.client.0.smithi150.stdout:ksmtuned 25958 root rtd unknown /proc/25958/root (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:ksmtuned 25958 root txt unknown /proc/25958/exe (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:ksmtuned 25958 root NOFD /proc/25958/fd (opendir: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 27560 root cwd unknown /proc/27560/cwd (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 27560 root rtd unknown /proc/27560/root (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 27560 root txt unknown /proc/27560/exe (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 27560 root NOFD /proc/27560/fd (opendir: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 27561 root cwd unknown /proc/27561/cwd (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 27561 root rtd unknown /proc/27561/root (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 27561 root txt unknown /proc/27561/exe (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 27561 root NOFD /proc/27561/fd (opendir: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28249 root cwd unknown /proc/28249/cwd (readlink: Permission denied) 2024-05-10T04:42:31.620 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28249 root rtd unknown /proc/28249/root (readlink: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28249 root txt unknown /proc/28249/exe (readlink: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28249 root NOFD /proc/28249/fd (opendir: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28286 root cwd unknown /proc/28286/cwd (readlink: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28286 root rtd unknown /proc/28286/root (readlink: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28286 root txt unknown /proc/28286/exe (readlink: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28286 root NOFD /proc/28286/fd (opendir: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28323 root cwd unknown /proc/28323/cwd (readlink: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28323 root rtd unknown /proc/28323/root (readlink: Permission denied) 2024-05-10T04:42:31.621 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28323 root txt unknown /proc/28323/exe (readlink: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28323 root NOFD /proc/28323/fd (opendir: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28360 root cwd unknown /proc/28360/cwd (readlink: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28360 root rtd unknown /proc/28360/root (readlink: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28360 root txt unknown /proc/28360/exe (readlink: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28360 root NOFD /proc/28360/fd (opendir: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28397 root cwd unknown /proc/28397/cwd (readlink: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28397 root rtd unknown /proc/28397/root (readlink: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28397 root txt unknown /proc/28397/exe (readlink: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:kdmflush/ 28397 root NOFD /proc/28397/fd (opendir: Permission denied) 2024-05-10T04:42:31.622 INFO:tasks.workunit.client.0.smithi150.stdout:xfsalloc 28610 root cwd unknown /proc/28610/cwd (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfsalloc 28610 root rtd unknown /proc/28610/root (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfsalloc 28610 root txt unknown /proc/28610/exe (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfsalloc 28610 root NOFD /proc/28610/fd (opendir: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfs_mru_c 28611 root cwd unknown /proc/28611/cwd (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfs_mru_c 28611 root rtd unknown /proc/28611/root (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfs_mru_c 28611 root txt unknown /proc/28611/exe (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfs_mru_c 28611 root NOFD /proc/28611/fd (opendir: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-buf/d 28612 root cwd unknown /proc/28612/cwd (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-buf/d 28612 root rtd unknown /proc/28612/root (readlink: Permission denied) 2024-05-10T04:42:31.623 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-buf/d 28612 root txt unknown /proc/28612/exe (readlink: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-buf/d 28612 root NOFD /proc/28612/fd (opendir: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-conv/ 28613 root cwd unknown /proc/28613/cwd (readlink: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-conv/ 28613 root rtd unknown /proc/28613/root (readlink: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-conv/ 28613 root txt unknown /proc/28613/exe (readlink: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-conv/ 28613 root NOFD /proc/28613/fd (opendir: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-cil/d 28614 root cwd unknown /proc/28614/cwd (readlink: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-cil/d 28614 root rtd unknown /proc/28614/root (readlink: Permission denied) 2024-05-10T04:42:31.624 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-cil/d 28614 root txt unknown /proc/28614/exe (readlink: Permission denied) 2024-05-10T04:42:31.625 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-cil/d 28614 root NOFD /proc/28614/fd (opendir: Permission denied) 2024-05-10T04:42:31.625 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-recla 28615 root cwd unknown /proc/28615/cwd (readlink: Permission denied) 2024-05-10T04:42:31.625 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-recla 28615 root rtd unknown /proc/28615/root (readlink: Permission denied) 2024-05-10T04:42:31.625 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-recla 28615 root txt unknown /proc/28615/exe (readlink: Permission denied) 2024-05-10T04:42:31.625 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-recla 28615 root NOFD /proc/28615/fd (opendir: Permission denied) 2024-05-10T04:42:31.625 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-block 28616 root cwd unknown /proc/28616/cwd (readlink: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-block 28616 root rtd unknown /proc/28616/root (readlink: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-block 28616 root txt unknown /proc/28616/exe (readlink: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-block 28616 root NOFD /proc/28616/fd (opendir: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-log/d 28617 root cwd unknown /proc/28617/cwd (readlink: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-log/d 28617 root rtd unknown /proc/28617/root (readlink: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-log/d 28617 root txt unknown /proc/28617/exe (readlink: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfs-log/d 28617 root NOFD /proc/28617/fd (opendir: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfsaild/d 28618 root cwd unknown /proc/28618/cwd (readlink: Permission denied) 2024-05-10T04:42:31.626 INFO:tasks.workunit.client.0.smithi150.stdout:xfsaild/d 28618 root rtd unknown /proc/28618/root (readlink: Permission denied) 2024-05-10T04:42:31.627 INFO:tasks.workunit.client.0.smithi150.stdout:xfsaild/d 28618 root txt unknown /proc/28618/exe (readlink: Permission denied) 2024-05-10T04:42:31.627 INFO:tasks.workunit.client.0.smithi150.stdout:xfsaild/d 28618 root NOFD /proc/28618/fd (opendir: Permission denied) 2024-05-10T04:42:31.627 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32292 root cwd unknown /proc/32292/cwd (readlink: Permission denied) 2024-05-10T04:42:31.627 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32292 root rtd unknown /proc/32292/root (readlink: Permission denied) 2024-05-10T04:42:31.627 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32292 root txt unknown /proc/32292/exe (readlink: Permission denied) 2024-05-10T04:42:31.627 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32292 root NOFD /proc/32292/fd (opendir: Permission denied) 2024-05-10T04:42:31.627 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32296 ubuntu cwd unknown /proc/32296/cwd (readlink: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32296 ubuntu rtd unknown /proc/32296/root (readlink: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32296 ubuntu txt unknown /proc/32296/exe (readlink: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 32296 ubuntu NOFD /proc/32296/fd (opendir: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:chronyd 32366 chrony cwd unknown /proc/32366/cwd (readlink: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:chronyd 32366 chrony rtd unknown /proc/32366/root (readlink: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:chronyd 32366 chrony txt unknown /proc/32366/exe (readlink: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:chronyd 32366 chrony NOFD /proc/32366/fd (opendir: Permission denied) 2024-05-10T04:42:31.628 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-cras 33563 ceph cwd unknown /proc/33563/cwd (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-cras 33563 ceph rtd unknown /proc/33563/root (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-cras 33563 ceph txt unknown /proc/33563/exe (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-cras 33563 ceph NOFD /proc/33563/fd (opendir: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 40616 root cwd unknown /proc/40616/cwd (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 40616 root rtd unknown /proc/40616/root (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 40616 root txt unknown /proc/40616/exe (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 40616 root NOFD /proc/40616/fd (opendir: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 40988 root cwd unknown /proc/40988/cwd (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 40988 root rtd unknown /proc/40988/root (readlink: Permission denied) 2024-05-10T04:42:31.629 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 40988 root txt unknown /proc/40988/exe (readlink: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 40988 root NOFD /proc/40988/fd (opendir: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 40989 root cwd unknown /proc/40989/cwd (readlink: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 40989 root rtd unknown /proc/40989/root (readlink: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 40989 root txt unknown /proc/40989/exe (readlink: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 40989 root NOFD /proc/40989/fd (opendir: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41072 root cwd unknown /proc/41072/cwd (readlink: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41072 root rtd unknown /proc/41072/root (readlink: Permission denied) 2024-05-10T04:42:31.630 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41072 root txt unknown /proc/41072/exe (readlink: Permission denied) 2024-05-10T04:42:31.631 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41072 root NOFD /proc/41072/fd (opendir: Permission denied) 2024-05-10T04:42:31.631 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41092 root cwd unknown /proc/41092/cwd (readlink: Permission denied) 2024-05-10T04:42:31.631 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41092 root rtd unknown /proc/41092/root (readlink: Permission denied) 2024-05-10T04:42:31.631 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41092 root txt unknown /proc/41092/exe (readlink: Permission denied) 2024-05-10T04:42:31.631 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41092 root NOFD /proc/41092/fd (opendir: Permission denied) 2024-05-10T04:42:31.631 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41093 root cwd unknown /proc/41093/cwd (readlink: Permission denied) 2024-05-10T04:42:31.631 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41093 root rtd unknown /proc/41093/root (readlink: Permission denied) 2024-05-10T04:42:31.632 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41093 root txt unknown /proc/41093/exe (readlink: Permission denied) 2024-05-10T04:42:31.632 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 41093 root NOFD /proc/41093/fd (opendir: Permission denied) 2024-05-10T04:42:31.632 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41113 root cwd unknown /proc/41113/cwd (readlink: Permission denied) 2024-05-10T04:42:31.632 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41113 root rtd unknown /proc/41113/root (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41113 root txt unknown /proc/41113/exe (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 41113 root NOFD /proc/41113/fd (opendir: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41252 root cwd unknown /proc/41252/cwd (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41252 root rtd unknown /proc/41252/root (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41252 root txt unknown /proc/41252/exe (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41252 root NOFD /proc/41252/fd (opendir: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41509 root cwd unknown /proc/41509/cwd (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41509 root rtd unknown /proc/41509/root (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41509 root txt unknown /proc/41509/exe (readlink: Permission denied) 2024-05-10T04:42:31.633 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 41509 root NOFD /proc/41509/fd (opendir: Permission denied) 2024-05-10T04:42:31.634 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 41669 root cwd unknown /proc/41669/cwd (readlink: Permission denied) 2024-05-10T04:42:31.634 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 41669 root rtd unknown /proc/41669/root (readlink: Permission denied) 2024-05-10T04:42:31.634 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 41669 root txt unknown /proc/41669/exe (readlink: Permission denied) 2024-05-10T04:42:31.634 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 41669 root NOFD /proc/41669/fd (opendir: Permission denied) 2024-05-10T04:42:31.634 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 41846 root cwd unknown /proc/41846/cwd (readlink: Permission denied) 2024-05-10T04:42:31.634 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 41846 root rtd unknown /proc/41846/root (readlink: Permission denied) 2024-05-10T04:42:31.634 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 41846 root txt unknown /proc/41846/exe (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 41846 root NOFD /proc/41846/fd (opendir: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42381 root cwd unknown /proc/42381/cwd (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42381 root rtd unknown /proc/42381/root (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42381 root txt unknown /proc/42381/exe (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42381 root NOFD /proc/42381/fd (opendir: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42392 root cwd unknown /proc/42392/cwd (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42392 root rtd unknown /proc/42392/root (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42392 root txt unknown /proc/42392/exe (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42392 root NOFD /proc/42392/fd (opendir: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 ceph cwd unknown /proc/42404/cwd (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 ceph rtd unknown /proc/42404/root (readlink: Permission denied) 2024-05-10T04:42:31.635 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 ceph txt unknown /proc/42404/exe (readlink: Permission denied) 2024-05-10T04:42:31.636 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 ceph NOFD /proc/42404/fd (opendir: Permission denied) 2024-05-10T04:42:31.636 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42410 log ceph cwd unknown /proc/42404/task/42410/cwd (readlink: Permission denied) 2024-05-10T04:42:31.636 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42410 log ceph rtd unknown /proc/42404/task/42410/root (readlink: Permission denied) 2024-05-10T04:42:31.636 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42410 log ceph txt unknown /proc/42404/task/42410/exe (readlink: Permission denied) 2024-05-10T04:42:31.636 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42410 log ceph NOFD /proc/42404/task/42410/fd (opendir: Permission denied) 2024-05-10T04:42:31.638 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42413 service ceph cwd unknown /proc/42404/task/42413/cwd (readlink: Permission denied) 2024-05-10T04:42:31.638 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42413 service ceph rtd unknown /proc/42404/task/42413/root (readlink: Permission denied) 2024-05-10T04:42:31.638 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42413 service ceph txt unknown /proc/42404/task/42413/exe (readlink: Permission denied) 2024-05-10T04:42:31.638 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42413 service ceph NOFD /proc/42404/task/42413/fd (opendir: Permission denied) 2024-05-10T04:42:31.638 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42414 admin_soc ceph cwd unknown /proc/42404/task/42414/cwd (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42414 admin_soc ceph rtd unknown /proc/42404/task/42414/root (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42414 admin_soc ceph txt unknown /proc/42404/task/42414/exe (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42414 admin_soc ceph NOFD /proc/42404/task/42414/fd (opendir: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42416 signal_ha ceph cwd unknown /proc/42404/task/42416/cwd (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42416 signal_ha ceph rtd unknown /proc/42404/task/42416/root (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42416 signal_ha ceph txt unknown /proc/42404/task/42416/exe (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42416 signal_ha ceph NOFD /proc/42404/task/42416/fd (opendir: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42417 rocksdb:l ceph cwd unknown /proc/42404/task/42417/cwd (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42417 rocksdb:l ceph rtd unknown /proc/42404/task/42417/root (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42417 rocksdb:l ceph txt unknown /proc/42404/task/42417/exe (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42417 rocksdb:l ceph NOFD /proc/42404/task/42417/fd (opendir: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42418 rocksdb:h ceph cwd unknown /proc/42404/task/42418/cwd (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42418 rocksdb:h ceph rtd unknown /proc/42404/task/42418/root (readlink: Permission denied) 2024-05-10T04:42:31.639 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42418 rocksdb:h ceph txt unknown /proc/42404/task/42418/exe (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42418 rocksdb:h ceph NOFD /proc/42404/task/42418/fd (opendir: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42438 ceph-mon ceph cwd unknown /proc/42404/task/42438/cwd (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42438 ceph-mon ceph rtd unknown /proc/42404/task/42438/root (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42438 ceph-mon ceph txt unknown /proc/42404/task/42438/exe (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42438 ceph-mon ceph NOFD /proc/42404/task/42438/fd (opendir: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42439 fn_monsto ceph cwd unknown /proc/42404/task/42439/cwd (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42439 fn_monsto ceph rtd unknown /proc/42404/task/42439/root (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42439 fn_monsto ceph txt unknown /proc/42404/task/42439/exe (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42439 fn_monsto ceph NOFD /proc/42404/task/42439/fd (opendir: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42440 msgr-work ceph cwd unknown /proc/42404/task/42440/cwd (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42440 msgr-work ceph rtd unknown /proc/42404/task/42440/root (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42440 msgr-work ceph txt unknown /proc/42404/task/42440/exe (readlink: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42440 msgr-work ceph NOFD /proc/42404/task/42440/fd (opendir: Permission denied) 2024-05-10T04:42:31.640 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42441 msgr-work ceph cwd unknown /proc/42404/task/42441/cwd (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42441 msgr-work ceph rtd unknown /proc/42404/task/42441/root (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42441 msgr-work ceph txt unknown /proc/42404/task/42441/exe (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42441 msgr-work ceph NOFD /proc/42404/task/42441/fd (opendir: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42442 msgr-work ceph cwd unknown /proc/42404/task/42442/cwd (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42442 msgr-work ceph rtd unknown /proc/42404/task/42442/root (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42442 msgr-work ceph txt unknown /proc/42404/task/42442/exe (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42442 msgr-work ceph NOFD /proc/42404/task/42442/fd (opendir: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42443 OpHistory ceph cwd unknown /proc/42404/task/42443/cwd (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42443 OpHistory ceph rtd unknown /proc/42404/task/42443/root (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42443 OpHistory ceph txt unknown /proc/42404/task/42443/exe (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42443 OpHistory ceph NOFD /proc/42404/task/42443/fd (opendir: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42445 fin ceph cwd unknown /proc/42404/task/42445/cwd (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42445 fin ceph rtd unknown /proc/42404/task/42445/root (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42445 fin ceph txt unknown /proc/42404/task/42445/exe (readlink: Permission denied) 2024-05-10T04:42:31.641 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42445 fin ceph NOFD /proc/42404/task/42445/fd (opendir: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42446 safe_time ceph cwd unknown /proc/42404/task/42446/cwd (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42446 safe_time ceph rtd unknown /proc/42404/task/42446/root (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42446 safe_time ceph txt unknown /proc/42404/task/42446/exe (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42446 safe_time ceph NOFD /proc/42404/task/42446/fd (opendir: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42447 cpu_tp ceph cwd unknown /proc/42404/task/42447/cwd (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42447 cpu_tp ceph rtd unknown /proc/42404/task/42447/root (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42447 cpu_tp ceph txt unknown /proc/42404/task/42447/exe (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42447 cpu_tp ceph NOFD /proc/42404/task/42447/fd (opendir: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42448 cpu_tp ceph cwd unknown /proc/42404/task/42448/cwd (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42448 cpu_tp ceph rtd unknown /proc/42404/task/42448/root (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42448 cpu_tp ceph txt unknown /proc/42404/task/42448/exe (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42448 cpu_tp ceph NOFD /proc/42404/task/42448/fd (opendir: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42449 cpu_tp ceph cwd unknown /proc/42404/task/42449/cwd (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42449 cpu_tp ceph rtd unknown /proc/42404/task/42449/root (readlink: Permission denied) 2024-05-10T04:42:31.642 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42449 cpu_tp ceph txt unknown /proc/42404/task/42449/exe (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42449 cpu_tp ceph NOFD /proc/42404/task/42449/fd (opendir: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42450 cpu_tp ceph cwd unknown /proc/42404/task/42450/cwd (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42450 cpu_tp ceph rtd unknown /proc/42404/task/42450/root (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42450 cpu_tp ceph txt unknown /proc/42404/task/42450/exe (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42450 cpu_tp ceph NOFD /proc/42404/task/42450/fd (opendir: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42451 ms_dispat ceph cwd unknown /proc/42404/task/42451/cwd (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42451 ms_dispat ceph rtd unknown /proc/42404/task/42451/root (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42451 ms_dispat ceph txt unknown /proc/42404/task/42451/exe (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42451 ms_dispat ceph NOFD /proc/42404/task/42451/fd (opendir: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42452 ms_local ceph cwd unknown /proc/42404/task/42452/cwd (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42452 ms_local ceph rtd unknown /proc/42404/task/42452/root (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42452 ms_local ceph txt unknown /proc/42404/task/42452/exe (readlink: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42452 ms_local ceph NOFD /proc/42404/task/42452/fd (opendir: Permission denied) 2024-05-10T04:42:31.643 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42453 safe_time ceph cwd unknown /proc/42404/task/42453/cwd (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42453 safe_time ceph rtd unknown /proc/42404/task/42453/root (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42453 safe_time ceph txt unknown /proc/42404/task/42453/exe (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42453 safe_time ceph NOFD /proc/42404/task/42453/fd (opendir: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42454 ms_dispat ceph cwd unknown /proc/42404/task/42454/cwd (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42454 ms_dispat ceph rtd unknown /proc/42404/task/42454/root (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42454 ms_dispat ceph txt unknown /proc/42404/task/42454/exe (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42454 ms_dispat ceph NOFD /proc/42404/task/42454/fd (opendir: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42455 ms_local ceph cwd unknown /proc/42404/task/42455/cwd (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42455 ms_local ceph rtd unknown /proc/42404/task/42455/root (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42455 ms_local ceph txt unknown /proc/42404/task/42455/exe (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 42455 ms_local ceph NOFD /proc/42404/task/42455/fd (opendir: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 92427 rstore_co ceph cwd unknown /proc/42404/task/92427/cwd (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 92427 rstore_co ceph rtd unknown /proc/42404/task/92427/root (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 92427 rstore_co ceph txt unknown /proc/42404/task/92427/exe (readlink: Permission denied) 2024-05-10T04:42:31.644 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mon 42404 92427 rstore_co ceph NOFD /proc/42404/task/92427/fd (opendir: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42654 root cwd unknown /proc/42654/cwd (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42654 root rtd unknown /proc/42654/root (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42654 root txt unknown /proc/42654/exe (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 42654 root NOFD /proc/42654/fd (opendir: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42665 root cwd unknown /proc/42665/cwd (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42665 root rtd unknown /proc/42665/root (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42665 root txt unknown /proc/42665/exe (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 42665 root NOFD /proc/42665/fd (opendir: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 ceph cwd unknown /proc/42677/cwd (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 ceph rtd unknown /proc/42677/root (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 ceph txt unknown /proc/42677/exe (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 ceph NOFD /proc/42677/fd (opendir: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45696 log ceph cwd unknown /proc/42677/task/45696/cwd (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45696 log ceph rtd unknown /proc/42677/task/45696/root (readlink: Permission denied) 2024-05-10T04:42:31.645 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45696 log ceph txt unknown /proc/42677/task/45696/exe (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45696 log ceph NOFD /proc/42677/task/45696/fd (opendir: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45698 msgr-work ceph cwd unknown /proc/42677/task/45698/cwd (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45698 msgr-work ceph rtd unknown /proc/42677/task/45698/root (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45698 msgr-work ceph txt unknown /proc/42677/task/45698/exe (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45698 msgr-work ceph NOFD /proc/42677/task/45698/fd (opendir: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45699 msgr-work ceph cwd unknown /proc/42677/task/45699/cwd (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45699 msgr-work ceph rtd unknown /proc/42677/task/45699/root (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45699 msgr-work ceph txt unknown /proc/42677/task/45699/exe (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45699 msgr-work ceph NOFD /proc/42677/task/45699/fd (opendir: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45700 msgr-work ceph cwd unknown /proc/42677/task/45700/cwd (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45700 msgr-work ceph rtd unknown /proc/42677/task/45700/root (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45700 msgr-work ceph txt unknown /proc/42677/task/45700/exe (readlink: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45700 msgr-work ceph NOFD /proc/42677/task/45700/fd (opendir: Permission denied) 2024-05-10T04:42:31.646 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45704 service ceph cwd unknown /proc/42677/task/45704/cwd (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45704 service ceph rtd unknown /proc/42677/task/45704/root (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45704 service ceph txt unknown /proc/42677/task/45704/exe (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45704 service ceph NOFD /proc/42677/task/45704/fd (opendir: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45706 admin_soc ceph cwd unknown /proc/42677/task/45706/cwd (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45706 admin_soc ceph rtd unknown /proc/42677/task/45706/root (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45706 admin_soc ceph txt unknown /proc/42677/task/45706/exe (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45706 admin_soc ceph NOFD /proc/42677/task/45706/fd (opendir: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45707 ceph_time ceph cwd unknown /proc/42677/task/45707/cwd (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45707 ceph_time ceph rtd unknown /proc/42677/task/45707/root (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45707 ceph_time ceph txt unknown /proc/42677/task/45707/exe (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45707 ceph_time ceph NOFD /proc/42677/task/45707/fd (opendir: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45708 fn_anonym ceph cwd unknown /proc/42677/task/45708/cwd (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45708 fn_anonym ceph rtd unknown /proc/42677/task/45708/root (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45708 fn_anonym ceph txt unknown /proc/42677/task/45708/exe (readlink: Permission denied) 2024-05-10T04:42:31.647 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45708 fn_anonym ceph NOFD /proc/42677/task/45708/fd (opendir: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45709 signal_ha ceph cwd unknown /proc/42677/task/45709/cwd (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45709 signal_ha ceph rtd unknown /proc/42677/task/45709/root (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45709 signal_ha ceph txt unknown /proc/42677/task/45709/exe (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45709 signal_ha ceph NOFD /proc/42677/task/45709/fd (opendir: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45710 mgrsb-fin ceph cwd unknown /proc/42677/task/45710/cwd (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45710 mgrsb-fin ceph rtd unknown /proc/42677/task/45710/root (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45710 mgrsb-fin ceph txt unknown /proc/42677/task/45710/exe (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45710 mgrsb-fin ceph NOFD /proc/42677/task/45710/fd (opendir: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45711 ms_dispat ceph cwd unknown /proc/42677/task/45711/cwd (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45711 ms_dispat ceph rtd unknown /proc/42677/task/45711/root (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45711 ms_dispat ceph txt unknown /proc/42677/task/45711/exe (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45711 ms_dispat ceph NOFD /proc/42677/task/45711/fd (opendir: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45712 ms_local ceph cwd unknown /proc/42677/task/45712/cwd (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45712 ms_local ceph rtd unknown /proc/42677/task/45712/root (readlink: Permission denied) 2024-05-10T04:42:31.648 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45712 ms_local ceph txt unknown /proc/42677/task/45712/exe (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45712 ms_local ceph NOFD /proc/42677/task/45712/fd (opendir: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45713 io_contex ceph cwd unknown /proc/42677/task/45713/cwd (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45713 io_contex ceph rtd unknown /proc/42677/task/45713/root (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45713 io_contex ceph txt unknown /proc/42677/task/45713/exe (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45713 io_contex ceph NOFD /proc/42677/task/45713/fd (opendir: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45714 io_contex ceph cwd unknown /proc/42677/task/45714/cwd (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45714 io_contex ceph rtd unknown /proc/42677/task/45714/root (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45714 io_contex ceph txt unknown /proc/42677/task/45714/exe (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45714 io_contex ceph NOFD /proc/42677/task/45714/fd (opendir: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45715 safe_time ceph cwd unknown /proc/42677/task/45715/cwd (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45715 safe_time ceph rtd unknown /proc/42677/task/45715/root (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45715 safe_time ceph txt unknown /proc/42677/task/45715/exe (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45715 safe_time ceph NOFD /proc/42677/task/45715/fd (opendir: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45716 safe_time ceph cwd unknown /proc/42677/task/45716/cwd (readlink: Permission denied) 2024-05-10T04:42:31.649 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45716 safe_time ceph rtd unknown /proc/42677/task/45716/root (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45716 safe_time ceph txt unknown /proc/42677/task/45716/exe (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45716 safe_time ceph NOFD /proc/42677/task/45716/fd (opendir: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45717 safe_time ceph cwd unknown /proc/42677/task/45717/cwd (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45717 safe_time ceph rtd unknown /proc/42677/task/45717/root (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45717 safe_time ceph txt unknown /proc/42677/task/45717/exe (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45717 safe_time ceph NOFD /proc/42677/task/45717/fd (opendir: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45718 fn_anonym ceph cwd unknown /proc/42677/task/45718/cwd (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45718 fn_anonym ceph rtd unknown /proc/42677/task/45718/root (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45718 fn_anonym ceph txt unknown /proc/42677/task/45718/exe (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45718 fn_anonym ceph NOFD /proc/42677/task/45718/fd (opendir: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45719 flusher ceph cwd unknown /proc/42677/task/45719/cwd (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45719 flusher ceph rtd unknown /proc/42677/task/45719/root (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45719 flusher ceph txt unknown /proc/42677/task/45719/exe (readlink: Permission denied) 2024-05-10T04:42:31.650 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45719 flusher ceph NOFD /proc/42677/task/45719/fd (opendir: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45720 safe_time ceph cwd unknown /proc/42677/task/45720/cwd (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45720 safe_time ceph rtd unknown /proc/42677/task/45720/root (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45720 safe_time ceph txt unknown /proc/42677/task/45720/exe (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45720 safe_time ceph NOFD /proc/42677/task/45720/fd (opendir: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45949 mgr-fin ceph cwd unknown /proc/42677/task/45949/cwd (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45949 mgr-fin ceph rtd unknown /proc/42677/task/45949/root (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45949 mgr-fin ceph txt unknown /proc/42677/task/45949/exe (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45949 mgr-fin ceph NOFD /proc/42677/task/45949/fd (opendir: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45950 ms_dispat ceph cwd unknown /proc/42677/task/45950/cwd (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45950 ms_dispat ceph rtd unknown /proc/42677/task/45950/root (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45950 ms_dispat ceph txt unknown /proc/42677/task/45950/exe (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45950 ms_dispat ceph NOFD /proc/42677/task/45950/fd (opendir: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45951 ms_local ceph cwd unknown /proc/42677/task/45951/cwd (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45951 ms_local ceph rtd unknown /proc/42677/task/45951/root (readlink: Permission denied) 2024-05-10T04:42:31.651 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45951 ms_local ceph txt unknown /proc/42677/task/45951/exe (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45951 ms_local ceph NOFD /proc/42677/task/45951/fd (opendir: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45952 safe_time ceph cwd unknown /proc/42677/task/45952/cwd (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45952 safe_time ceph rtd unknown /proc/42677/task/45952/root (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45952 safe_time ceph txt unknown /proc/42677/task/45952/exe (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45952 safe_time ceph NOFD /proc/42677/task/45952/fd (opendir: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45953 cmdfin ceph cwd unknown /proc/42677/task/45953/cwd (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45953 cmdfin ceph rtd unknown /proc/42677/task/45953/root (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45953 cmdfin ceph txt unknown /proc/42677/task/45953/exe (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45953 cmdfin ceph NOFD /proc/42677/task/45953/fd (opendir: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45957 io_contex ceph cwd unknown /proc/42677/task/45957/cwd (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45957 io_contex ceph rtd unknown /proc/42677/task/45957/root (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45957 io_contex ceph txt unknown /proc/42677/task/45957/exe (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45957 io_contex ceph NOFD /proc/42677/task/45957/fd (opendir: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45958 io_contex ceph cwd unknown /proc/42677/task/45958/cwd (readlink: Permission denied) 2024-05-10T04:42:31.652 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45958 io_contex ceph rtd unknown /proc/42677/task/45958/root (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45958 io_contex ceph txt unknown /proc/42677/task/45958/exe (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45958 io_contex ceph NOFD /proc/42677/task/45958/fd (opendir: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45959 ceph_time ceph cwd unknown /proc/42677/task/45959/cwd (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45959 ceph_time ceph rtd unknown /proc/42677/task/45959/root (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45959 ceph_time ceph txt unknown /proc/42677/task/45959/exe (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45959 ceph_time ceph NOFD /proc/42677/task/45959/fd (opendir: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45960 ms_dispat ceph cwd unknown /proc/42677/task/45960/cwd (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45960 ms_dispat ceph rtd unknown /proc/42677/task/45960/root (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45960 ms_dispat ceph txt unknown /proc/42677/task/45960/exe (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45960 ms_dispat ceph NOFD /proc/42677/task/45960/fd (opendir: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45961 ms_local ceph cwd unknown /proc/42677/task/45961/cwd (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45961 ms_local ceph rtd unknown /proc/42677/task/45961/root (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45961 ms_local ceph txt unknown /proc/42677/task/45961/exe (readlink: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45961 ms_local ceph NOFD /proc/42677/task/45961/fd (opendir: Permission denied) 2024-05-10T04:42:31.653 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45962 safe_time ceph cwd unknown /proc/42677/task/45962/cwd (readlink: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45962 safe_time ceph rtd unknown /proc/42677/task/45962/root (readlink: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45962 safe_time ceph txt unknown /proc/42677/task/45962/exe (readlink: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45962 safe_time ceph NOFD /proc/42677/task/45962/fd (opendir: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45963 safe_time ceph cwd unknown /proc/42677/task/45963/cwd (readlink: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45963 safe_time ceph rtd unknown /proc/42677/task/45963/root (readlink: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45963 safe_time ceph txt unknown /proc/42677/task/45963/exe (readlink: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45963 safe_time ceph NOFD /proc/42677/task/45963/fd (opendir: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45964 balancer ceph cwd unknown /proc/42677/task/45964/cwd (readlink: Permission denied) 2024-05-10T04:42:31.654 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45964 balancer ceph rtd unknown /proc/42677/task/45964/root (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45964 balancer ceph txt unknown /proc/42677/task/45964/exe (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45964 balancer ceph NOFD /proc/42677/task/45964/fd (opendir: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45965 m-fin-bal ceph cwd unknown /proc/42677/task/45965/cwd (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45965 m-fin-bal ceph rtd unknown /proc/42677/task/45965/root (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45965 m-fin-bal ceph txt unknown /proc/42677/task/45965/exe (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45965 m-fin-bal ceph NOFD /proc/42677/task/45965/fd (opendir: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45966 mgr-fin ceph cwd unknown /proc/42677/task/45966/cwd (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45966 mgr-fin ceph rtd unknown /proc/42677/task/45966/root (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45966 mgr-fin ceph txt unknown /proc/42677/task/45966/exe (readlink: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45966 mgr-fin ceph NOFD /proc/42677/task/45966/fd (opendir: Permission denied) 2024-05-10T04:42:31.656 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45967 mgr-fin ceph cwd unknown /proc/42677/task/45967/cwd (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45967 mgr-fin ceph rtd unknown /proc/42677/task/45967/root (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45967 mgr-fin ceph txt unknown /proc/42677/task/45967/exe (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45967 mgr-fin ceph NOFD /proc/42677/task/45967/fd (opendir: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45968 mgr-fin ceph cwd unknown /proc/42677/task/45968/cwd (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45968 mgr-fin ceph rtd unknown /proc/42677/task/45968/root (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45968 mgr-fin ceph txt unknown /proc/42677/task/45968/exe (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45968 mgr-fin ceph NOFD /proc/42677/task/45968/fd (opendir: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45969 mgr-fin ceph cwd unknown /proc/42677/task/45969/cwd (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45969 mgr-fin ceph rtd unknown /proc/42677/task/45969/root (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45969 mgr-fin ceph txt unknown /proc/42677/task/45969/exe (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45969 mgr-fin ceph NOFD /proc/42677/task/45969/fd (opendir: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45970 mgr-fin ceph cwd unknown /proc/42677/task/45970/cwd (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45970 mgr-fin ceph rtd unknown /proc/42677/task/45970/root (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45970 mgr-fin ceph txt unknown /proc/42677/task/45970/exe (readlink: Permission denied) 2024-05-10T04:42:31.657 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45970 mgr-fin ceph NOFD /proc/42677/task/45970/fd (opendir: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45971 mgr-fin ceph cwd unknown /proc/42677/task/45971/cwd (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45971 mgr-fin ceph rtd unknown /proc/42677/task/45971/root (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45971 mgr-fin ceph txt unknown /proc/42677/task/45971/exe (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45971 mgr-fin ceph NOFD /proc/42677/task/45971/fd (opendir: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45972 mgr-fin ceph cwd unknown /proc/42677/task/45972/cwd (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45972 mgr-fin ceph rtd unknown /proc/42677/task/45972/root (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45972 mgr-fin ceph txt unknown /proc/42677/task/45972/exe (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45972 mgr-fin ceph NOFD /proc/42677/task/45972/fd (opendir: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45973 mgr-fin ceph cwd unknown /proc/42677/task/45973/cwd (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45973 mgr-fin ceph rtd unknown /proc/42677/task/45973/root (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45973 mgr-fin ceph txt unknown /proc/42677/task/45973/exe (readlink: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45973 mgr-fin ceph NOFD /proc/42677/task/45973/fd (opendir: Permission denied) 2024-05-10T04:42:31.658 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45974 mgr-fin ceph cwd unknown /proc/42677/task/45974/cwd (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45974 mgr-fin ceph rtd unknown /proc/42677/task/45974/root (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45974 mgr-fin ceph txt unknown /proc/42677/task/45974/exe (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45974 mgr-fin ceph NOFD /proc/42677/task/45974/fd (opendir: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45975 mgr-fin ceph cwd unknown /proc/42677/task/45975/cwd (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45975 mgr-fin ceph rtd unknown /proc/42677/task/45975/root (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45975 mgr-fin ceph txt unknown /proc/42677/task/45975/exe (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45975 mgr-fin ceph NOFD /proc/42677/task/45975/fd (opendir: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45976 mgr-fin ceph cwd unknown /proc/42677/task/45976/cwd (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45976 mgr-fin ceph rtd unknown /proc/42677/task/45976/root (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45976 mgr-fin ceph txt unknown /proc/42677/task/45976/exe (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45976 mgr-fin ceph NOFD /proc/42677/task/45976/fd (opendir: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45977 mgr-fin ceph cwd unknown /proc/42677/task/45977/cwd (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45977 mgr-fin ceph rtd unknown /proc/42677/task/45977/root (readlink: Permission denied) 2024-05-10T04:42:31.659 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45977 mgr-fin ceph txt unknown /proc/42677/task/45977/exe (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45977 mgr-fin ceph NOFD /proc/42677/task/45977/fd (opendir: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45978 mgr-fin ceph cwd unknown /proc/42677/task/45978/cwd (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45978 mgr-fin ceph rtd unknown /proc/42677/task/45978/root (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45978 mgr-fin ceph txt unknown /proc/42677/task/45978/exe (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45978 mgr-fin ceph NOFD /proc/42677/task/45978/fd (opendir: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45979 mgr-fin ceph cwd unknown /proc/42677/task/45979/cwd (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45979 mgr-fin ceph rtd unknown /proc/42677/task/45979/root (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45979 mgr-fin ceph txt unknown /proc/42677/task/45979/exe (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45979 mgr-fin ceph NOFD /proc/42677/task/45979/fd (opendir: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45980 mgr-fin ceph cwd unknown /proc/42677/task/45980/cwd (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45980 mgr-fin ceph rtd unknown /proc/42677/task/45980/root (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45980 mgr-fin ceph txt unknown /proc/42677/task/45980/exe (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45980 mgr-fin ceph NOFD /proc/42677/task/45980/fd (opendir: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45981 mgr-fin ceph cwd unknown /proc/42677/task/45981/cwd (readlink: Permission denied) 2024-05-10T04:42:31.660 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45981 mgr-fin ceph rtd unknown /proc/42677/task/45981/root (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45981 mgr-fin ceph txt unknown /proc/42677/task/45981/exe (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45981 mgr-fin ceph NOFD /proc/42677/task/45981/fd (opendir: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45982 cephadm ceph cwd unknown /proc/42677/task/45982/cwd (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45982 cephadm ceph rtd unknown /proc/42677/task/45982/root (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45982 cephadm ceph txt unknown /proc/42677/task/45982/exe (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45982 cephadm ceph NOFD /proc/42677/task/45982/fd (opendir: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45983 m-fin-cep ceph cwd unknown /proc/42677/task/45983/cwd (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45983 m-fin-cep ceph rtd unknown /proc/42677/task/45983/root (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45983 m-fin-cep ceph txt unknown /proc/42677/task/45983/exe (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45983 m-fin-cep ceph NOFD /proc/42677/task/45983/fd (opendir: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45984 mgr-fin ceph cwd unknown /proc/42677/task/45984/cwd (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45984 mgr-fin ceph rtd unknown /proc/42677/task/45984/root (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45984 mgr-fin ceph txt unknown /proc/42677/task/45984/exe (readlink: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45984 mgr-fin ceph NOFD /proc/42677/task/45984/fd (opendir: Permission denied) 2024-05-10T04:42:31.661 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45985 mgr-fin ceph cwd unknown /proc/42677/task/45985/cwd (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45985 mgr-fin ceph rtd unknown /proc/42677/task/45985/root (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45985 mgr-fin ceph txt unknown /proc/42677/task/45985/exe (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45985 mgr-fin ceph NOFD /proc/42677/task/45985/fd (opendir: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45986 mgr-fin ceph cwd unknown /proc/42677/task/45986/cwd (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45986 mgr-fin ceph rtd unknown /proc/42677/task/45986/root (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45986 mgr-fin ceph txt unknown /proc/42677/task/45986/exe (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45986 mgr-fin ceph NOFD /proc/42677/task/45986/fd (opendir: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45987 cephadm ceph cwd unknown /proc/42677/task/45987/cwd (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45987 cephadm ceph rtd unknown /proc/42677/task/45987/root (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45987 cephadm ceph txt unknown /proc/42677/task/45987/exe (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45987 cephadm ceph NOFD /proc/42677/task/45987/fd (opendir: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45988 crash ceph cwd unknown /proc/42677/task/45988/cwd (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45988 crash ceph rtd unknown /proc/42677/task/45988/root (readlink: Permission denied) 2024-05-10T04:42:31.662 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45988 crash ceph txt unknown /proc/42677/task/45988/exe (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45988 crash ceph NOFD /proc/42677/task/45988/fd (opendir: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45989 m-fin-cra ceph cwd unknown /proc/42677/task/45989/cwd (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45989 m-fin-cra ceph rtd unknown /proc/42677/task/45989/root (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45989 m-fin-cra ceph txt unknown /proc/42677/task/45989/exe (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45989 m-fin-cra ceph NOFD /proc/42677/task/45989/fd (opendir: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45990 mgr-fin ceph cwd unknown /proc/42677/task/45990/cwd (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45990 mgr-fin ceph rtd unknown /proc/42677/task/45990/root (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45990 mgr-fin ceph txt unknown /proc/42677/task/45990/exe (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45990 mgr-fin ceph NOFD /proc/42677/task/45990/fd (opendir: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45991 mgr-fin ceph cwd unknown /proc/42677/task/45991/cwd (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45991 mgr-fin ceph rtd unknown /proc/42677/task/45991/root (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45991 mgr-fin ceph txt unknown /proc/42677/task/45991/exe (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45991 mgr-fin ceph NOFD /proc/42677/task/45991/fd (opendir: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45992 mgr-fin ceph cwd unknown /proc/42677/task/45992/cwd (readlink: Permission denied) 2024-05-10T04:42:31.663 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45992 mgr-fin ceph rtd unknown /proc/42677/task/45992/root (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45992 mgr-fin ceph txt unknown /proc/42677/task/45992/exe (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45992 mgr-fin ceph NOFD /proc/42677/task/45992/fd (opendir: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45993 dashboard ceph cwd unknown /proc/42677/task/45993/cwd (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45993 dashboard ceph rtd unknown /proc/42677/task/45993/root (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45993 dashboard ceph txt unknown /proc/42677/task/45993/exe (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45993 dashboard ceph NOFD /proc/42677/task/45993/fd (opendir: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45994 m-fin-das ceph cwd unknown /proc/42677/task/45994/cwd (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45994 m-fin-das ceph rtd unknown /proc/42677/task/45994/root (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45994 m-fin-das ceph txt unknown /proc/42677/task/45994/exe (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45994 m-fin-das ceph NOFD /proc/42677/task/45994/fd (opendir: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45995 mgr-fin ceph cwd unknown /proc/42677/task/45995/cwd (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45995 mgr-fin ceph rtd unknown /proc/42677/task/45995/root (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45995 mgr-fin ceph txt unknown /proc/42677/task/45995/exe (readlink: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45995 mgr-fin ceph NOFD /proc/42677/task/45995/fd (opendir: Permission denied) 2024-05-10T04:42:31.664 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45996 mgr-fin ceph cwd unknown /proc/42677/task/45996/cwd (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45996 mgr-fin ceph rtd unknown /proc/42677/task/45996/root (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45996 mgr-fin ceph txt unknown /proc/42677/task/45996/exe (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45996 mgr-fin ceph NOFD /proc/42677/task/45996/fd (opendir: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45997 mgr-fin ceph cwd unknown /proc/42677/task/45997/cwd (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45997 mgr-fin ceph rtd unknown /proc/42677/task/45997/root (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45997 mgr-fin ceph txt unknown /proc/42677/task/45997/exe (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45997 mgr-fin ceph NOFD /proc/42677/task/45997/fd (opendir: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45998 devicehea ceph cwd unknown /proc/42677/task/45998/cwd (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45998 devicehea ceph rtd unknown /proc/42677/task/45998/root (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45998 devicehea ceph txt unknown /proc/42677/task/45998/exe (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45998 devicehea ceph NOFD /proc/42677/task/45998/fd (opendir: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45999 m-fin-dev ceph cwd unknown /proc/42677/task/45999/cwd (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45999 m-fin-dev ceph rtd unknown /proc/42677/task/45999/root (readlink: Permission denied) 2024-05-10T04:42:31.665 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45999 m-fin-dev ceph txt unknown /proc/42677/task/45999/exe (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 45999 m-fin-dev ceph NOFD /proc/42677/task/45999/fd (opendir: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46000 mgr-fin ceph cwd unknown /proc/42677/task/46000/cwd (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46000 mgr-fin ceph rtd unknown /proc/42677/task/46000/root (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46000 mgr-fin ceph txt unknown /proc/42677/task/46000/exe (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46000 mgr-fin ceph NOFD /proc/42677/task/46000/fd (opendir: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46002 m-fin-ios ceph cwd unknown /proc/42677/task/46002/cwd (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46002 m-fin-ios ceph rtd unknown /proc/42677/task/46002/root (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46002 m-fin-ios ceph txt unknown /proc/42677/task/46002/exe (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46002 m-fin-ios ceph NOFD /proc/42677/task/46002/fd (opendir: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46003 mgr-fin ceph cwd unknown /proc/42677/task/46003/cwd (readlink: Permission denied) 2024-05-10T04:42:31.666 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46003 mgr-fin ceph rtd unknown /proc/42677/task/46003/root (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46003 mgr-fin ceph txt unknown /proc/42677/task/46003/exe (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46003 mgr-fin ceph NOFD /proc/42677/task/46003/fd (opendir: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46004 mgr-fin ceph cwd unknown /proc/42677/task/46004/cwd (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46004 mgr-fin ceph rtd unknown /proc/42677/task/46004/root (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46004 mgr-fin ceph txt unknown /proc/42677/task/46004/exe (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46004 mgr-fin ceph NOFD /proc/42677/task/46004/fd (opendir: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46006 m-fin-nfs ceph cwd unknown /proc/42677/task/46006/cwd (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46006 m-fin-nfs ceph rtd unknown /proc/42677/task/46006/root (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46006 m-fin-nfs ceph txt unknown /proc/42677/task/46006/exe (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46006 m-fin-nfs ceph NOFD /proc/42677/task/46006/fd (opendir: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46008 m-fin-orc ceph cwd unknown /proc/42677/task/46008/cwd (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46008 m-fin-orc ceph rtd unknown /proc/42677/task/46008/root (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46008 m-fin-orc ceph txt unknown /proc/42677/task/46008/exe (readlink: Permission denied) 2024-05-10T04:42:31.667 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46008 m-fin-orc ceph NOFD /proc/42677/task/46008/fd (opendir: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46009 pg_autosc ceph cwd unknown /proc/42677/task/46009/cwd (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46009 pg_autosc ceph rtd unknown /proc/42677/task/46009/root (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46009 pg_autosc ceph txt unknown /proc/42677/task/46009/exe (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46009 pg_autosc ceph NOFD /proc/42677/task/46009/fd (opendir: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46010 m-fin-pg_ ceph cwd unknown /proc/42677/task/46010/cwd (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46010 m-fin-pg_ ceph rtd unknown /proc/42677/task/46010/root (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46010 m-fin-pg_ ceph txt unknown /proc/42677/task/46010/exe (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46010 m-fin-pg_ ceph NOFD /proc/42677/task/46010/fd (opendir: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46011 progress ceph cwd unknown /proc/42677/task/46011/cwd (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46011 progress ceph rtd unknown /proc/42677/task/46011/root (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46011 progress ceph txt unknown /proc/42677/task/46011/exe (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46011 progress ceph NOFD /proc/42677/task/46011/fd (opendir: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46012 m-fin-pro ceph cwd unknown /proc/42677/task/46012/cwd (readlink: Permission denied) 2024-05-10T04:42:31.668 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46012 m-fin-pro ceph rtd unknown /proc/42677/task/46012/root (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46012 m-fin-pro ceph txt unknown /proc/42677/task/46012/exe (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46012 m-fin-pro ceph NOFD /proc/42677/task/46012/fd (opendir: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46016 io_contex ceph cwd unknown /proc/42677/task/46016/cwd (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46016 io_contex ceph rtd unknown /proc/42677/task/46016/root (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46016 io_contex ceph txt unknown /proc/42677/task/46016/exe (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46016 io_contex ceph NOFD /proc/42677/task/46016/fd (opendir: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46017 io_contex ceph cwd unknown /proc/42677/task/46017/cwd (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46017 io_contex ceph rtd unknown /proc/42677/task/46017/root (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46017 io_contex ceph txt unknown /proc/42677/task/46017/exe (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46017 io_contex ceph NOFD /proc/42677/task/46017/fd (opendir: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46018 ceph_time ceph cwd unknown /proc/42677/task/46018/cwd (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46018 ceph_time ceph rtd unknown /proc/42677/task/46018/root (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46018 ceph_time ceph txt unknown /proc/42677/task/46018/exe (readlink: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46018 ceph_time ceph NOFD /proc/42677/task/46018/fd (opendir: Permission denied) 2024-05-10T04:42:31.669 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46019 ms_dispat ceph cwd unknown /proc/42677/task/46019/cwd (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46019 ms_dispat ceph rtd unknown /proc/42677/task/46019/root (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46019 ms_dispat ceph txt unknown /proc/42677/task/46019/exe (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46019 ms_dispat ceph NOFD /proc/42677/task/46019/fd (opendir: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46020 ms_local ceph cwd unknown /proc/42677/task/46020/cwd (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46020 ms_local ceph rtd unknown /proc/42677/task/46020/root (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46020 ms_local ceph txt unknown /proc/42677/task/46020/exe (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46020 ms_local ceph NOFD /proc/42677/task/46020/fd (opendir: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46021 safe_time ceph cwd unknown /proc/42677/task/46021/cwd (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46021 safe_time ceph rtd unknown /proc/42677/task/46021/root (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46021 safe_time ceph txt unknown /proc/42677/task/46021/exe (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46021 safe_time ceph NOFD /proc/42677/task/46021/fd (opendir: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46022 safe_time ceph cwd unknown /proc/42677/task/46022/cwd (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46022 safe_time ceph rtd unknown /proc/42677/task/46022/root (readlink: Permission denied) 2024-05-10T04:42:31.670 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46022 safe_time ceph txt unknown /proc/42677/task/46022/exe (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46022 safe_time ceph NOFD /proc/42677/task/46022/fd (opendir: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46023 mgr-fin ceph cwd unknown /proc/42677/task/46023/cwd (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46023 mgr-fin ceph rtd unknown /proc/42677/task/46023/root (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46023 mgr-fin ceph txt unknown /proc/42677/task/46023/exe (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46023 mgr-fin ceph NOFD /proc/42677/task/46023/fd (opendir: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46025 m-fin-rbd ceph cwd unknown /proc/42677/task/46025/cwd (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46025 m-fin-rbd ceph rtd unknown /proc/42677/task/46025/root (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46025 m-fin-rbd ceph txt unknown /proc/42677/task/46025/exe (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46025 m-fin-rbd ceph NOFD /proc/42677/task/46025/fd (opendir: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46026 restful ceph cwd unknown /proc/42677/task/46026/cwd (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46026 restful ceph rtd unknown /proc/42677/task/46026/root (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46026 restful ceph txt unknown /proc/42677/task/46026/exe (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46026 restful ceph NOFD /proc/42677/task/46026/fd (opendir: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46027 m-fin-res ceph cwd unknown /proc/42677/task/46027/cwd (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46027 m-fin-res ceph rtd unknown /proc/42677/task/46027/root (readlink: Permission denied) 2024-05-10T04:42:31.671 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46027 m-fin-res ceph txt unknown /proc/42677/task/46027/exe (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46027 m-fin-res ceph NOFD /proc/42677/task/46027/fd (opendir: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46028 mgr-fin ceph cwd unknown /proc/42677/task/46028/cwd (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46028 mgr-fin ceph rtd unknown /proc/42677/task/46028/root (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46028 mgr-fin ceph txt unknown /proc/42677/task/46028/exe (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46028 mgr-fin ceph NOFD /proc/42677/task/46028/fd (opendir: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46030 m-fin-sta ceph cwd unknown /proc/42677/task/46030/cwd (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46030 m-fin-sta ceph rtd unknown /proc/42677/task/46030/root (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46030 m-fin-sta ceph txt unknown /proc/42677/task/46030/exe (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46030 m-fin-sta ceph NOFD /proc/42677/task/46030/fd (opendir: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46031 mgr-fin ceph cwd unknown /proc/42677/task/46031/cwd (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46031 mgr-fin ceph rtd unknown /proc/42677/task/46031/root (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46031 mgr-fin ceph txt unknown /proc/42677/task/46031/exe (readlink: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46031 mgr-fin ceph NOFD /proc/42677/task/46031/fd (opendir: Permission denied) 2024-05-10T04:42:31.672 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46032 mgr-fin ceph cwd unknown /proc/42677/task/46032/cwd (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46032 mgr-fin ceph rtd unknown /proc/42677/task/46032/root (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46032 mgr-fin ceph txt unknown /proc/42677/task/46032/exe (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46032 mgr-fin ceph NOFD /proc/42677/task/46032/fd (opendir: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46033 mgr-fin ceph cwd unknown /proc/42677/task/46033/cwd (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46033 mgr-fin ceph rtd unknown /proc/42677/task/46033/root (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46033 mgr-fin ceph txt unknown /proc/42677/task/46033/exe (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46033 mgr-fin ceph NOFD /proc/42677/task/46033/fd (opendir: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46034 telemetry ceph cwd unknown /proc/42677/task/46034/cwd (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46034 telemetry ceph rtd unknown /proc/42677/task/46034/root (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46034 telemetry ceph txt unknown /proc/42677/task/46034/exe (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46034 telemetry ceph NOFD /proc/42677/task/46034/fd (opendir: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46035 m-fin-tel ceph cwd unknown /proc/42677/task/46035/cwd (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46035 m-fin-tel ceph rtd unknown /proc/42677/task/46035/root (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46035 m-fin-tel ceph txt unknown /proc/42677/task/46035/exe (readlink: Permission denied) 2024-05-10T04:42:31.673 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46035 m-fin-tel ceph NOFD /proc/42677/task/46035/fd (opendir: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46036 mgr-fin ceph cwd unknown /proc/42677/task/46036/cwd (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46036 mgr-fin ceph rtd unknown /proc/42677/task/46036/root (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46036 mgr-fin ceph txt unknown /proc/42677/task/46036/exe (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46036 mgr-fin ceph NOFD /proc/42677/task/46036/fd (opendir: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46040 io_contex ceph cwd unknown /proc/42677/task/46040/cwd (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46040 io_contex ceph rtd unknown /proc/42677/task/46040/root (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46040 io_contex ceph txt unknown /proc/42677/task/46040/exe (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46040 io_contex ceph NOFD /proc/42677/task/46040/fd (opendir: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46042 io_contex ceph cwd unknown /proc/42677/task/46042/cwd (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46042 io_contex ceph rtd unknown /proc/42677/task/46042/root (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46042 io_contex ceph txt unknown /proc/42677/task/46042/exe (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46042 io_contex ceph NOFD /proc/42677/task/46042/fd (opendir: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46043 ceph_time ceph cwd unknown /proc/42677/task/46043/cwd (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46043 ceph_time ceph rtd unknown /proc/42677/task/46043/root (readlink: Permission denied) 2024-05-10T04:42:31.674 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46043 ceph_time ceph txt unknown /proc/42677/task/46043/exe (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46043 ceph_time ceph NOFD /proc/42677/task/46043/fd (opendir: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46044 ms_dispat ceph cwd unknown /proc/42677/task/46044/cwd (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46044 ms_dispat ceph rtd unknown /proc/42677/task/46044/root (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46044 ms_dispat ceph txt unknown /proc/42677/task/46044/exe (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46044 ms_dispat ceph NOFD /proc/42677/task/46044/fd (opendir: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46045 ms_local ceph cwd unknown /proc/42677/task/46045/cwd (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46045 ms_local ceph rtd unknown /proc/42677/task/46045/root (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46045 ms_local ceph txt unknown /proc/42677/task/46045/exe (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46045 ms_local ceph NOFD /proc/42677/task/46045/fd (opendir: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46046 safe_time ceph cwd unknown /proc/42677/task/46046/cwd (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46046 safe_time ceph rtd unknown /proc/42677/task/46046/root (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46046 safe_time ceph txt unknown /proc/42677/task/46046/exe (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46046 safe_time ceph NOFD /proc/42677/task/46046/fd (opendir: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46047 safe_time ceph cwd unknown /proc/42677/task/46047/cwd (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46047 safe_time ceph rtd unknown /proc/42677/task/46047/root (readlink: Permission denied) 2024-05-10T04:42:31.675 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46047 safe_time ceph txt unknown /proc/42677/task/46047/exe (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46047 safe_time ceph NOFD /proc/42677/task/46047/fd (opendir: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46048 mgr-fin ceph cwd unknown /proc/42677/task/46048/cwd (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46048 mgr-fin ceph rtd unknown /proc/42677/task/46048/root (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46048 mgr-fin ceph txt unknown /proc/42677/task/46048/exe (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46048 mgr-fin ceph NOFD /proc/42677/task/46048/fd (opendir: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46049 mgr-fin ceph cwd unknown /proc/42677/task/46049/cwd (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46049 mgr-fin ceph rtd unknown /proc/42677/task/46049/root (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46049 mgr-fin ceph txt unknown /proc/42677/task/46049/exe (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46049 mgr-fin ceph NOFD /proc/42677/task/46049/fd (opendir: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46050 mgr-fin ceph cwd unknown /proc/42677/task/46050/cwd (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46050 mgr-fin ceph rtd unknown /proc/42677/task/46050/root (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46050 mgr-fin ceph txt unknown /proc/42677/task/46050/exe (readlink: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46050 mgr-fin ceph NOFD /proc/42677/task/46050/fd (opendir: Permission denied) 2024-05-10T04:42:31.676 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46051 mgr-fin ceph cwd unknown /proc/42677/task/46051/cwd (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46051 mgr-fin ceph rtd unknown /proc/42677/task/46051/root (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46051 mgr-fin ceph txt unknown /proc/42677/task/46051/exe (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46051 mgr-fin ceph NOFD /proc/42677/task/46051/fd (opendir: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46052 mgr-fin ceph cwd unknown /proc/42677/task/46052/cwd (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46052 mgr-fin ceph rtd unknown /proc/42677/task/46052/root (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46052 mgr-fin ceph txt unknown /proc/42677/task/46052/exe (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46052 mgr-fin ceph NOFD /proc/42677/task/46052/fd (opendir: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46053 mgr-fin ceph cwd unknown /proc/42677/task/46053/cwd (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46053 mgr-fin ceph rtd unknown /proc/42677/task/46053/root (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46053 mgr-fin ceph txt unknown /proc/42677/task/46053/exe (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46053 mgr-fin ceph NOFD /proc/42677/task/46053/fd (opendir: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46054 mgr-fin ceph cwd unknown /proc/42677/task/46054/cwd (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46054 mgr-fin ceph rtd unknown /proc/42677/task/46054/root (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46054 mgr-fin ceph txt unknown /proc/42677/task/46054/exe (readlink: Permission denied) 2024-05-10T04:42:31.677 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46054 mgr-fin ceph NOFD /proc/42677/task/46054/fd (opendir: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46055 mgr-fin ceph cwd unknown /proc/42677/task/46055/cwd (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46055 mgr-fin ceph rtd unknown /proc/42677/task/46055/root (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46055 mgr-fin ceph txt unknown /proc/42677/task/46055/exe (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46055 mgr-fin ceph NOFD /proc/42677/task/46055/fd (opendir: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46056 mgr-fin ceph cwd unknown /proc/42677/task/46056/cwd (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46056 mgr-fin ceph rtd unknown /proc/42677/task/46056/root (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46056 mgr-fin ceph txt unknown /proc/42677/task/46056/exe (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46056 mgr-fin ceph NOFD /proc/42677/task/46056/fd (opendir: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46057 mgr-fin ceph cwd unknown /proc/42677/task/46057/cwd (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46057 mgr-fin ceph rtd unknown /proc/42677/task/46057/root (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46057 mgr-fin ceph txt unknown /proc/42677/task/46057/exe (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46057 mgr-fin ceph NOFD /proc/42677/task/46057/fd (opendir: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46058 mgr-fin ceph cwd unknown /proc/42677/task/46058/cwd (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46058 mgr-fin ceph rtd unknown /proc/42677/task/46058/root (readlink: Permission denied) 2024-05-10T04:42:31.678 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46058 mgr-fin ceph txt unknown /proc/42677/task/46058/exe (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46058 mgr-fin ceph NOFD /proc/42677/task/46058/fd (opendir: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46059 mgr-fin ceph cwd unknown /proc/42677/task/46059/cwd (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46059 mgr-fin ceph rtd unknown /proc/42677/task/46059/root (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46059 mgr-fin ceph txt unknown /proc/42677/task/46059/exe (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46059 mgr-fin ceph NOFD /proc/42677/task/46059/fd (opendir: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46061 m-fin-vol ceph cwd unknown /proc/42677/task/46061/cwd (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46061 m-fin-vol ceph rtd unknown /proc/42677/task/46061/root (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46061 m-fin-vol ceph txt unknown /proc/42677/task/46061/exe (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46061 m-fin-vol ceph NOFD /proc/42677/task/46061/fd (opendir: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46474 mgr-fin ceph cwd unknown /proc/42677/task/46474/cwd (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46474 mgr-fin ceph rtd unknown /proc/42677/task/46474/root (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46474 mgr-fin ceph txt unknown /proc/42677/task/46474/exe (readlink: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46474 mgr-fin ceph NOFD /proc/42677/task/46474/fd (opendir: Permission denied) 2024-05-10T04:42:31.679 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46475 mgr-fin ceph cwd unknown /proc/42677/task/46475/cwd (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46475 mgr-fin ceph rtd unknown /proc/42677/task/46475/root (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46475 mgr-fin ceph txt unknown /proc/42677/task/46475/exe (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46475 mgr-fin ceph NOFD /proc/42677/task/46475/fd (opendir: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46476 mgr-fin ceph cwd unknown /proc/42677/task/46476/cwd (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46476 mgr-fin ceph rtd unknown /proc/42677/task/46476/root (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46476 mgr-fin ceph txt unknown /proc/42677/task/46476/exe (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46476 mgr-fin ceph NOFD /proc/42677/task/46476/fd (opendir: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46477 mgr-fin ceph cwd unknown /proc/42677/task/46477/cwd (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46477 mgr-fin ceph rtd unknown /proc/42677/task/46477/root (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46477 mgr-fin ceph txt unknown /proc/42677/task/46477/exe (readlink: Permission denied) 2024-05-10T04:42:31.680 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46477 mgr-fin ceph NOFD /proc/42677/task/46477/fd (opendir: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46478 mgr-fin ceph cwd unknown /proc/42677/task/46478/cwd (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46478 mgr-fin ceph rtd unknown /proc/42677/task/46478/root (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46478 mgr-fin ceph txt unknown /proc/42677/task/46478/exe (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46478 mgr-fin ceph NOFD /proc/42677/task/46478/fd (opendir: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46479 mgr-fin ceph cwd unknown /proc/42677/task/46479/cwd (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46479 mgr-fin ceph rtd unknown /proc/42677/task/46479/root (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46479 mgr-fin ceph txt unknown /proc/42677/task/46479/exe (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46479 mgr-fin ceph NOFD /proc/42677/task/46479/fd (opendir: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46480 mgr-fin ceph cwd unknown /proc/42677/task/46480/cwd (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46480 mgr-fin ceph rtd unknown /proc/42677/task/46480/root (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46480 mgr-fin ceph txt unknown /proc/42677/task/46480/exe (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46480 mgr-fin ceph NOFD /proc/42677/task/46480/fd (opendir: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46481 mgr-fin ceph cwd unknown /proc/42677/task/46481/cwd (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46481 mgr-fin ceph rtd unknown /proc/42677/task/46481/root (readlink: Permission denied) 2024-05-10T04:42:31.681 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46481 mgr-fin ceph txt unknown /proc/42677/task/46481/exe (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46481 mgr-fin ceph NOFD /proc/42677/task/46481/fd (opendir: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46482 mgr-fin ceph cwd unknown /proc/42677/task/46482/cwd (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46482 mgr-fin ceph rtd unknown /proc/42677/task/46482/root (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46482 mgr-fin ceph txt unknown /proc/42677/task/46482/exe (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46482 mgr-fin ceph NOFD /proc/42677/task/46482/fd (opendir: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46483 mgr-fin ceph cwd unknown /proc/42677/task/46483/cwd (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46483 mgr-fin ceph rtd unknown /proc/42677/task/46483/root (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46483 mgr-fin ceph txt unknown /proc/42677/task/46483/exe (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46483 mgr-fin ceph NOFD /proc/42677/task/46483/fd (opendir: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46484 mgr-fin ceph cwd unknown /proc/42677/task/46484/cwd (readlink: Permission denied) 2024-05-10T04:42:31.682 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46484 mgr-fin ceph rtd unknown /proc/42677/task/46484/root (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46484 mgr-fin ceph txt unknown /proc/42677/task/46484/exe (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46484 mgr-fin ceph NOFD /proc/42677/task/46484/fd (opendir: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46513 mgr-fin ceph cwd unknown /proc/42677/task/46513/cwd (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46513 mgr-fin ceph rtd unknown /proc/42677/task/46513/root (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46513 mgr-fin ceph txt unknown /proc/42677/task/46513/exe (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46513 mgr-fin ceph NOFD /proc/42677/task/46513/fd (opendir: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46515 mgr-fin ceph cwd unknown /proc/42677/task/46515/cwd (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46515 mgr-fin ceph rtd unknown /proc/42677/task/46515/root (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46515 mgr-fin ceph txt unknown /proc/42677/task/46515/exe (readlink: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46515 mgr-fin ceph NOFD /proc/42677/task/46515/fd (opendir: Permission denied) 2024-05-10T04:42:31.683 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46516 mgr-fin ceph cwd unknown /proc/42677/task/46516/cwd (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46516 mgr-fin ceph rtd unknown /proc/42677/task/46516/root (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46516 mgr-fin ceph txt unknown /proc/42677/task/46516/exe (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46516 mgr-fin ceph NOFD /proc/42677/task/46516/fd (opendir: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46517 mgr-fin ceph cwd unknown /proc/42677/task/46517/cwd (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46517 mgr-fin ceph rtd unknown /proc/42677/task/46517/root (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46517 mgr-fin ceph txt unknown /proc/42677/task/46517/exe (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46517 mgr-fin ceph NOFD /proc/42677/task/46517/fd (opendir: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46518 mgr-fin ceph cwd unknown /proc/42677/task/46518/cwd (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46518 mgr-fin ceph rtd unknown /proc/42677/task/46518/root (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46518 mgr-fin ceph txt unknown /proc/42677/task/46518/exe (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46518 mgr-fin ceph NOFD /proc/42677/task/46518/fd (opendir: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46519 mgr-fin ceph cwd unknown /proc/42677/task/46519/cwd (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46519 mgr-fin ceph rtd unknown /proc/42677/task/46519/root (readlink: Permission denied) 2024-05-10T04:42:31.684 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46519 mgr-fin ceph txt unknown /proc/42677/task/46519/exe (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46519 mgr-fin ceph NOFD /proc/42677/task/46519/fd (opendir: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46520 mgr-fin ceph cwd unknown /proc/42677/task/46520/cwd (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46520 mgr-fin ceph rtd unknown /proc/42677/task/46520/root (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46520 mgr-fin ceph txt unknown /proc/42677/task/46520/exe (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46520 mgr-fin ceph NOFD /proc/42677/task/46520/fd (opendir: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46521 mgr-fin ceph cwd unknown /proc/42677/task/46521/cwd (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46521 mgr-fin ceph rtd unknown /proc/42677/task/46521/root (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46521 mgr-fin ceph txt unknown /proc/42677/task/46521/exe (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46521 mgr-fin ceph NOFD /proc/42677/task/46521/fd (opendir: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46522 mgr-fin ceph cwd unknown /proc/42677/task/46522/cwd (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46522 mgr-fin ceph rtd unknown /proc/42677/task/46522/root (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46522 mgr-fin ceph txt unknown /proc/42677/task/46522/exe (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46522 mgr-fin ceph NOFD /proc/42677/task/46522/fd (opendir: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46523 mgr-fin ceph cwd unknown /proc/42677/task/46523/cwd (readlink: Permission denied) 2024-05-10T04:42:31.685 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46523 mgr-fin ceph rtd unknown /proc/42677/task/46523/root (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46523 mgr-fin ceph txt unknown /proc/42677/task/46523/exe (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46523 mgr-fin ceph NOFD /proc/42677/task/46523/fd (opendir: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46524 mgr-fin ceph cwd unknown /proc/42677/task/46524/cwd (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46524 mgr-fin ceph rtd unknown /proc/42677/task/46524/root (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46524 mgr-fin ceph txt unknown /proc/42677/task/46524/exe (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46524 mgr-fin ceph NOFD /proc/42677/task/46524/fd (opendir: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46964 dashboard ceph cwd unknown /proc/42677/task/46964/cwd (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46964 dashboard ceph rtd unknown /proc/42677/task/46964/root (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46964 dashboard ceph txt unknown /proc/42677/task/46964/exe (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46964 dashboard ceph NOFD /proc/42677/task/46964/fd (opendir: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46965 dashboard ceph cwd unknown /proc/42677/task/46965/cwd (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46965 dashboard ceph rtd unknown /proc/42677/task/46965/root (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46965 dashboard ceph txt unknown /proc/42677/task/46965/exe (readlink: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46965 dashboard ceph NOFD /proc/42677/task/46965/fd (opendir: Permission denied) 2024-05-10T04:42:31.686 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46966 dashboard ceph cwd unknown /proc/42677/task/46966/cwd (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46966 dashboard ceph rtd unknown /proc/42677/task/46966/root (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46966 dashboard ceph txt unknown /proc/42677/task/46966/exe (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46966 dashboard ceph NOFD /proc/42677/task/46966/fd (opendir: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46967 dashboard ceph cwd unknown /proc/42677/task/46967/cwd (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46967 dashboard ceph rtd unknown /proc/42677/task/46967/root (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46967 dashboard ceph txt unknown /proc/42677/task/46967/exe (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46967 dashboard ceph NOFD /proc/42677/task/46967/fd (opendir: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46968 dashboard ceph cwd unknown /proc/42677/task/46968/cwd (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46968 dashboard ceph rtd unknown /proc/42677/task/46968/root (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46968 dashboard ceph txt unknown /proc/42677/task/46968/exe (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46968 dashboard ceph NOFD /proc/42677/task/46968/fd (opendir: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46969 dashboard ceph cwd unknown /proc/42677/task/46969/cwd (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46969 dashboard ceph rtd unknown /proc/42677/task/46969/root (readlink: Permission denied) 2024-05-10T04:42:31.687 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46969 dashboard ceph txt unknown /proc/42677/task/46969/exe (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46969 dashboard ceph NOFD /proc/42677/task/46969/fd (opendir: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46970 dashboard ceph cwd unknown /proc/42677/task/46970/cwd (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46970 dashboard ceph rtd unknown /proc/42677/task/46970/root (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46970 dashboard ceph txt unknown /proc/42677/task/46970/exe (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46970 dashboard ceph NOFD /proc/42677/task/46970/fd (opendir: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46971 dashboard ceph cwd unknown /proc/42677/task/46971/cwd (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46971 dashboard ceph rtd unknown /proc/42677/task/46971/root (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46971 dashboard ceph txt unknown /proc/42677/task/46971/exe (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46971 dashboard ceph NOFD /proc/42677/task/46971/fd (opendir: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46972 dashboard ceph cwd unknown /proc/42677/task/46972/cwd (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46972 dashboard ceph rtd unknown /proc/42677/task/46972/root (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46972 dashboard ceph txt unknown /proc/42677/task/46972/exe (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46972 dashboard ceph NOFD /proc/42677/task/46972/fd (opendir: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46973 dashboard ceph cwd unknown /proc/42677/task/46973/cwd (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46973 dashboard ceph rtd unknown /proc/42677/task/46973/root (readlink: Permission denied) 2024-05-10T04:42:31.688 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46973 dashboard ceph txt unknown /proc/42677/task/46973/exe (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46973 dashboard ceph NOFD /proc/42677/task/46973/fd (opendir: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46974 dashboard ceph cwd unknown /proc/42677/task/46974/cwd (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46974 dashboard ceph rtd unknown /proc/42677/task/46974/root (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46974 dashboard ceph txt unknown /proc/42677/task/46974/exe (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46974 dashboard ceph NOFD /proc/42677/task/46974/fd (opendir: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46988 dashboard ceph cwd unknown /proc/42677/task/46988/cwd (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46988 dashboard ceph rtd unknown /proc/42677/task/46988/root (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46988 dashboard ceph txt unknown /proc/42677/task/46988/exe (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 46988 dashboard ceph NOFD /proc/42677/task/46988/fd (opendir: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54847 cephadm ceph cwd unknown /proc/42677/task/54847/cwd (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54847 cephadm ceph rtd unknown /proc/42677/task/54847/root (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54847 cephadm ceph txt unknown /proc/42677/task/54847/exe (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54847 cephadm ceph NOFD /proc/42677/task/54847/fd (opendir: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54848 cephadm ceph cwd unknown /proc/42677/task/54848/cwd (readlink: Permission denied) 2024-05-10T04:42:31.689 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54848 cephadm ceph rtd unknown /proc/42677/task/54848/root (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54848 cephadm ceph txt unknown /proc/42677/task/54848/exe (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54848 cephadm ceph NOFD /proc/42677/task/54848/fd (opendir: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54849 cephadm ceph cwd unknown /proc/42677/task/54849/cwd (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54849 cephadm ceph rtd unknown /proc/42677/task/54849/root (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54849 cephadm ceph txt unknown /proc/42677/task/54849/exe (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54849 cephadm ceph NOFD /proc/42677/task/54849/fd (opendir: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54850 cephadm ceph cwd unknown /proc/42677/task/54850/cwd (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54850 cephadm ceph rtd unknown /proc/42677/task/54850/root (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54850 cephadm ceph txt unknown /proc/42677/task/54850/exe (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54850 cephadm ceph NOFD /proc/42677/task/54850/fd (opendir: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54851 cephadm ceph cwd unknown /proc/42677/task/54851/cwd (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54851 cephadm ceph rtd unknown /proc/42677/task/54851/root (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54851 cephadm ceph txt unknown /proc/42677/task/54851/exe (readlink: Permission denied) 2024-05-10T04:42:31.690 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54851 cephadm ceph NOFD /proc/42677/task/54851/fd (opendir: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54852 cephadm ceph cwd unknown /proc/42677/task/54852/cwd (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54852 cephadm ceph rtd unknown /proc/42677/task/54852/root (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54852 cephadm ceph txt unknown /proc/42677/task/54852/exe (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54852 cephadm ceph NOFD /proc/42677/task/54852/fd (opendir: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54853 cephadm ceph cwd unknown /proc/42677/task/54853/cwd (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54853 cephadm ceph rtd unknown /proc/42677/task/54853/root (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54853 cephadm ceph txt unknown /proc/42677/task/54853/exe (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54853 cephadm ceph NOFD /proc/42677/task/54853/fd (opendir: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54854 cephadm ceph cwd unknown /proc/42677/task/54854/cwd (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54854 cephadm ceph rtd unknown /proc/42677/task/54854/root (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54854 cephadm ceph txt unknown /proc/42677/task/54854/exe (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54854 cephadm ceph NOFD /proc/42677/task/54854/fd (opendir: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54855 cephadm ceph cwd unknown /proc/42677/task/54855/cwd (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54855 cephadm ceph rtd unknown /proc/42677/task/54855/root (readlink: Permission denied) 2024-05-10T04:42:31.691 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54855 cephadm ceph txt unknown /proc/42677/task/54855/exe (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54855 cephadm ceph NOFD /proc/42677/task/54855/fd (opendir: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54856 cephadm ceph cwd unknown /proc/42677/task/54856/cwd (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54856 cephadm ceph rtd unknown /proc/42677/task/54856/root (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54856 cephadm ceph txt unknown /proc/42677/task/54856/exe (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 54856 cephadm ceph NOFD /proc/42677/task/54856/fd (opendir: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55289 cephadm ceph cwd unknown /proc/42677/task/55289/cwd (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55289 cephadm ceph rtd unknown /proc/42677/task/55289/root (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55289 cephadm ceph txt unknown /proc/42677/task/55289/exe (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55289 cephadm ceph NOFD /proc/42677/task/55289/fd (opendir: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55290 cephadm ceph cwd unknown /proc/42677/task/55290/cwd (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55290 cephadm ceph rtd unknown /proc/42677/task/55290/root (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55290 cephadm ceph txt unknown /proc/42677/task/55290/exe (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55290 cephadm ceph NOFD /proc/42677/task/55290/fd (opendir: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55291 cephadm ceph cwd unknown /proc/42677/task/55291/cwd (readlink: Permission denied) 2024-05-10T04:42:31.692 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55291 cephadm ceph rtd unknown /proc/42677/task/55291/root (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55291 cephadm ceph txt unknown /proc/42677/task/55291/exe (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55291 cephadm ceph NOFD /proc/42677/task/55291/fd (opendir: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55292 cephadm ceph cwd unknown /proc/42677/task/55292/cwd (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55292 cephadm ceph rtd unknown /proc/42677/task/55292/root (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55292 cephadm ceph txt unknown /proc/42677/task/55292/exe (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55292 cephadm ceph NOFD /proc/42677/task/55292/fd (opendir: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55293 cephadm ceph cwd unknown /proc/42677/task/55293/cwd (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55293 cephadm ceph rtd unknown /proc/42677/task/55293/root (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55293 cephadm ceph txt unknown /proc/42677/task/55293/exe (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55293 cephadm ceph NOFD /proc/42677/task/55293/fd (opendir: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55294 cephadm ceph cwd unknown /proc/42677/task/55294/cwd (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55294 cephadm ceph rtd unknown /proc/42677/task/55294/root (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55294 cephadm ceph txt unknown /proc/42677/task/55294/exe (readlink: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55294 cephadm ceph NOFD /proc/42677/task/55294/fd (opendir: Permission denied) 2024-05-10T04:42:31.693 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55295 cephadm ceph cwd unknown /proc/42677/task/55295/cwd (readlink: Permission denied) 2024-05-10T04:42:31.694 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55295 cephadm ceph rtd unknown /proc/42677/task/55295/root (readlink: Permission denied) 2024-05-10T04:42:31.694 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55295 cephadm ceph txt unknown /proc/42677/task/55295/exe (readlink: Permission denied) 2024-05-10T04:42:31.694 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55295 cephadm ceph NOFD /proc/42677/task/55295/fd (opendir: Permission denied) 2024-05-10T04:42:31.694 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55296 cephadm ceph cwd unknown /proc/42677/task/55296/cwd (readlink: Permission denied) 2024-05-10T04:42:31.694 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55296 cephadm ceph rtd unknown /proc/42677/task/55296/root (readlink: Permission denied) 2024-05-10T04:42:31.695 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55296 cephadm ceph txt unknown /proc/42677/task/55296/exe (readlink: Permission denied) 2024-05-10T04:42:31.695 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55296 cephadm ceph NOFD /proc/42677/task/55296/fd (opendir: Permission denied) 2024-05-10T04:42:31.695 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55297 cephadm ceph cwd unknown /proc/42677/task/55297/cwd (readlink: Permission denied) 2024-05-10T04:42:31.695 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55297 cephadm ceph rtd unknown /proc/42677/task/55297/root (readlink: Permission denied) 2024-05-10T04:42:31.696 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55297 cephadm ceph txt unknown /proc/42677/task/55297/exe (readlink: Permission denied) 2024-05-10T04:42:31.696 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55297 cephadm ceph NOFD /proc/42677/task/55297/fd (opendir: Permission denied) 2024-05-10T04:42:31.696 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55298 cephadm ceph cwd unknown /proc/42677/task/55298/cwd (readlink: Permission denied) 2024-05-10T04:42:31.696 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55298 cephadm ceph rtd unknown /proc/42677/task/55298/root (readlink: Permission denied) 2024-05-10T04:42:31.696 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55298 cephadm ceph txt unknown /proc/42677/task/55298/exe (readlink: Permission denied) 2024-05-10T04:42:31.696 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 55298 cephadm ceph NOFD /proc/42677/task/55298/fd (opendir: Permission denied) 2024-05-10T04:42:31.697 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67360 devicehea ceph cwd unknown /proc/42677/task/67360/cwd (readlink: Permission denied) 2024-05-10T04:42:31.697 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67360 devicehea ceph rtd unknown /proc/42677/task/67360/root (readlink: Permission denied) 2024-05-10T04:42:31.697 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67360 devicehea ceph txt unknown /proc/42677/task/67360/exe (readlink: Permission denied) 2024-05-10T04:42:31.698 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67360 devicehea ceph NOFD /proc/42677/task/67360/fd (opendir: Permission denied) 2024-05-10T04:42:31.698 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67366 io_contex ceph cwd unknown /proc/42677/task/67366/cwd (readlink: Permission denied) 2024-05-10T04:42:31.698 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67366 io_contex ceph rtd unknown /proc/42677/task/67366/root (readlink: Permission denied) 2024-05-10T04:42:31.699 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67366 io_contex ceph txt unknown /proc/42677/task/67366/exe (readlink: Permission denied) 2024-05-10T04:42:31.699 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67366 io_contex ceph NOFD /proc/42677/task/67366/fd (opendir: Permission denied) 2024-05-10T04:42:31.699 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67367 io_contex ceph cwd unknown /proc/42677/task/67367/cwd (readlink: Permission denied) 2024-05-10T04:42:31.699 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67367 io_contex ceph rtd unknown /proc/42677/task/67367/root (readlink: Permission denied) 2024-05-10T04:42:31.699 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67367 io_contex ceph txt unknown /proc/42677/task/67367/exe (readlink: Permission denied) 2024-05-10T04:42:31.699 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67367 io_contex ceph NOFD /proc/42677/task/67367/fd (opendir: Permission denied) 2024-05-10T04:42:31.700 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67368 ceph_time ceph cwd unknown /proc/42677/task/67368/cwd (readlink: Permission denied) 2024-05-10T04:42:31.700 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67368 ceph_time ceph rtd unknown /proc/42677/task/67368/root (readlink: Permission denied) 2024-05-10T04:42:31.700 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67368 ceph_time ceph txt unknown /proc/42677/task/67368/exe (readlink: Permission denied) 2024-05-10T04:42:31.700 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67368 ceph_time ceph NOFD /proc/42677/task/67368/fd (opendir: Permission denied) 2024-05-10T04:42:31.701 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67369 ms_dispat ceph cwd unknown /proc/42677/task/67369/cwd (readlink: Permission denied) 2024-05-10T04:42:31.701 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67369 ms_dispat ceph rtd unknown /proc/42677/task/67369/root (readlink: Permission denied) 2024-05-10T04:42:31.702 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67369 ms_dispat ceph txt unknown /proc/42677/task/67369/exe (readlink: Permission denied) 2024-05-10T04:42:31.702 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67369 ms_dispat ceph NOFD /proc/42677/task/67369/fd (opendir: Permission denied) 2024-05-10T04:42:31.702 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67370 ms_local ceph cwd unknown /proc/42677/task/67370/cwd (readlink: Permission denied) 2024-05-10T04:42:31.702 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67370 ms_local ceph rtd unknown /proc/42677/task/67370/root (readlink: Permission denied) 2024-05-10T04:42:31.702 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67370 ms_local ceph txt unknown /proc/42677/task/67370/exe (readlink: Permission denied) 2024-05-10T04:42:31.702 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67370 ms_local ceph NOFD /proc/42677/task/67370/fd (opendir: Permission denied) 2024-05-10T04:42:31.702 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67371 safe_time ceph cwd unknown /proc/42677/task/67371/cwd (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67371 safe_time ceph rtd unknown /proc/42677/task/67371/root (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67371 safe_time ceph txt unknown /proc/42677/task/67371/exe (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67371 safe_time ceph NOFD /proc/42677/task/67371/fd (opendir: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67372 safe_time ceph cwd unknown /proc/42677/task/67372/cwd (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67372 safe_time ceph rtd unknown /proc/42677/task/67372/root (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67372 safe_time ceph txt unknown /proc/42677/task/67372/exe (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mgr 42677 67372 safe_time ceph NOFD /proc/42677/task/67372/fd (opendir: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 44160 root cwd unknown /proc/44160/cwd (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 44160 root rtd unknown /proc/44160/root (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 44160 root txt unknown /proc/44160/exe (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:systemd 44160 root NOFD /proc/44160/fd (opendir: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 44164 root cwd unknown /proc/44164/cwd (readlink: Permission denied) 2024-05-10T04:42:31.703 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 44164 root rtd unknown /proc/44164/root (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 44164 root txt unknown /proc/44164/exe (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:(sd-pam) 44164 root NOFD /proc/44164/fd (opendir: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 44166 root cwd unknown /proc/44166/cwd (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 44166 root rtd unknown /proc/44166/root (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 44166 root txt unknown /proc/44166/exe (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/3 44166 root NOFD /proc/44166/fd (opendir: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46041 root cwd unknown /proc/46041/cwd (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46041 root rtd unknown /proc/46041/root (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46041 root txt unknown /proc/46041/exe (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46041 root NOFD /proc/46041/fd (opendir: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46064 root cwd unknown /proc/46064/cwd (readlink: Permission denied) 2024-05-10T04:42:31.704 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46064 root rtd unknown /proc/46064/root (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46064 root txt unknown /proc/46064/exe (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:sshd 46064 root NOFD /proc/46064/fd (opendir: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 54984 root cwd unknown /proc/54984/cwd (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 54984 root rtd unknown /proc/54984/root (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 54984 root txt unknown /proc/54984/exe (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 54984 root NOFD /proc/54984/fd (opendir: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 60251 root cwd unknown /proc/60251/cwd (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 60251 root rtd unknown /proc/60251/root (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 60251 root txt unknown /proc/60251/exe (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 60251 root NOFD /proc/60251/fd (opendir: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 60262 root cwd unknown /proc/60262/cwd (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 60262 root rtd unknown /proc/60262/root (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 60262 root txt unknown /proc/60262/exe (readlink: Permission denied) 2024-05-10T04:42:31.705 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 60262 root NOFD /proc/60262/fd (opendir: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 ceph cwd unknown /proc/60274/cwd (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 ceph rtd unknown /proc/60274/root (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 ceph txt unknown /proc/60274/exe (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 ceph NOFD /proc/60274/fd (opendir: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60275 log ceph cwd unknown /proc/60274/task/60275/cwd (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60275 log ceph rtd unknown /proc/60274/task/60275/root (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60275 log ceph txt unknown /proc/60274/task/60275/exe (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60275 log ceph NOFD /proc/60274/task/60275/fd (opendir: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60277 msgr-work ceph cwd unknown /proc/60274/task/60277/cwd (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60277 msgr-work ceph rtd unknown /proc/60274/task/60277/root (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60277 msgr-work ceph txt unknown /proc/60274/task/60277/exe (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60277 msgr-work ceph NOFD /proc/60274/task/60277/fd (opendir: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60278 msgr-work ceph cwd unknown /proc/60274/task/60278/cwd (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60278 msgr-work ceph rtd unknown /proc/60274/task/60278/root (readlink: Permission denied) 2024-05-10T04:42:31.706 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60278 msgr-work ceph txt unknown /proc/60274/task/60278/exe (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60278 msgr-work ceph NOFD /proc/60274/task/60278/fd (opendir: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60279 msgr-work ceph cwd unknown /proc/60274/task/60279/cwd (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60279 msgr-work ceph rtd unknown /proc/60274/task/60279/root (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60279 msgr-work ceph txt unknown /proc/60274/task/60279/exe (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60279 msgr-work ceph NOFD /proc/60274/task/60279/fd (opendir: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60283 service ceph cwd unknown /proc/60274/task/60283/cwd (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60283 service ceph rtd unknown /proc/60274/task/60283/root (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60283 service ceph txt unknown /proc/60274/task/60283/exe (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60283 service ceph NOFD /proc/60274/task/60283/fd (opendir: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60284 admin_soc ceph cwd unknown /proc/60274/task/60284/cwd (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60284 admin_soc ceph rtd unknown /proc/60274/task/60284/root (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60284 admin_soc ceph txt unknown /proc/60274/task/60284/exe (readlink: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60284 admin_soc ceph NOFD /proc/60274/task/60284/fd (opendir: Permission denied) 2024-05-10T04:42:31.707 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60574 signal_ha ceph cwd unknown /proc/60274/task/60574/cwd (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60574 signal_ha ceph rtd unknown /proc/60274/task/60574/root (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60574 signal_ha ceph txt unknown /proc/60274/task/60574/exe (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60574 signal_ha ceph NOFD /proc/60274/task/60574/fd (opendir: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60575 io_contex ceph cwd unknown /proc/60274/task/60575/cwd (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60575 io_contex ceph rtd unknown /proc/60274/task/60575/root (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60575 io_contex ceph txt unknown /proc/60274/task/60575/exe (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60575 io_contex ceph NOFD /proc/60274/task/60575/fd (opendir: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60576 io_contex ceph cwd unknown /proc/60274/task/60576/cwd (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60576 io_contex ceph rtd unknown /proc/60274/task/60576/root (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60576 io_contex ceph txt unknown /proc/60274/task/60576/exe (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60576 io_contex ceph NOFD /proc/60274/task/60576/fd (opendir: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60638 OpHistory ceph cwd unknown /proc/60274/task/60638/cwd (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60638 OpHistory ceph rtd unknown /proc/60274/task/60638/root (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60638 OpHistory ceph txt unknown /proc/60274/task/60638/exe (readlink: Permission denied) 2024-05-10T04:42:31.708 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60638 OpHistory ceph NOFD /proc/60274/task/60638/fd (opendir: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60639 ceph_time ceph cwd unknown /proc/60274/task/60639/cwd (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60639 ceph_time ceph rtd unknown /proc/60274/task/60639/root (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60639 ceph_time ceph txt unknown /proc/60274/task/60639/exe (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 60639 ceph_time ceph NOFD /proc/60274/task/60639/fd (opendir: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61130 ceph-osd ceph cwd unknown /proc/60274/task/61130/cwd (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61130 ceph-osd ceph rtd unknown /proc/60274/task/61130/root (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61130 ceph-osd ceph txt unknown /proc/60274/task/61130/exe (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61130 ceph-osd ceph NOFD /proc/60274/task/61130/fd (opendir: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61131 safe_time ceph cwd unknown /proc/60274/task/61131/cwd (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61131 safe_time ceph rtd unknown /proc/60274/task/61131/root (readlink: Permission denied) 2024-05-10T04:42:31.709 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61131 safe_time ceph txt unknown /proc/60274/task/61131/exe (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61131 safe_time ceph NOFD /proc/60274/task/61131/fd (opendir: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61132 safe_time ceph cwd unknown /proc/60274/task/61132/cwd (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61132 safe_time ceph rtd unknown /proc/60274/task/61132/root (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61132 safe_time ceph txt unknown /proc/60274/task/61132/exe (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61132 safe_time ceph NOFD /proc/60274/task/61132/fd (opendir: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61133 safe_time ceph cwd unknown /proc/60274/task/61133/cwd (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61133 safe_time ceph rtd unknown /proc/60274/task/61133/root (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61133 safe_time ceph txt unknown /proc/60274/task/61133/exe (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61133 safe_time ceph NOFD /proc/60274/task/61133/fd (opendir: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61134 safe_time ceph cwd unknown /proc/60274/task/61134/cwd (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61134 safe_time ceph rtd unknown /proc/60274/task/61134/root (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61134 safe_time ceph txt unknown /proc/60274/task/61134/exe (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61134 safe_time ceph NOFD /proc/60274/task/61134/fd (opendir: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61135 fn_anonym ceph cwd unknown /proc/60274/task/61135/cwd (readlink: Permission denied) 2024-05-10T04:42:31.710 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61135 fn_anonym ceph rtd unknown /proc/60274/task/61135/root (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61135 fn_anonym ceph txt unknown /proc/60274/task/61135/exe (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61135 fn_anonym ceph NOFD /proc/60274/task/61135/fd (opendir: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61144 rocksdb:l ceph cwd unknown /proc/60274/task/61144/cwd (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61144 rocksdb:l ceph rtd unknown /proc/60274/task/61144/root (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61144 rocksdb:l ceph txt unknown /proc/60274/task/61144/exe (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61144 rocksdb:l ceph NOFD /proc/60274/task/61144/fd (opendir: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61147 rocksdb:l ceph cwd unknown /proc/60274/task/61147/cwd (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61147 rocksdb:l ceph rtd unknown /proc/60274/task/61147/root (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61147 rocksdb:l ceph txt unknown /proc/60274/task/61147/exe (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61147 rocksdb:l ceph NOFD /proc/60274/task/61147/fd (opendir: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61148 rocksdb:l ceph cwd unknown /proc/60274/task/61148/cwd (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61148 rocksdb:l ceph rtd unknown /proc/60274/task/61148/root (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61148 rocksdb:l ceph txt unknown /proc/60274/task/61148/exe (readlink: Permission denied) 2024-05-10T04:42:31.711 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61148 rocksdb:l ceph NOFD /proc/60274/task/61148/fd (opendir: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61149 rocksdb:h ceph cwd unknown /proc/60274/task/61149/cwd (readlink: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61149 rocksdb:h ceph rtd unknown /proc/60274/task/61149/root (readlink: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61149 rocksdb:h ceph txt unknown /proc/60274/task/61149/exe (readlink: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61149 rocksdb:h ceph NOFD /proc/60274/task/61149/fd (opendir: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61575 bstore_ai ceph cwd unknown /proc/60274/task/61575/cwd (readlink: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61575 bstore_ai ceph rtd unknown /proc/60274/task/61575/root (readlink: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61575 bstore_ai ceph txt unknown /proc/60274/task/61575/exe (readlink: Permission denied) 2024-05-10T04:42:31.712 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 61575 bstore_ai ceph NOFD /proc/60274/task/61575/fd (opendir: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62415 bstore_ai ceph cwd unknown /proc/60274/task/62415/cwd (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62415 bstore_ai ceph rtd unknown /proc/60274/task/62415/root (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62415 bstore_ai ceph txt unknown /proc/60274/task/62415/exe (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62415 bstore_ai ceph NOFD /proc/60274/task/62415/fd (opendir: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62728 ceph-osd ceph cwd unknown /proc/60274/task/62728/cwd (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62728 ceph-osd ceph rtd unknown /proc/60274/task/62728/root (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62728 ceph-osd ceph txt unknown /proc/60274/task/62728/exe (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62728 ceph-osd ceph NOFD /proc/60274/task/62728/fd (opendir: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62733 cfin ceph cwd unknown /proc/60274/task/62733/cwd (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62733 cfin ceph rtd unknown /proc/60274/task/62733/root (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62733 cfin ceph txt unknown /proc/60274/task/62733/exe (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62733 cfin ceph NOFD /proc/60274/task/62733/fd (opendir: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62734 bstore_kv ceph cwd unknown /proc/60274/task/62734/cwd (readlink: Permission denied) 2024-05-10T04:42:31.713 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62734 bstore_kv ceph rtd unknown /proc/60274/task/62734/root (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62734 bstore_kv ceph txt unknown /proc/60274/task/62734/exe (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62734 bstore_kv ceph NOFD /proc/60274/task/62734/fd (opendir: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62735 bstore_kv ceph cwd unknown /proc/60274/task/62735/cwd (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62735 bstore_kv ceph rtd unknown /proc/60274/task/62735/root (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62735 bstore_kv ceph txt unknown /proc/60274/task/62735/exe (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62735 bstore_kv ceph NOFD /proc/60274/task/62735/fd (opendir: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62737 bstore_me ceph cwd unknown /proc/60274/task/62737/cwd (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62737 bstore_me ceph rtd unknown /proc/60274/task/62737/root (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62737 bstore_me ceph txt unknown /proc/60274/task/62737/exe (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62737 bstore_me ceph NOFD /proc/60274/task/62737/fd (opendir: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62841 ms_dispat ceph cwd unknown /proc/60274/task/62841/cwd (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62841 ms_dispat ceph rtd unknown /proc/60274/task/62841/root (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62841 ms_dispat ceph txt unknown /proc/60274/task/62841/exe (readlink: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62841 ms_dispat ceph NOFD /proc/60274/task/62841/fd (opendir: Permission denied) 2024-05-10T04:42:31.714 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62842 ms_local ceph cwd unknown /proc/60274/task/62842/cwd (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62842 ms_local ceph rtd unknown /proc/60274/task/62842/root (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62842 ms_local ceph txt unknown /proc/60274/task/62842/exe (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62842 ms_local ceph NOFD /proc/60274/task/62842/fd (opendir: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62843 safe_time ceph cwd unknown /proc/60274/task/62843/cwd (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62843 safe_time ceph rtd unknown /proc/60274/task/62843/root (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62843 safe_time ceph txt unknown /proc/60274/task/62843/exe (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62843 safe_time ceph NOFD /proc/60274/task/62843/fd (opendir: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62845 safe_time ceph cwd unknown /proc/60274/task/62845/cwd (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62845 safe_time ceph rtd unknown /proc/60274/task/62845/root (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62845 safe_time ceph txt unknown /proc/60274/task/62845/exe (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62845 safe_time ceph NOFD /proc/60274/task/62845/fd (opendir: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62846 ms_dispat ceph cwd unknown /proc/60274/task/62846/cwd (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62846 ms_dispat ceph rtd unknown /proc/60274/task/62846/root (readlink: Permission denied) 2024-05-10T04:42:31.715 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62846 ms_dispat ceph txt unknown /proc/60274/task/62846/exe (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62846 ms_dispat ceph NOFD /proc/60274/task/62846/fd (opendir: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62848 ms_local ceph cwd unknown /proc/60274/task/62848/cwd (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62848 ms_local ceph rtd unknown /proc/60274/task/62848/root (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62848 ms_local ceph txt unknown /proc/60274/task/62848/exe (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62848 ms_local ceph NOFD /proc/60274/task/62848/fd (opendir: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62849 ms_dispat ceph cwd unknown /proc/60274/task/62849/cwd (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62849 ms_dispat ceph rtd unknown /proc/60274/task/62849/root (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62849 ms_dispat ceph txt unknown /proc/60274/task/62849/exe (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62849 ms_dispat ceph NOFD /proc/60274/task/62849/fd (opendir: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62850 ms_local ceph cwd unknown /proc/60274/task/62850/cwd (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62850 ms_local ceph rtd unknown /proc/60274/task/62850/root (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62850 ms_local ceph txt unknown /proc/60274/task/62850/exe (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62850 ms_local ceph NOFD /proc/60274/task/62850/fd (opendir: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62851 ms_dispat ceph cwd unknown /proc/60274/task/62851/cwd (readlink: Permission denied) 2024-05-10T04:42:31.716 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62851 ms_dispat ceph rtd unknown /proc/60274/task/62851/root (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62851 ms_dispat ceph txt unknown /proc/60274/task/62851/exe (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62851 ms_dispat ceph NOFD /proc/60274/task/62851/fd (opendir: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62852 ms_local ceph cwd unknown /proc/60274/task/62852/cwd (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62852 ms_local ceph rtd unknown /proc/60274/task/62852/root (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62852 ms_local ceph txt unknown /proc/60274/task/62852/exe (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62852 ms_local ceph NOFD /proc/60274/task/62852/fd (opendir: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62854 ms_dispat ceph cwd unknown /proc/60274/task/62854/cwd (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62854 ms_dispat ceph rtd unknown /proc/60274/task/62854/root (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62854 ms_dispat ceph txt unknown /proc/60274/task/62854/exe (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62854 ms_dispat ceph NOFD /proc/60274/task/62854/fd (opendir: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62855 ms_local ceph cwd unknown /proc/60274/task/62855/cwd (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62855 ms_local ceph rtd unknown /proc/60274/task/62855/root (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62855 ms_local ceph txt unknown /proc/60274/task/62855/exe (readlink: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62855 ms_local ceph NOFD /proc/60274/task/62855/fd (opendir: Permission denied) 2024-05-10T04:42:31.717 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62856 ms_dispat ceph cwd unknown /proc/60274/task/62856/cwd (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62856 ms_dispat ceph rtd unknown /proc/60274/task/62856/root (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62856 ms_dispat ceph txt unknown /proc/60274/task/62856/exe (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62856 ms_dispat ceph NOFD /proc/60274/task/62856/fd (opendir: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62857 ms_local ceph cwd unknown /proc/60274/task/62857/cwd (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62857 ms_local ceph rtd unknown /proc/60274/task/62857/root (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62857 ms_local ceph txt unknown /proc/60274/task/62857/exe (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62857 ms_local ceph NOFD /proc/60274/task/62857/fd (opendir: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62859 ms_dispat ceph cwd unknown /proc/60274/task/62859/cwd (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62859 ms_dispat ceph rtd unknown /proc/60274/task/62859/root (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62859 ms_dispat ceph txt unknown /proc/60274/task/62859/exe (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62859 ms_dispat ceph NOFD /proc/60274/task/62859/fd (opendir: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62860 ms_local ceph cwd unknown /proc/60274/task/62860/cwd (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62860 ms_local ceph rtd unknown /proc/60274/task/62860/root (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62860 ms_local ceph txt unknown /proc/60274/task/62860/exe (readlink: Permission denied) 2024-05-10T04:42:31.718 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62860 ms_local ceph NOFD /proc/60274/task/62860/fd (opendir: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62861 fn_anonym ceph cwd unknown /proc/60274/task/62861/cwd (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62861 fn_anonym ceph rtd unknown /proc/60274/task/62861/root (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62861 fn_anonym ceph txt unknown /proc/60274/task/62861/exe (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62861 fn_anonym ceph NOFD /proc/60274/task/62861/fd (opendir: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62862 finisher ceph cwd unknown /proc/60274/task/62862/cwd (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62862 finisher ceph rtd unknown /proc/60274/task/62862/root (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62862 finisher ceph txt unknown /proc/60274/task/62862/exe (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62862 finisher ceph NOFD /proc/60274/task/62862/fd (opendir: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62864 safe_time ceph cwd unknown /proc/60274/task/62864/cwd (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62864 safe_time ceph rtd unknown /proc/60274/task/62864/root (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62864 safe_time ceph txt unknown /proc/60274/task/62864/exe (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62864 safe_time ceph NOFD /proc/60274/task/62864/fd (opendir: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62865 safe_time ceph cwd unknown /proc/60274/task/62865/cwd (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62865 safe_time ceph rtd unknown /proc/60274/task/62865/root (readlink: Permission denied) 2024-05-10T04:42:31.719 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62865 safe_time ceph txt unknown /proc/60274/task/62865/exe (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62865 safe_time ceph NOFD /proc/60274/task/62865/fd (opendir: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62866 ceph-osd ceph cwd unknown /proc/60274/task/62866/cwd (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62866 ceph-osd ceph rtd unknown /proc/60274/task/62866/root (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62866 ceph-osd ceph txt unknown /proc/60274/task/62866/exe (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62866 ceph-osd ceph NOFD /proc/60274/task/62866/fd (opendir: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62867 osd_srv_a ceph cwd unknown /proc/60274/task/62867/cwd (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62867 osd_srv_a ceph rtd unknown /proc/60274/task/62867/root (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62867 osd_srv_a ceph txt unknown /proc/60274/task/62867/exe (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62867 osd_srv_a ceph NOFD /proc/60274/task/62867/fd (opendir: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62868 tp_osd_tp ceph cwd unknown /proc/60274/task/62868/cwd (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62868 tp_osd_tp ceph rtd unknown /proc/60274/task/62868/root (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62868 tp_osd_tp ceph txt unknown /proc/60274/task/62868/exe (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62868 tp_osd_tp ceph NOFD /proc/60274/task/62868/fd (opendir: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62870 tp_osd_tp ceph cwd unknown /proc/60274/task/62870/cwd (readlink: Permission denied) 2024-05-10T04:42:31.720 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62870 tp_osd_tp ceph rtd unknown /proc/60274/task/62870/root (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62870 tp_osd_tp ceph txt unknown /proc/60274/task/62870/exe (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62870 tp_osd_tp ceph NOFD /proc/60274/task/62870/fd (opendir: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62871 tp_osd_tp ceph cwd unknown /proc/60274/task/62871/cwd (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62871 tp_osd_tp ceph rtd unknown /proc/60274/task/62871/root (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62871 tp_osd_tp ceph txt unknown /proc/60274/task/62871/exe (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62871 tp_osd_tp ceph NOFD /proc/60274/task/62871/fd (opendir: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62872 tp_osd_tp ceph cwd unknown /proc/60274/task/62872/cwd (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62872 tp_osd_tp ceph rtd unknown /proc/60274/task/62872/root (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62872 tp_osd_tp ceph txt unknown /proc/60274/task/62872/exe (readlink: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62872 tp_osd_tp ceph NOFD /proc/60274/task/62872/fd (opendir: Permission denied) 2024-05-10T04:42:31.721 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62874 tp_osd_tp ceph cwd unknown /proc/60274/task/62874/cwd (readlink: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62874 tp_osd_tp ceph rtd unknown /proc/60274/task/62874/root (readlink: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62874 tp_osd_tp ceph txt unknown /proc/60274/task/62874/exe (readlink: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62874 tp_osd_tp ceph NOFD /proc/60274/task/62874/fd (opendir: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62875 tp_osd_tp ceph cwd unknown /proc/60274/task/62875/cwd (readlink: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62875 tp_osd_tp ceph rtd unknown /proc/60274/task/62875/root (readlink: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62875 tp_osd_tp ceph txt unknown /proc/60274/task/62875/exe (readlink: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62875 tp_osd_tp ceph NOFD /proc/60274/task/62875/fd (opendir: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62876 tp_osd_tp ceph cwd unknown /proc/60274/task/62876/cwd (readlink: Permission denied) 2024-05-10T04:42:31.722 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62876 tp_osd_tp ceph rtd unknown /proc/60274/task/62876/root (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62876 tp_osd_tp ceph txt unknown /proc/60274/task/62876/exe (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62876 tp_osd_tp ceph NOFD /proc/60274/task/62876/fd (opendir: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62877 tp_osd_tp ceph cwd unknown /proc/60274/task/62877/cwd (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62877 tp_osd_tp ceph rtd unknown /proc/60274/task/62877/root (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62877 tp_osd_tp ceph txt unknown /proc/60274/task/62877/exe (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62877 tp_osd_tp ceph NOFD /proc/60274/task/62877/fd (opendir: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62878 tp_osd_tp ceph cwd unknown /proc/60274/task/62878/cwd (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62878 tp_osd_tp ceph rtd unknown /proc/60274/task/62878/root (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62878 tp_osd_tp ceph txt unknown /proc/60274/task/62878/exe (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62878 tp_osd_tp ceph NOFD /proc/60274/task/62878/fd (opendir: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62880 tp_osd_tp ceph cwd unknown /proc/60274/task/62880/cwd (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62880 tp_osd_tp ceph rtd unknown /proc/60274/task/62880/root (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62880 tp_osd_tp ceph txt unknown /proc/60274/task/62880/exe (readlink: Permission denied) 2024-05-10T04:42:31.723 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62880 tp_osd_tp ceph NOFD /proc/60274/task/62880/fd (opendir: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62881 tp_osd_tp ceph cwd unknown /proc/60274/task/62881/cwd (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62881 tp_osd_tp ceph rtd unknown /proc/60274/task/62881/root (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62881 tp_osd_tp ceph txt unknown /proc/60274/task/62881/exe (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62881 tp_osd_tp ceph NOFD /proc/60274/task/62881/fd (opendir: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62882 tp_osd_tp ceph cwd unknown /proc/60274/task/62882/cwd (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62882 tp_osd_tp ceph rtd unknown /proc/60274/task/62882/root (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62882 tp_osd_tp ceph txt unknown /proc/60274/task/62882/exe (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62882 tp_osd_tp ceph NOFD /proc/60274/task/62882/fd (opendir: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62884 tp_osd_tp ceph cwd unknown /proc/60274/task/62884/cwd (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62884 tp_osd_tp ceph rtd unknown /proc/60274/task/62884/root (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62884 tp_osd_tp ceph txt unknown /proc/60274/task/62884/exe (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62884 tp_osd_tp ceph NOFD /proc/60274/task/62884/fd (opendir: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62885 tp_osd_tp ceph cwd unknown /proc/60274/task/62885/cwd (readlink: Permission denied) 2024-05-10T04:42:31.724 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62885 tp_osd_tp ceph rtd unknown /proc/60274/task/62885/root (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62885 tp_osd_tp ceph txt unknown /proc/60274/task/62885/exe (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62885 tp_osd_tp ceph NOFD /proc/60274/task/62885/fd (opendir: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62886 tp_osd_tp ceph cwd unknown /proc/60274/task/62886/cwd (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62886 tp_osd_tp ceph rtd unknown /proc/60274/task/62886/root (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62886 tp_osd_tp ceph txt unknown /proc/60274/task/62886/exe (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62886 tp_osd_tp ceph NOFD /proc/60274/task/62886/fd (opendir: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62887 tp_osd_tp ceph cwd unknown /proc/60274/task/62887/cwd (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62887 tp_osd_tp ceph rtd unknown /proc/60274/task/62887/root (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62887 tp_osd_tp ceph txt unknown /proc/60274/task/62887/exe (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62887 tp_osd_tp ceph NOFD /proc/60274/task/62887/fd (opendir: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62889 osd_srv_h ceph cwd unknown /proc/60274/task/62889/cwd (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62889 osd_srv_h ceph rtd unknown /proc/60274/task/62889/root (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62889 osd_srv_h ceph txt unknown /proc/60274/task/62889/exe (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 60274 62889 osd_srv_h ceph NOFD /proc/60274/task/62889/fd (opendir: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 60619 root cwd unknown /proc/60619/cwd (readlink: Permission denied) 2024-05-10T04:42:31.725 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 60619 root rtd unknown /proc/60619/root (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 60619 root txt unknown /proc/60619/exe (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/0 60619 root NOFD /proc/60619/fd (opendir: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 61063 root cwd unknown /proc/61063/cwd (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 61063 root rtd unknown /proc/61063/root (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 61063 root txt unknown /proc/61063/exe (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 61063 root NOFD /proc/61063/fd (opendir: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 61086 root cwd unknown /proc/61086/cwd (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 61086 root rtd unknown /proc/61086/root (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 61086 root txt unknown /proc/61086/exe (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 61086 root NOFD /proc/61086/fd (opendir: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 68363 root cwd unknown /proc/68363/cwd (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 68363 root rtd unknown /proc/68363/root (readlink: Permission denied) 2024-05-10T04:42:31.726 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 68363 root txt unknown /proc/68363/exe (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/4 68363 root NOFD /proc/68363/fd (opendir: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 68766 root cwd unknown /proc/68766/cwd (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 68766 root rtd unknown /proc/68766/root (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 68766 root txt unknown /proc/68766/exe (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 68766 root NOFD /proc/68766/fd (opendir: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 68777 root cwd unknown /proc/68777/cwd (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 68777 root rtd unknown /proc/68777/root (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 68777 root txt unknown /proc/68777/exe (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 68777 root NOFD /proc/68777/fd (opendir: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 ceph cwd unknown /proc/68789/cwd (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 ceph rtd unknown /proc/68789/root (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 ceph txt unknown /proc/68789/exe (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 ceph NOFD /proc/68789/fd (opendir: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68790 log ceph cwd unknown /proc/68789/task/68790/cwd (readlink: Permission denied) 2024-05-10T04:42:31.727 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68790 log ceph rtd unknown /proc/68789/task/68790/root (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68790 log ceph txt unknown /proc/68789/task/68790/exe (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68790 log ceph NOFD /proc/68789/task/68790/fd (opendir: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68792 msgr-work ceph cwd unknown /proc/68789/task/68792/cwd (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68792 msgr-work ceph rtd unknown /proc/68789/task/68792/root (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68792 msgr-work ceph txt unknown /proc/68789/task/68792/exe (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68792 msgr-work ceph NOFD /proc/68789/task/68792/fd (opendir: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68793 msgr-work ceph cwd unknown /proc/68789/task/68793/cwd (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68793 msgr-work ceph rtd unknown /proc/68789/task/68793/root (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68793 msgr-work ceph txt unknown /proc/68789/task/68793/exe (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68793 msgr-work ceph NOFD /proc/68789/task/68793/fd (opendir: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68794 msgr-work ceph cwd unknown /proc/68789/task/68794/cwd (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68794 msgr-work ceph rtd unknown /proc/68789/task/68794/root (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68794 msgr-work ceph txt unknown /proc/68789/task/68794/exe (readlink: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68794 msgr-work ceph NOFD /proc/68789/task/68794/fd (opendir: Permission denied) 2024-05-10T04:42:31.728 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68798 service ceph cwd unknown /proc/68789/task/68798/cwd (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68798 service ceph rtd unknown /proc/68789/task/68798/root (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68798 service ceph txt unknown /proc/68789/task/68798/exe (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68798 service ceph NOFD /proc/68789/task/68798/fd (opendir: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68799 admin_soc ceph cwd unknown /proc/68789/task/68799/cwd (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68799 admin_soc ceph rtd unknown /proc/68789/task/68799/root (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68799 admin_soc ceph txt unknown /proc/68789/task/68799/exe (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 68799 admin_soc ceph NOFD /proc/68789/task/68799/fd (opendir: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69089 signal_ha ceph cwd unknown /proc/68789/task/69089/cwd (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69089 signal_ha ceph rtd unknown /proc/68789/task/69089/root (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69089 signal_ha ceph txt unknown /proc/68789/task/69089/exe (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69089 signal_ha ceph NOFD /proc/68789/task/69089/fd (opendir: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69090 io_contex ceph cwd unknown /proc/68789/task/69090/cwd (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69090 io_contex ceph rtd unknown /proc/68789/task/69090/root (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69090 io_contex ceph txt unknown /proc/68789/task/69090/exe (readlink: Permission denied) 2024-05-10T04:42:31.729 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69090 io_contex ceph NOFD /proc/68789/task/69090/fd (opendir: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69091 io_contex ceph cwd unknown /proc/68789/task/69091/cwd (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69091 io_contex ceph rtd unknown /proc/68789/task/69091/root (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69091 io_contex ceph txt unknown /proc/68789/task/69091/exe (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69091 io_contex ceph NOFD /proc/68789/task/69091/fd (opendir: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69149 OpHistory ceph cwd unknown /proc/68789/task/69149/cwd (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69149 OpHistory ceph rtd unknown /proc/68789/task/69149/root (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69149 OpHistory ceph txt unknown /proc/68789/task/69149/exe (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69149 OpHistory ceph NOFD /proc/68789/task/69149/fd (opendir: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69150 ceph_time ceph cwd unknown /proc/68789/task/69150/cwd (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69150 ceph_time ceph rtd unknown /proc/68789/task/69150/root (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69150 ceph_time ceph txt unknown /proc/68789/task/69150/exe (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69150 ceph_time ceph NOFD /proc/68789/task/69150/fd (opendir: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69564 ceph-osd ceph cwd unknown /proc/68789/task/69564/cwd (readlink: Permission denied) 2024-05-10T04:42:31.730 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69564 ceph-osd ceph rtd unknown /proc/68789/task/69564/root (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69564 ceph-osd ceph txt unknown /proc/68789/task/69564/exe (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69564 ceph-osd ceph NOFD /proc/68789/task/69564/fd (opendir: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69565 safe_time ceph cwd unknown /proc/68789/task/69565/cwd (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69565 safe_time ceph rtd unknown /proc/68789/task/69565/root (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69565 safe_time ceph txt unknown /proc/68789/task/69565/exe (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69565 safe_time ceph NOFD /proc/68789/task/69565/fd (opendir: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69566 safe_time ceph cwd unknown /proc/68789/task/69566/cwd (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69566 safe_time ceph rtd unknown /proc/68789/task/69566/root (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69566 safe_time ceph txt unknown /proc/68789/task/69566/exe (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69566 safe_time ceph NOFD /proc/68789/task/69566/fd (opendir: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69567 safe_time ceph cwd unknown /proc/68789/task/69567/cwd (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69567 safe_time ceph rtd unknown /proc/68789/task/69567/root (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69567 safe_time ceph txt unknown /proc/68789/task/69567/exe (readlink: Permission denied) 2024-05-10T04:42:31.731 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69567 safe_time ceph NOFD /proc/68789/task/69567/fd (opendir: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69568 safe_time ceph cwd unknown /proc/68789/task/69568/cwd (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69568 safe_time ceph rtd unknown /proc/68789/task/69568/root (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69568 safe_time ceph txt unknown /proc/68789/task/69568/exe (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69568 safe_time ceph NOFD /proc/68789/task/69568/fd (opendir: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69569 fn_anonym ceph cwd unknown /proc/68789/task/69569/cwd (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69569 fn_anonym ceph rtd unknown /proc/68789/task/69569/root (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69569 fn_anonym ceph txt unknown /proc/68789/task/69569/exe (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69569 fn_anonym ceph NOFD /proc/68789/task/69569/fd (opendir: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69573 rocksdb:l ceph cwd unknown /proc/68789/task/69573/cwd (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69573 rocksdb:l ceph rtd unknown /proc/68789/task/69573/root (readlink: Permission denied) 2024-05-10T04:42:31.732 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69573 rocksdb:l ceph txt unknown /proc/68789/task/69573/exe (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69573 rocksdb:l ceph NOFD /proc/68789/task/69573/fd (opendir: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69574 rocksdb:l ceph cwd unknown /proc/68789/task/69574/cwd (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69574 rocksdb:l ceph rtd unknown /proc/68789/task/69574/root (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69574 rocksdb:l ceph txt unknown /proc/68789/task/69574/exe (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69574 rocksdb:l ceph NOFD /proc/68789/task/69574/fd (opendir: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69575 rocksdb:l ceph cwd unknown /proc/68789/task/69575/cwd (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69575 rocksdb:l ceph rtd unknown /proc/68789/task/69575/root (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69575 rocksdb:l ceph txt unknown /proc/68789/task/69575/exe (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69575 rocksdb:l ceph NOFD /proc/68789/task/69575/fd (opendir: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69576 rocksdb:h ceph cwd unknown /proc/68789/task/69576/cwd (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69576 rocksdb:h ceph rtd unknown /proc/68789/task/69576/root (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69576 rocksdb:h ceph txt unknown /proc/68789/task/69576/exe (readlink: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 69576 rocksdb:h ceph NOFD /proc/68789/task/69576/fd (opendir: Permission denied) 2024-05-10T04:42:31.733 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 70022 bstore_ai ceph cwd unknown /proc/68789/task/70022/cwd (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 70022 bstore_ai ceph rtd unknown /proc/68789/task/70022/root (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 70022 bstore_ai ceph txt unknown /proc/68789/task/70022/exe (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 70022 bstore_ai ceph NOFD /proc/68789/task/70022/fd (opendir: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72062 bstore_ai ceph cwd unknown /proc/68789/task/72062/cwd (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72062 bstore_ai ceph rtd unknown /proc/68789/task/72062/root (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72062 bstore_ai ceph txt unknown /proc/68789/task/72062/exe (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72062 bstore_ai ceph NOFD /proc/68789/task/72062/fd (opendir: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72367 ceph-osd ceph cwd unknown /proc/68789/task/72367/cwd (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72367 ceph-osd ceph rtd unknown /proc/68789/task/72367/root (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72367 ceph-osd ceph txt unknown /proc/68789/task/72367/exe (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72367 ceph-osd ceph NOFD /proc/68789/task/72367/fd (opendir: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72372 cfin ceph cwd unknown /proc/68789/task/72372/cwd (readlink: Permission denied) 2024-05-10T04:42:31.734 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72372 cfin ceph rtd unknown /proc/68789/task/72372/root (readlink: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72372 cfin ceph txt unknown /proc/68789/task/72372/exe (readlink: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72372 cfin ceph NOFD /proc/68789/task/72372/fd (opendir: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72373 bstore_kv ceph cwd unknown /proc/68789/task/72373/cwd (readlink: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72373 bstore_kv ceph rtd unknown /proc/68789/task/72373/root (readlink: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72373 bstore_kv ceph txt unknown /proc/68789/task/72373/exe (readlink: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72373 bstore_kv ceph NOFD /proc/68789/task/72373/fd (opendir: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72374 bstore_kv ceph cwd unknown /proc/68789/task/72374/cwd (readlink: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72374 bstore_kv ceph rtd unknown /proc/68789/task/72374/root (readlink: Permission denied) 2024-05-10T04:42:31.738 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72374 bstore_kv ceph txt unknown /proc/68789/task/72374/exe (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72374 bstore_kv ceph NOFD /proc/68789/task/72374/fd (opendir: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72376 bstore_me ceph cwd unknown /proc/68789/task/72376/cwd (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72376 bstore_me ceph rtd unknown /proc/68789/task/72376/root (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72376 bstore_me ceph txt unknown /proc/68789/task/72376/exe (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72376 bstore_me ceph NOFD /proc/68789/task/72376/fd (opendir: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72503 ms_dispat ceph cwd unknown /proc/68789/task/72503/cwd (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72503 ms_dispat ceph rtd unknown /proc/68789/task/72503/root (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72503 ms_dispat ceph txt unknown /proc/68789/task/72503/exe (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72503 ms_dispat ceph NOFD /proc/68789/task/72503/fd (opendir: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72505 ms_local ceph cwd unknown /proc/68789/task/72505/cwd (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72505 ms_local ceph rtd unknown /proc/68789/task/72505/root (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72505 ms_local ceph txt unknown /proc/68789/task/72505/exe (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72505 ms_local ceph NOFD /proc/68789/task/72505/fd (opendir: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72506 safe_time ceph cwd unknown /proc/68789/task/72506/cwd (readlink: Permission denied) 2024-05-10T04:42:31.739 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72506 safe_time ceph rtd unknown /proc/68789/task/72506/root (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72506 safe_time ceph txt unknown /proc/68789/task/72506/exe (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72506 safe_time ceph NOFD /proc/68789/task/72506/fd (opendir: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72507 safe_time ceph cwd unknown /proc/68789/task/72507/cwd (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72507 safe_time ceph rtd unknown /proc/68789/task/72507/root (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72507 safe_time ceph txt unknown /proc/68789/task/72507/exe (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72507 safe_time ceph NOFD /proc/68789/task/72507/fd (opendir: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72509 ms_dispat ceph cwd unknown /proc/68789/task/72509/cwd (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72509 ms_dispat ceph rtd unknown /proc/68789/task/72509/root (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72509 ms_dispat ceph txt unknown /proc/68789/task/72509/exe (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72509 ms_dispat ceph NOFD /proc/68789/task/72509/fd (opendir: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72510 ms_local ceph cwd unknown /proc/68789/task/72510/cwd (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72510 ms_local ceph rtd unknown /proc/68789/task/72510/root (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72510 ms_local ceph txt unknown /proc/68789/task/72510/exe (readlink: Permission denied) 2024-05-10T04:42:31.740 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72510 ms_local ceph NOFD /proc/68789/task/72510/fd (opendir: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72511 ms_dispat ceph cwd unknown /proc/68789/task/72511/cwd (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72511 ms_dispat ceph rtd unknown /proc/68789/task/72511/root (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72511 ms_dispat ceph txt unknown /proc/68789/task/72511/exe (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72511 ms_dispat ceph NOFD /proc/68789/task/72511/fd (opendir: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72512 ms_local ceph cwd unknown /proc/68789/task/72512/cwd (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72512 ms_local ceph rtd unknown /proc/68789/task/72512/root (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72512 ms_local ceph txt unknown /proc/68789/task/72512/exe (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72512 ms_local ceph NOFD /proc/68789/task/72512/fd (opendir: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72514 ms_dispat ceph cwd unknown /proc/68789/task/72514/cwd (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72514 ms_dispat ceph rtd unknown /proc/68789/task/72514/root (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72514 ms_dispat ceph txt unknown /proc/68789/task/72514/exe (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72514 ms_dispat ceph NOFD /proc/68789/task/72514/fd (opendir: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72515 ms_local ceph cwd unknown /proc/68789/task/72515/cwd (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72515 ms_local ceph rtd unknown /proc/68789/task/72515/root (readlink: Permission denied) 2024-05-10T04:42:31.741 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72515 ms_local ceph txt unknown /proc/68789/task/72515/exe (readlink: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72515 ms_local ceph NOFD /proc/68789/task/72515/fd (opendir: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72516 ms_dispat ceph cwd unknown /proc/68789/task/72516/cwd (readlink: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72516 ms_dispat ceph rtd unknown /proc/68789/task/72516/root (readlink: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72516 ms_dispat ceph txt unknown /proc/68789/task/72516/exe (readlink: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72516 ms_dispat ceph NOFD /proc/68789/task/72516/fd (opendir: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72517 ms_local ceph cwd unknown /proc/68789/task/72517/cwd (readlink: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72517 ms_local ceph rtd unknown /proc/68789/task/72517/root (readlink: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72517 ms_local ceph txt unknown /proc/68789/task/72517/exe (readlink: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72517 ms_local ceph NOFD /proc/68789/task/72517/fd (opendir: Permission denied) 2024-05-10T04:42:31.742 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72518 ms_dispat ceph cwd unknown /proc/68789/task/72518/cwd (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72518 ms_dispat ceph rtd unknown /proc/68789/task/72518/root (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72518 ms_dispat ceph txt unknown /proc/68789/task/72518/exe (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72518 ms_dispat ceph NOFD /proc/68789/task/72518/fd (opendir: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72520 ms_local ceph cwd unknown /proc/68789/task/72520/cwd (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72520 ms_local ceph rtd unknown /proc/68789/task/72520/root (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72520 ms_local ceph txt unknown /proc/68789/task/72520/exe (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72520 ms_local ceph NOFD /proc/68789/task/72520/fd (opendir: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72521 ms_dispat ceph cwd unknown /proc/68789/task/72521/cwd (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72521 ms_dispat ceph rtd unknown /proc/68789/task/72521/root (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72521 ms_dispat ceph txt unknown /proc/68789/task/72521/exe (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72521 ms_dispat ceph NOFD /proc/68789/task/72521/fd (opendir: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72522 ms_local ceph cwd unknown /proc/68789/task/72522/cwd (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72522 ms_local ceph rtd unknown /proc/68789/task/72522/root (readlink: Permission denied) 2024-05-10T04:42:31.743 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72522 ms_local ceph txt unknown /proc/68789/task/72522/exe (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72522 ms_local ceph NOFD /proc/68789/task/72522/fd (opendir: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72523 fn_anonym ceph cwd unknown /proc/68789/task/72523/cwd (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72523 fn_anonym ceph rtd unknown /proc/68789/task/72523/root (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72523 fn_anonym ceph txt unknown /proc/68789/task/72523/exe (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72523 fn_anonym ceph NOFD /proc/68789/task/72523/fd (opendir: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72524 finisher ceph cwd unknown /proc/68789/task/72524/cwd (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72524 finisher ceph rtd unknown /proc/68789/task/72524/root (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72524 finisher ceph txt unknown /proc/68789/task/72524/exe (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72524 finisher ceph NOFD /proc/68789/task/72524/fd (opendir: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72525 safe_time ceph cwd unknown /proc/68789/task/72525/cwd (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72525 safe_time ceph rtd unknown /proc/68789/task/72525/root (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72525 safe_time ceph txt unknown /proc/68789/task/72525/exe (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72525 safe_time ceph NOFD /proc/68789/task/72525/fd (opendir: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72527 safe_time ceph cwd unknown /proc/68789/task/72527/cwd (readlink: Permission denied) 2024-05-10T04:42:31.744 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72527 safe_time ceph rtd unknown /proc/68789/task/72527/root (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72527 safe_time ceph txt unknown /proc/68789/task/72527/exe (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72527 safe_time ceph NOFD /proc/68789/task/72527/fd (opendir: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72528 ceph-osd ceph cwd unknown /proc/68789/task/72528/cwd (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72528 ceph-osd ceph rtd unknown /proc/68789/task/72528/root (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72528 ceph-osd ceph txt unknown /proc/68789/task/72528/exe (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72528 ceph-osd ceph NOFD /proc/68789/task/72528/fd (opendir: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72529 osd_srv_a ceph cwd unknown /proc/68789/task/72529/cwd (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72529 osd_srv_a ceph rtd unknown /proc/68789/task/72529/root (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72529 osd_srv_a ceph txt unknown /proc/68789/task/72529/exe (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72529 osd_srv_a ceph NOFD /proc/68789/task/72529/fd (opendir: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72530 tp_osd_tp ceph cwd unknown /proc/68789/task/72530/cwd (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72530 tp_osd_tp ceph rtd unknown /proc/68789/task/72530/root (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72530 tp_osd_tp ceph txt unknown /proc/68789/task/72530/exe (readlink: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72530 tp_osd_tp ceph NOFD /proc/68789/task/72530/fd (opendir: Permission denied) 2024-05-10T04:42:31.745 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72531 tp_osd_tp ceph cwd unknown /proc/68789/task/72531/cwd (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72531 tp_osd_tp ceph rtd unknown /proc/68789/task/72531/root (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72531 tp_osd_tp ceph txt unknown /proc/68789/task/72531/exe (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72531 tp_osd_tp ceph NOFD /proc/68789/task/72531/fd (opendir: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72533 tp_osd_tp ceph cwd unknown /proc/68789/task/72533/cwd (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72533 tp_osd_tp ceph rtd unknown /proc/68789/task/72533/root (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72533 tp_osd_tp ceph txt unknown /proc/68789/task/72533/exe (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72533 tp_osd_tp ceph NOFD /proc/68789/task/72533/fd (opendir: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72534 tp_osd_tp ceph cwd unknown /proc/68789/task/72534/cwd (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72534 tp_osd_tp ceph rtd unknown /proc/68789/task/72534/root (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72534 tp_osd_tp ceph txt unknown /proc/68789/task/72534/exe (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72534 tp_osd_tp ceph NOFD /proc/68789/task/72534/fd (opendir: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72535 tp_osd_tp ceph cwd unknown /proc/68789/task/72535/cwd (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72535 tp_osd_tp ceph rtd unknown /proc/68789/task/72535/root (readlink: Permission denied) 2024-05-10T04:42:31.746 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72535 tp_osd_tp ceph txt unknown /proc/68789/task/72535/exe (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72535 tp_osd_tp ceph NOFD /proc/68789/task/72535/fd (opendir: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72536 tp_osd_tp ceph cwd unknown /proc/68789/task/72536/cwd (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72536 tp_osd_tp ceph rtd unknown /proc/68789/task/72536/root (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72536 tp_osd_tp ceph txt unknown /proc/68789/task/72536/exe (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72536 tp_osd_tp ceph NOFD /proc/68789/task/72536/fd (opendir: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72537 tp_osd_tp ceph cwd unknown /proc/68789/task/72537/cwd (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72537 tp_osd_tp ceph rtd unknown /proc/68789/task/72537/root (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72537 tp_osd_tp ceph txt unknown /proc/68789/task/72537/exe (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72537 tp_osd_tp ceph NOFD /proc/68789/task/72537/fd (opendir: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72538 tp_osd_tp ceph cwd unknown /proc/68789/task/72538/cwd (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72538 tp_osd_tp ceph rtd unknown /proc/68789/task/72538/root (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72538 tp_osd_tp ceph txt unknown /proc/68789/task/72538/exe (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72538 tp_osd_tp ceph NOFD /proc/68789/task/72538/fd (opendir: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72540 tp_osd_tp ceph cwd unknown /proc/68789/task/72540/cwd (readlink: Permission denied) 2024-05-10T04:42:31.747 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72540 tp_osd_tp ceph rtd unknown /proc/68789/task/72540/root (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72540 tp_osd_tp ceph txt unknown /proc/68789/task/72540/exe (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72540 tp_osd_tp ceph NOFD /proc/68789/task/72540/fd (opendir: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72541 tp_osd_tp ceph cwd unknown /proc/68789/task/72541/cwd (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72541 tp_osd_tp ceph rtd unknown /proc/68789/task/72541/root (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72541 tp_osd_tp ceph txt unknown /proc/68789/task/72541/exe (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72541 tp_osd_tp ceph NOFD /proc/68789/task/72541/fd (opendir: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72542 tp_osd_tp ceph cwd unknown /proc/68789/task/72542/cwd (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72542 tp_osd_tp ceph rtd unknown /proc/68789/task/72542/root (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72542 tp_osd_tp ceph txt unknown /proc/68789/task/72542/exe (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72542 tp_osd_tp ceph NOFD /proc/68789/task/72542/fd (opendir: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72543 tp_osd_tp ceph cwd unknown /proc/68789/task/72543/cwd (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72543 tp_osd_tp ceph rtd unknown /proc/68789/task/72543/root (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72543 tp_osd_tp ceph txt unknown /proc/68789/task/72543/exe (readlink: Permission denied) 2024-05-10T04:42:31.748 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72543 tp_osd_tp ceph NOFD /proc/68789/task/72543/fd (opendir: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72544 tp_osd_tp ceph cwd unknown /proc/68789/task/72544/cwd (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72544 tp_osd_tp ceph rtd unknown /proc/68789/task/72544/root (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72544 tp_osd_tp ceph txt unknown /proc/68789/task/72544/exe (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72544 tp_osd_tp ceph NOFD /proc/68789/task/72544/fd (opendir: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72546 tp_osd_tp ceph cwd unknown /proc/68789/task/72546/cwd (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72546 tp_osd_tp ceph rtd unknown /proc/68789/task/72546/root (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72546 tp_osd_tp ceph txt unknown /proc/68789/task/72546/exe (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72546 tp_osd_tp ceph NOFD /proc/68789/task/72546/fd (opendir: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72547 tp_osd_tp ceph cwd unknown /proc/68789/task/72547/cwd (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72547 tp_osd_tp ceph rtd unknown /proc/68789/task/72547/root (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72547 tp_osd_tp ceph txt unknown /proc/68789/task/72547/exe (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72547 tp_osd_tp ceph NOFD /proc/68789/task/72547/fd (opendir: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72548 tp_osd_tp ceph cwd unknown /proc/68789/task/72548/cwd (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72548 tp_osd_tp ceph rtd unknown /proc/68789/task/72548/root (readlink: Permission denied) 2024-05-10T04:42:31.749 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72548 tp_osd_tp ceph txt unknown /proc/68789/task/72548/exe (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72548 tp_osd_tp ceph NOFD /proc/68789/task/72548/fd (opendir: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72549 osd_srv_h ceph cwd unknown /proc/68789/task/72549/cwd (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72549 osd_srv_h ceph rtd unknown /proc/68789/task/72549/root (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72549 osd_srv_h ceph txt unknown /proc/68789/task/72549/exe (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 68789 72549 osd_srv_h ceph NOFD /proc/68789/task/72549/fd (opendir: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 69465 root cwd unknown /proc/69465/cwd (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 69465 root rtd unknown /proc/69465/root (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 69465 root txt unknown /proc/69465/exe (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 69465 root NOFD /proc/69465/fd (opendir: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 69497 root cwd unknown /proc/69497/cwd (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 69497 root rtd unknown /proc/69497/root (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 69497 root txt unknown /proc/69497/exe (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 69497 root NOFD /proc/69497/fd (opendir: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 73395 root cwd unknown /proc/73395/cwd (readlink: Permission denied) 2024-05-10T04:42:31.750 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 73395 root rtd unknown /proc/73395/root (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 73395 root txt unknown /proc/73395/exe (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/6 73395 root NOFD /proc/73395/fd (opendir: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74330 root cwd unknown /proc/74330/cwd (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74330 root rtd unknown /proc/74330/root (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74330 root txt unknown /proc/74330/exe (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74330 root NOFD /proc/74330/fd (opendir: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74331 root cwd unknown /proc/74331/cwd (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74331 root rtd unknown /proc/74331/root (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74331 root txt unknown /proc/74331/exe (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/7 74331 root NOFD /proc/74331/fd (opendir: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 76714 root cwd unknown /proc/76714/cwd (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 76714 root rtd unknown /proc/76714/root (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 76714 root txt unknown /proc/76714/exe (readlink: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 76714 root NOFD /proc/76714/fd (opendir: Permission denied) 2024-05-10T04:42:31.751 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 76725 root cwd unknown /proc/76725/cwd (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 76725 root rtd unknown /proc/76725/root (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 76725 root txt unknown /proc/76725/exe (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 76725 root NOFD /proc/76725/fd (opendir: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 ceph cwd unknown /proc/76737/cwd (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 ceph rtd unknown /proc/76737/root (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 ceph txt unknown /proc/76737/exe (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 ceph NOFD /proc/76737/fd (opendir: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76740 log ceph cwd unknown /proc/76737/task/76740/cwd (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76740 log ceph rtd unknown /proc/76737/task/76740/root (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76740 log ceph txt unknown /proc/76737/task/76740/exe (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76740 log ceph NOFD /proc/76737/task/76740/fd (opendir: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76742 msgr-work ceph cwd unknown /proc/76737/task/76742/cwd (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76742 msgr-work ceph rtd unknown /proc/76737/task/76742/root (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76742 msgr-work ceph txt unknown /proc/76737/task/76742/exe (readlink: Permission denied) 2024-05-10T04:42:31.752 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76742 msgr-work ceph NOFD /proc/76737/task/76742/fd (opendir: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76743 msgr-work ceph cwd unknown /proc/76737/task/76743/cwd (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76743 msgr-work ceph rtd unknown /proc/76737/task/76743/root (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76743 msgr-work ceph txt unknown /proc/76737/task/76743/exe (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76743 msgr-work ceph NOFD /proc/76737/task/76743/fd (opendir: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76744 msgr-work ceph cwd unknown /proc/76737/task/76744/cwd (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76744 msgr-work ceph rtd unknown /proc/76737/task/76744/root (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76744 msgr-work ceph txt unknown /proc/76737/task/76744/exe (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76744 msgr-work ceph NOFD /proc/76737/task/76744/fd (opendir: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76748 service ceph cwd unknown /proc/76737/task/76748/cwd (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76748 service ceph rtd unknown /proc/76737/task/76748/root (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76748 service ceph txt unknown /proc/76737/task/76748/exe (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76748 service ceph NOFD /proc/76737/task/76748/fd (opendir: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76749 admin_soc ceph cwd unknown /proc/76737/task/76749/cwd (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76749 admin_soc ceph rtd unknown /proc/76737/task/76749/root (readlink: Permission denied) 2024-05-10T04:42:31.753 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76749 admin_soc ceph txt unknown /proc/76737/task/76749/exe (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 76749 admin_soc ceph NOFD /proc/76737/task/76749/fd (opendir: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77038 signal_ha ceph cwd unknown /proc/76737/task/77038/cwd (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77038 signal_ha ceph rtd unknown /proc/76737/task/77038/root (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77038 signal_ha ceph txt unknown /proc/76737/task/77038/exe (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77038 signal_ha ceph NOFD /proc/76737/task/77038/fd (opendir: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77039 io_contex ceph cwd unknown /proc/76737/task/77039/cwd (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77039 io_contex ceph rtd unknown /proc/76737/task/77039/root (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77039 io_contex ceph txt unknown /proc/76737/task/77039/exe (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77039 io_contex ceph NOFD /proc/76737/task/77039/fd (opendir: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77040 io_contex ceph cwd unknown /proc/76737/task/77040/cwd (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77040 io_contex ceph rtd unknown /proc/76737/task/77040/root (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77040 io_contex ceph txt unknown /proc/76737/task/77040/exe (readlink: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77040 io_contex ceph NOFD /proc/76737/task/77040/fd (opendir: Permission denied) 2024-05-10T04:42:31.754 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77103 OpHistory ceph cwd unknown /proc/76737/task/77103/cwd (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77103 OpHistory ceph rtd unknown /proc/76737/task/77103/root (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77103 OpHistory ceph txt unknown /proc/76737/task/77103/exe (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77103 OpHistory ceph NOFD /proc/76737/task/77103/fd (opendir: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77104 ceph_time ceph cwd unknown /proc/76737/task/77104/cwd (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77104 ceph_time ceph rtd unknown /proc/76737/task/77104/root (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77104 ceph_time ceph txt unknown /proc/76737/task/77104/exe (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77104 ceph_time ceph NOFD /proc/76737/task/77104/fd (opendir: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77512 ceph-osd ceph cwd unknown /proc/76737/task/77512/cwd (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77512 ceph-osd ceph rtd unknown /proc/76737/task/77512/root (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77512 ceph-osd ceph txt unknown /proc/76737/task/77512/exe (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77512 ceph-osd ceph NOFD /proc/76737/task/77512/fd (opendir: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77513 safe_time ceph cwd unknown /proc/76737/task/77513/cwd (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77513 safe_time ceph rtd unknown /proc/76737/task/77513/root (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77513 safe_time ceph txt unknown /proc/76737/task/77513/exe (readlink: Permission denied) 2024-05-10T04:42:31.755 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77513 safe_time ceph NOFD /proc/76737/task/77513/fd (opendir: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77515 safe_time ceph cwd unknown /proc/76737/task/77515/cwd (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77515 safe_time ceph rtd unknown /proc/76737/task/77515/root (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77515 safe_time ceph txt unknown /proc/76737/task/77515/exe (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77515 safe_time ceph NOFD /proc/76737/task/77515/fd (opendir: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77516 safe_time ceph cwd unknown /proc/76737/task/77516/cwd (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77516 safe_time ceph rtd unknown /proc/76737/task/77516/root (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77516 safe_time ceph txt unknown /proc/76737/task/77516/exe (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77516 safe_time ceph NOFD /proc/76737/task/77516/fd (opendir: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77517 safe_time ceph cwd unknown /proc/76737/task/77517/cwd (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77517 safe_time ceph rtd unknown /proc/76737/task/77517/root (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77517 safe_time ceph txt unknown /proc/76737/task/77517/exe (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77517 safe_time ceph NOFD /proc/76737/task/77517/fd (opendir: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77518 fn_anonym ceph cwd unknown /proc/76737/task/77518/cwd (readlink: Permission denied) 2024-05-10T04:42:31.756 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77518 fn_anonym ceph rtd unknown /proc/76737/task/77518/root (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77518 fn_anonym ceph txt unknown /proc/76737/task/77518/exe (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77518 fn_anonym ceph NOFD /proc/76737/task/77518/fd (opendir: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77522 rocksdb:l ceph cwd unknown /proc/76737/task/77522/cwd (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77522 rocksdb:l ceph rtd unknown /proc/76737/task/77522/root (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77522 rocksdb:l ceph txt unknown /proc/76737/task/77522/exe (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77522 rocksdb:l ceph NOFD /proc/76737/task/77522/fd (opendir: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77523 rocksdb:l ceph cwd unknown /proc/76737/task/77523/cwd (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77523 rocksdb:l ceph rtd unknown /proc/76737/task/77523/root (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77523 rocksdb:l ceph txt unknown /proc/76737/task/77523/exe (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77523 rocksdb:l ceph NOFD /proc/76737/task/77523/fd (opendir: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77524 rocksdb:l ceph cwd unknown /proc/76737/task/77524/cwd (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77524 rocksdb:l ceph rtd unknown /proc/76737/task/77524/root (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77524 rocksdb:l ceph txt unknown /proc/76737/task/77524/exe (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77524 rocksdb:l ceph NOFD /proc/76737/task/77524/fd (opendir: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77525 rocksdb:h ceph cwd unknown /proc/76737/task/77525/cwd (readlink: Permission denied) 2024-05-10T04:42:31.757 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77525 rocksdb:h ceph rtd unknown /proc/76737/task/77525/root (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77525 rocksdb:h ceph txt unknown /proc/76737/task/77525/exe (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77525 rocksdb:h ceph NOFD /proc/76737/task/77525/fd (opendir: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77973 bstore_ai ceph cwd unknown /proc/76737/task/77973/cwd (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77973 bstore_ai ceph rtd unknown /proc/76737/task/77973/root (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77973 bstore_ai ceph txt unknown /proc/76737/task/77973/exe (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 77973 bstore_ai ceph NOFD /proc/76737/task/77973/fd (opendir: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78181 bstore_ai ceph cwd unknown /proc/76737/task/78181/cwd (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78181 bstore_ai ceph rtd unknown /proc/76737/task/78181/root (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78181 bstore_ai ceph txt unknown /proc/76737/task/78181/exe (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78181 bstore_ai ceph NOFD /proc/76737/task/78181/fd (opendir: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78365 ceph-osd ceph cwd unknown /proc/76737/task/78365/cwd (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78365 ceph-osd ceph rtd unknown /proc/76737/task/78365/root (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78365 ceph-osd ceph txt unknown /proc/76737/task/78365/exe (readlink: Permission denied) 2024-05-10T04:42:31.758 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78365 ceph-osd ceph NOFD /proc/76737/task/78365/fd (opendir: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78366 cfin ceph cwd unknown /proc/76737/task/78366/cwd (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78366 cfin ceph rtd unknown /proc/76737/task/78366/root (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78366 cfin ceph txt unknown /proc/76737/task/78366/exe (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78366 cfin ceph NOFD /proc/76737/task/78366/fd (opendir: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78367 bstore_kv ceph cwd unknown /proc/76737/task/78367/cwd (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78367 bstore_kv ceph rtd unknown /proc/76737/task/78367/root (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78367 bstore_kv ceph txt unknown /proc/76737/task/78367/exe (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78367 bstore_kv ceph NOFD /proc/76737/task/78367/fd (opendir: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78368 bstore_kv ceph cwd unknown /proc/76737/task/78368/cwd (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78368 bstore_kv ceph rtd unknown /proc/76737/task/78368/root (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78368 bstore_kv ceph txt unknown /proc/76737/task/78368/exe (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78368 bstore_kv ceph NOFD /proc/76737/task/78368/fd (opendir: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78370 bstore_me ceph cwd unknown /proc/76737/task/78370/cwd (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78370 bstore_me ceph rtd unknown /proc/76737/task/78370/root (readlink: Permission denied) 2024-05-10T04:42:31.759 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78370 bstore_me ceph txt unknown /proc/76737/task/78370/exe (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78370 bstore_me ceph NOFD /proc/76737/task/78370/fd (opendir: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78373 ms_dispat ceph cwd unknown /proc/76737/task/78373/cwd (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78373 ms_dispat ceph rtd unknown /proc/76737/task/78373/root (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78373 ms_dispat ceph txt unknown /proc/76737/task/78373/exe (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78373 ms_dispat ceph NOFD /proc/76737/task/78373/fd (opendir: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78374 ms_local ceph cwd unknown /proc/76737/task/78374/cwd (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78374 ms_local ceph rtd unknown /proc/76737/task/78374/root (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78374 ms_local ceph txt unknown /proc/76737/task/78374/exe (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78374 ms_local ceph NOFD /proc/76737/task/78374/fd (opendir: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78375 safe_time ceph cwd unknown /proc/76737/task/78375/cwd (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78375 safe_time ceph rtd unknown /proc/76737/task/78375/root (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78375 safe_time ceph txt unknown /proc/76737/task/78375/exe (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78375 safe_time ceph NOFD /proc/76737/task/78375/fd (opendir: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78376 safe_time ceph cwd unknown /proc/76737/task/78376/cwd (readlink: Permission denied) 2024-05-10T04:42:31.760 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78376 safe_time ceph rtd unknown /proc/76737/task/78376/root (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78376 safe_time ceph txt unknown /proc/76737/task/78376/exe (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78376 safe_time ceph NOFD /proc/76737/task/78376/fd (opendir: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78377 ms_dispat ceph cwd unknown /proc/76737/task/78377/cwd (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78377 ms_dispat ceph rtd unknown /proc/76737/task/78377/root (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78377 ms_dispat ceph txt unknown /proc/76737/task/78377/exe (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78377 ms_dispat ceph NOFD /proc/76737/task/78377/fd (opendir: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78378 ms_local ceph cwd unknown /proc/76737/task/78378/cwd (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78378 ms_local ceph rtd unknown /proc/76737/task/78378/root (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78378 ms_local ceph txt unknown /proc/76737/task/78378/exe (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78378 ms_local ceph NOFD /proc/76737/task/78378/fd (opendir: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78379 ms_dispat ceph cwd unknown /proc/76737/task/78379/cwd (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78379 ms_dispat ceph rtd unknown /proc/76737/task/78379/root (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78379 ms_dispat ceph txt unknown /proc/76737/task/78379/exe (readlink: Permission denied) 2024-05-10T04:42:31.761 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78379 ms_dispat ceph NOFD /proc/76737/task/78379/fd (opendir: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78380 ms_local ceph cwd unknown /proc/76737/task/78380/cwd (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78380 ms_local ceph rtd unknown /proc/76737/task/78380/root (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78380 ms_local ceph txt unknown /proc/76737/task/78380/exe (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78380 ms_local ceph NOFD /proc/76737/task/78380/fd (opendir: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78381 ms_dispat ceph cwd unknown /proc/76737/task/78381/cwd (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78381 ms_dispat ceph rtd unknown /proc/76737/task/78381/root (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78381 ms_dispat ceph txt unknown /proc/76737/task/78381/exe (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78381 ms_dispat ceph NOFD /proc/76737/task/78381/fd (opendir: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78382 ms_local ceph cwd unknown /proc/76737/task/78382/cwd (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78382 ms_local ceph rtd unknown /proc/76737/task/78382/root (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78382 ms_local ceph txt unknown /proc/76737/task/78382/exe (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78382 ms_local ceph NOFD /proc/76737/task/78382/fd (opendir: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78383 ms_dispat ceph cwd unknown /proc/76737/task/78383/cwd (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78383 ms_dispat ceph rtd unknown /proc/76737/task/78383/root (readlink: Permission denied) 2024-05-10T04:42:31.762 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78383 ms_dispat ceph txt unknown /proc/76737/task/78383/exe (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78383 ms_dispat ceph NOFD /proc/76737/task/78383/fd (opendir: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78384 ms_local ceph cwd unknown /proc/76737/task/78384/cwd (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78384 ms_local ceph rtd unknown /proc/76737/task/78384/root (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78384 ms_local ceph txt unknown /proc/76737/task/78384/exe (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78384 ms_local ceph NOFD /proc/76737/task/78384/fd (opendir: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78385 ms_dispat ceph cwd unknown /proc/76737/task/78385/cwd (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78385 ms_dispat ceph rtd unknown /proc/76737/task/78385/root (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78385 ms_dispat ceph txt unknown /proc/76737/task/78385/exe (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78385 ms_dispat ceph NOFD /proc/76737/task/78385/fd (opendir: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78386 ms_local ceph cwd unknown /proc/76737/task/78386/cwd (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78386 ms_local ceph rtd unknown /proc/76737/task/78386/root (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78386 ms_local ceph txt unknown /proc/76737/task/78386/exe (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78386 ms_local ceph NOFD /proc/76737/task/78386/fd (opendir: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78387 ms_dispat ceph cwd unknown /proc/76737/task/78387/cwd (readlink: Permission denied) 2024-05-10T04:42:31.763 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78387 ms_dispat ceph rtd unknown /proc/76737/task/78387/root (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78387 ms_dispat ceph txt unknown /proc/76737/task/78387/exe (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78387 ms_dispat ceph NOFD /proc/76737/task/78387/fd (opendir: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78388 ms_local ceph cwd unknown /proc/76737/task/78388/cwd (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78388 ms_local ceph rtd unknown /proc/76737/task/78388/root (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78388 ms_local ceph txt unknown /proc/76737/task/78388/exe (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78388 ms_local ceph NOFD /proc/76737/task/78388/fd (opendir: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78389 fn_anonym ceph cwd unknown /proc/76737/task/78389/cwd (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78389 fn_anonym ceph rtd unknown /proc/76737/task/78389/root (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78389 fn_anonym ceph txt unknown /proc/76737/task/78389/exe (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78389 fn_anonym ceph NOFD /proc/76737/task/78389/fd (opendir: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78390 finisher ceph cwd unknown /proc/76737/task/78390/cwd (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78390 finisher ceph rtd unknown /proc/76737/task/78390/root (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78390 finisher ceph txt unknown /proc/76737/task/78390/exe (readlink: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78390 finisher ceph NOFD /proc/76737/task/78390/fd (opendir: Permission denied) 2024-05-10T04:42:31.764 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78391 safe_time ceph cwd unknown /proc/76737/task/78391/cwd (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78391 safe_time ceph rtd unknown /proc/76737/task/78391/root (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78391 safe_time ceph txt unknown /proc/76737/task/78391/exe (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78391 safe_time ceph NOFD /proc/76737/task/78391/fd (opendir: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78392 safe_time ceph cwd unknown /proc/76737/task/78392/cwd (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78392 safe_time ceph rtd unknown /proc/76737/task/78392/root (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78392 safe_time ceph txt unknown /proc/76737/task/78392/exe (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78392 safe_time ceph NOFD /proc/76737/task/78392/fd (opendir: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78393 ceph-osd ceph cwd unknown /proc/76737/task/78393/cwd (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78393 ceph-osd ceph rtd unknown /proc/76737/task/78393/root (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78393 ceph-osd ceph txt unknown /proc/76737/task/78393/exe (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78393 ceph-osd ceph NOFD /proc/76737/task/78393/fd (opendir: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78394 osd_srv_a ceph cwd unknown /proc/76737/task/78394/cwd (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78394 osd_srv_a ceph rtd unknown /proc/76737/task/78394/root (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78394 osd_srv_a ceph txt unknown /proc/76737/task/78394/exe (readlink: Permission denied) 2024-05-10T04:42:31.765 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78394 osd_srv_a ceph NOFD /proc/76737/task/78394/fd (opendir: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78395 tp_osd_tp ceph cwd unknown /proc/76737/task/78395/cwd (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78395 tp_osd_tp ceph rtd unknown /proc/76737/task/78395/root (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78395 tp_osd_tp ceph txt unknown /proc/76737/task/78395/exe (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78395 tp_osd_tp ceph NOFD /proc/76737/task/78395/fd (opendir: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78396 tp_osd_tp ceph cwd unknown /proc/76737/task/78396/cwd (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78396 tp_osd_tp ceph rtd unknown /proc/76737/task/78396/root (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78396 tp_osd_tp ceph txt unknown /proc/76737/task/78396/exe (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78396 tp_osd_tp ceph NOFD /proc/76737/task/78396/fd (opendir: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78397 tp_osd_tp ceph cwd unknown /proc/76737/task/78397/cwd (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78397 tp_osd_tp ceph rtd unknown /proc/76737/task/78397/root (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78397 tp_osd_tp ceph txt unknown /proc/76737/task/78397/exe (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78397 tp_osd_tp ceph NOFD /proc/76737/task/78397/fd (opendir: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78398 tp_osd_tp ceph cwd unknown /proc/76737/task/78398/cwd (readlink: Permission denied) 2024-05-10T04:42:31.766 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78398 tp_osd_tp ceph rtd unknown /proc/76737/task/78398/root (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78398 tp_osd_tp ceph txt unknown /proc/76737/task/78398/exe (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78398 tp_osd_tp ceph NOFD /proc/76737/task/78398/fd (opendir: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78399 tp_osd_tp ceph cwd unknown /proc/76737/task/78399/cwd (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78399 tp_osd_tp ceph rtd unknown /proc/76737/task/78399/root (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78399 tp_osd_tp ceph txt unknown /proc/76737/task/78399/exe (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78399 tp_osd_tp ceph NOFD /proc/76737/task/78399/fd (opendir: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78400 tp_osd_tp ceph cwd unknown /proc/76737/task/78400/cwd (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78400 tp_osd_tp ceph rtd unknown /proc/76737/task/78400/root (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78400 tp_osd_tp ceph txt unknown /proc/76737/task/78400/exe (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78400 tp_osd_tp ceph NOFD /proc/76737/task/78400/fd (opendir: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78401 tp_osd_tp ceph cwd unknown /proc/76737/task/78401/cwd (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78401 tp_osd_tp ceph rtd unknown /proc/76737/task/78401/root (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78401 tp_osd_tp ceph txt unknown /proc/76737/task/78401/exe (readlink: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78401 tp_osd_tp ceph NOFD /proc/76737/task/78401/fd (opendir: Permission denied) 2024-05-10T04:42:31.767 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78402 tp_osd_tp ceph cwd unknown /proc/76737/task/78402/cwd (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78402 tp_osd_tp ceph rtd unknown /proc/76737/task/78402/root (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78402 tp_osd_tp ceph txt unknown /proc/76737/task/78402/exe (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78402 tp_osd_tp ceph NOFD /proc/76737/task/78402/fd (opendir: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78403 tp_osd_tp ceph cwd unknown /proc/76737/task/78403/cwd (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78403 tp_osd_tp ceph rtd unknown /proc/76737/task/78403/root (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78403 tp_osd_tp ceph txt unknown /proc/76737/task/78403/exe (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78403 tp_osd_tp ceph NOFD /proc/76737/task/78403/fd (opendir: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78404 tp_osd_tp ceph cwd unknown /proc/76737/task/78404/cwd (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78404 tp_osd_tp ceph rtd unknown /proc/76737/task/78404/root (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78404 tp_osd_tp ceph txt unknown /proc/76737/task/78404/exe (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78404 tp_osd_tp ceph NOFD /proc/76737/task/78404/fd (opendir: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78405 tp_osd_tp ceph cwd unknown /proc/76737/task/78405/cwd (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78405 tp_osd_tp ceph rtd unknown /proc/76737/task/78405/root (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78405 tp_osd_tp ceph txt unknown /proc/76737/task/78405/exe (readlink: Permission denied) 2024-05-10T04:42:31.768 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78405 tp_osd_tp ceph NOFD /proc/76737/task/78405/fd (opendir: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78406 tp_osd_tp ceph cwd unknown /proc/76737/task/78406/cwd (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78406 tp_osd_tp ceph rtd unknown /proc/76737/task/78406/root (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78406 tp_osd_tp ceph txt unknown /proc/76737/task/78406/exe (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78406 tp_osd_tp ceph NOFD /proc/76737/task/78406/fd (opendir: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78407 tp_osd_tp ceph cwd unknown /proc/76737/task/78407/cwd (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78407 tp_osd_tp ceph rtd unknown /proc/76737/task/78407/root (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78407 tp_osd_tp ceph txt unknown /proc/76737/task/78407/exe (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78407 tp_osd_tp ceph NOFD /proc/76737/task/78407/fd (opendir: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78408 tp_osd_tp ceph cwd unknown /proc/76737/task/78408/cwd (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78408 tp_osd_tp ceph rtd unknown /proc/76737/task/78408/root (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78408 tp_osd_tp ceph txt unknown /proc/76737/task/78408/exe (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78408 tp_osd_tp ceph NOFD /proc/76737/task/78408/fd (opendir: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78409 tp_osd_tp ceph cwd unknown /proc/76737/task/78409/cwd (readlink: Permission denied) 2024-05-10T04:42:31.769 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78409 tp_osd_tp ceph rtd unknown /proc/76737/task/78409/root (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78409 tp_osd_tp ceph txt unknown /proc/76737/task/78409/exe (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78409 tp_osd_tp ceph NOFD /proc/76737/task/78409/fd (opendir: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78410 tp_osd_tp ceph cwd unknown /proc/76737/task/78410/cwd (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78410 tp_osd_tp ceph rtd unknown /proc/76737/task/78410/root (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78410 tp_osd_tp ceph txt unknown /proc/76737/task/78410/exe (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78410 tp_osd_tp ceph NOFD /proc/76737/task/78410/fd (opendir: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78411 osd_srv_h ceph cwd unknown /proc/76737/task/78411/cwd (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78411 osd_srv_h ceph rtd unknown /proc/76737/task/78411/root (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78411 osd_srv_h ceph txt unknown /proc/76737/task/78411/exe (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 76737 78411 osd_srv_h ceph NOFD /proc/76737/task/78411/fd (opendir: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 77471 root cwd unknown /proc/77471/cwd (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 77471 root rtd unknown /proc/77471/root (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 77471 root txt unknown /proc/77471/exe (readlink: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 77471 root NOFD /proc/77471/fd (opendir: Permission denied) 2024-05-10T04:42:31.770 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 77491 root cwd unknown /proc/77491/cwd (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 77491 root rtd unknown /proc/77491/root (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 77491 root txt unknown /proc/77491/exe (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 77491 root NOFD /proc/77491/fd (opendir: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 82505 root cwd unknown /proc/82505/cwd (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 82505 root rtd unknown /proc/82505/root (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 82505 root txt unknown /proc/82505/exe (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 82505 root NOFD /proc/82505/fd (opendir: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 84528 root cwd unknown /proc/84528/cwd (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 84528 root rtd unknown /proc/84528/root (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 84528 root txt unknown /proc/84528/exe (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/2 84528 root NOFD /proc/84528/fd (opendir: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 85428 root cwd unknown /proc/85428/cwd (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 85428 root rtd unknown /proc/85428/root (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 85428 root txt unknown /proc/85428/exe (readlink: Permission denied) 2024-05-10T04:42:31.771 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 85428 root NOFD /proc/85428/fd (opendir: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 85439 root cwd unknown /proc/85439/cwd (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 85439 root rtd unknown /proc/85439/root (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 85439 root txt unknown /proc/85439/exe (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 85439 root NOFD /proc/85439/fd (opendir: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 ceph cwd unknown /proc/85450/cwd (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 ceph rtd unknown /proc/85450/root (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 ceph txt unknown /proc/85450/exe (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 ceph NOFD /proc/85450/fd (opendir: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85453 log ceph cwd unknown /proc/85450/task/85453/cwd (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85453 log ceph rtd unknown /proc/85450/task/85453/root (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85453 log ceph txt unknown /proc/85450/task/85453/exe (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85453 log ceph NOFD /proc/85450/task/85453/fd (opendir: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85455 msgr-work ceph cwd unknown /proc/85450/task/85455/cwd (readlink: Permission denied) 2024-05-10T04:42:31.772 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85455 msgr-work ceph rtd unknown /proc/85450/task/85455/root (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85455 msgr-work ceph txt unknown /proc/85450/task/85455/exe (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85455 msgr-work ceph NOFD /proc/85450/task/85455/fd (opendir: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85456 msgr-work ceph cwd unknown /proc/85450/task/85456/cwd (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85456 msgr-work ceph rtd unknown /proc/85450/task/85456/root (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85456 msgr-work ceph txt unknown /proc/85450/task/85456/exe (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85456 msgr-work ceph NOFD /proc/85450/task/85456/fd (opendir: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85457 msgr-work ceph cwd unknown /proc/85450/task/85457/cwd (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85457 msgr-work ceph rtd unknown /proc/85450/task/85457/root (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85457 msgr-work ceph txt unknown /proc/85450/task/85457/exe (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85457 msgr-work ceph NOFD /proc/85450/task/85457/fd (opendir: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85461 service ceph cwd unknown /proc/85450/task/85461/cwd (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85461 service ceph rtd unknown /proc/85450/task/85461/root (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85461 service ceph txt unknown /proc/85450/task/85461/exe (readlink: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85461 service ceph NOFD /proc/85450/task/85461/fd (opendir: Permission denied) 2024-05-10T04:42:31.773 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85462 admin_soc ceph cwd unknown /proc/85450/task/85462/cwd (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85462 admin_soc ceph rtd unknown /proc/85450/task/85462/root (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85462 admin_soc ceph txt unknown /proc/85450/task/85462/exe (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85462 admin_soc ceph NOFD /proc/85450/task/85462/fd (opendir: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85727 signal_ha ceph cwd unknown /proc/85450/task/85727/cwd (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85727 signal_ha ceph rtd unknown /proc/85450/task/85727/root (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85727 signal_ha ceph txt unknown /proc/85450/task/85727/exe (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85727 signal_ha ceph NOFD /proc/85450/task/85727/fd (opendir: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85728 io_contex ceph cwd unknown /proc/85450/task/85728/cwd (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85728 io_contex ceph rtd unknown /proc/85450/task/85728/root (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85728 io_contex ceph txt unknown /proc/85450/task/85728/exe (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85728 io_contex ceph NOFD /proc/85450/task/85728/fd (opendir: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85729 io_contex ceph cwd unknown /proc/85450/task/85729/cwd (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85729 io_contex ceph rtd unknown /proc/85450/task/85729/root (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85729 io_contex ceph txt unknown /proc/85450/task/85729/exe (readlink: Permission denied) 2024-05-10T04:42:31.774 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85729 io_contex ceph NOFD /proc/85450/task/85729/fd (opendir: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85805 OpHistory ceph cwd unknown /proc/85450/task/85805/cwd (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85805 OpHistory ceph rtd unknown /proc/85450/task/85805/root (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85805 OpHistory ceph txt unknown /proc/85450/task/85805/exe (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85805 OpHistory ceph NOFD /proc/85450/task/85805/fd (opendir: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85806 ceph_time ceph cwd unknown /proc/85450/task/85806/cwd (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85806 ceph_time ceph rtd unknown /proc/85450/task/85806/root (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85806 ceph_time ceph txt unknown /proc/85450/task/85806/exe (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 85806 ceph_time ceph NOFD /proc/85450/task/85806/fd (opendir: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86249 ceph-osd ceph cwd unknown /proc/85450/task/86249/cwd (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86249 ceph-osd ceph rtd unknown /proc/85450/task/86249/root (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86249 ceph-osd ceph txt unknown /proc/85450/task/86249/exe (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86249 ceph-osd ceph NOFD /proc/85450/task/86249/fd (opendir: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86250 safe_time ceph cwd unknown /proc/85450/task/86250/cwd (readlink: Permission denied) 2024-05-10T04:42:31.775 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86250 safe_time ceph rtd unknown /proc/85450/task/86250/root (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86250 safe_time ceph txt unknown /proc/85450/task/86250/exe (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86250 safe_time ceph NOFD /proc/85450/task/86250/fd (opendir: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86251 safe_time ceph cwd unknown /proc/85450/task/86251/cwd (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86251 safe_time ceph rtd unknown /proc/85450/task/86251/root (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86251 safe_time ceph txt unknown /proc/85450/task/86251/exe (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86251 safe_time ceph NOFD /proc/85450/task/86251/fd (opendir: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86252 safe_time ceph cwd unknown /proc/85450/task/86252/cwd (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86252 safe_time ceph rtd unknown /proc/85450/task/86252/root (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86252 safe_time ceph txt unknown /proc/85450/task/86252/exe (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86252 safe_time ceph NOFD /proc/85450/task/86252/fd (opendir: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86253 safe_time ceph cwd unknown /proc/85450/task/86253/cwd (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86253 safe_time ceph rtd unknown /proc/85450/task/86253/root (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86253 safe_time ceph txt unknown /proc/85450/task/86253/exe (readlink: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86253 safe_time ceph NOFD /proc/85450/task/86253/fd (opendir: Permission denied) 2024-05-10T04:42:31.776 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86254 fn_anonym ceph cwd unknown /proc/85450/task/86254/cwd (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86254 fn_anonym ceph rtd unknown /proc/85450/task/86254/root (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86254 fn_anonym ceph txt unknown /proc/85450/task/86254/exe (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86254 fn_anonym ceph NOFD /proc/85450/task/86254/fd (opendir: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86258 rocksdb:l ceph cwd unknown /proc/85450/task/86258/cwd (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86258 rocksdb:l ceph rtd unknown /proc/85450/task/86258/root (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86258 rocksdb:l ceph txt unknown /proc/85450/task/86258/exe (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86258 rocksdb:l ceph NOFD /proc/85450/task/86258/fd (opendir: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86259 rocksdb:l ceph cwd unknown /proc/85450/task/86259/cwd (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86259 rocksdb:l ceph rtd unknown /proc/85450/task/86259/root (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86259 rocksdb:l ceph txt unknown /proc/85450/task/86259/exe (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86259 rocksdb:l ceph NOFD /proc/85450/task/86259/fd (opendir: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86260 rocksdb:l ceph cwd unknown /proc/85450/task/86260/cwd (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86260 rocksdb:l ceph rtd unknown /proc/85450/task/86260/root (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86260 rocksdb:l ceph txt unknown /proc/85450/task/86260/exe (readlink: Permission denied) 2024-05-10T04:42:31.777 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86260 rocksdb:l ceph NOFD /proc/85450/task/86260/fd (opendir: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86261 rocksdb:h ceph cwd unknown /proc/85450/task/86261/cwd (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86261 rocksdb:h ceph rtd unknown /proc/85450/task/86261/root (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86261 rocksdb:h ceph txt unknown /proc/85450/task/86261/exe (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86261 rocksdb:h ceph NOFD /proc/85450/task/86261/fd (opendir: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86702 bstore_ai ceph cwd unknown /proc/85450/task/86702/cwd (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86702 bstore_ai ceph rtd unknown /proc/85450/task/86702/root (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86702 bstore_ai ceph txt unknown /proc/85450/task/86702/exe (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86702 bstore_ai ceph NOFD /proc/85450/task/86702/fd (opendir: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86954 bstore_ai ceph cwd unknown /proc/85450/task/86954/cwd (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86954 bstore_ai ceph rtd unknown /proc/85450/task/86954/root (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86954 bstore_ai ceph txt unknown /proc/85450/task/86954/exe (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 86954 bstore_ai ceph NOFD /proc/85450/task/86954/fd (opendir: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87264 ceph-osd ceph cwd unknown /proc/85450/task/87264/cwd (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87264 ceph-osd ceph rtd unknown /proc/85450/task/87264/root (readlink: Permission denied) 2024-05-10T04:42:31.778 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87264 ceph-osd ceph txt unknown /proc/85450/task/87264/exe (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87264 ceph-osd ceph NOFD /proc/85450/task/87264/fd (opendir: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87268 cfin ceph cwd unknown /proc/85450/task/87268/cwd (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87268 cfin ceph rtd unknown /proc/85450/task/87268/root (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87268 cfin ceph txt unknown /proc/85450/task/87268/exe (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87268 cfin ceph NOFD /proc/85450/task/87268/fd (opendir: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87270 bstore_kv ceph cwd unknown /proc/85450/task/87270/cwd (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87270 bstore_kv ceph rtd unknown /proc/85450/task/87270/root (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87270 bstore_kv ceph txt unknown /proc/85450/task/87270/exe (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87270 bstore_kv ceph NOFD /proc/85450/task/87270/fd (opendir: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87271 bstore_kv ceph cwd unknown /proc/85450/task/87271/cwd (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87271 bstore_kv ceph rtd unknown /proc/85450/task/87271/root (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87271 bstore_kv ceph txt unknown /proc/85450/task/87271/exe (readlink: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87271 bstore_kv ceph NOFD /proc/85450/task/87271/fd (opendir: Permission denied) 2024-05-10T04:42:31.779 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87272 bstore_me ceph cwd unknown /proc/85450/task/87272/cwd (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87272 bstore_me ceph rtd unknown /proc/85450/task/87272/root (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87272 bstore_me ceph txt unknown /proc/85450/task/87272/exe (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87272 bstore_me ceph NOFD /proc/85450/task/87272/fd (opendir: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87389 ms_dispat ceph cwd unknown /proc/85450/task/87389/cwd (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87389 ms_dispat ceph rtd unknown /proc/85450/task/87389/root (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87389 ms_dispat ceph txt unknown /proc/85450/task/87389/exe (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87389 ms_dispat ceph NOFD /proc/85450/task/87389/fd (opendir: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87390 ms_local ceph cwd unknown /proc/85450/task/87390/cwd (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87390 ms_local ceph rtd unknown /proc/85450/task/87390/root (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87390 ms_local ceph txt unknown /proc/85450/task/87390/exe (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87390 ms_local ceph NOFD /proc/85450/task/87390/fd (opendir: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87392 safe_time ceph cwd unknown /proc/85450/task/87392/cwd (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87392 safe_time ceph rtd unknown /proc/85450/task/87392/root (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87392 safe_time ceph txt unknown /proc/85450/task/87392/exe (readlink: Permission denied) 2024-05-10T04:42:31.780 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87392 safe_time ceph NOFD /proc/85450/task/87392/fd (opendir: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87393 safe_time ceph cwd unknown /proc/85450/task/87393/cwd (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87393 safe_time ceph rtd unknown /proc/85450/task/87393/root (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87393 safe_time ceph txt unknown /proc/85450/task/87393/exe (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87393 safe_time ceph NOFD /proc/85450/task/87393/fd (opendir: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87394 ms_dispat ceph cwd unknown /proc/85450/task/87394/cwd (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87394 ms_dispat ceph rtd unknown /proc/85450/task/87394/root (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87394 ms_dispat ceph txt unknown /proc/85450/task/87394/exe (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87394 ms_dispat ceph NOFD /proc/85450/task/87394/fd (opendir: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87396 ms_local ceph cwd unknown /proc/85450/task/87396/cwd (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87396 ms_local ceph rtd unknown /proc/85450/task/87396/root (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87396 ms_local ceph txt unknown /proc/85450/task/87396/exe (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87396 ms_local ceph NOFD /proc/85450/task/87396/fd (opendir: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87397 ms_dispat ceph cwd unknown /proc/85450/task/87397/cwd (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87397 ms_dispat ceph rtd unknown /proc/85450/task/87397/root (readlink: Permission denied) 2024-05-10T04:42:31.781 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87397 ms_dispat ceph txt unknown /proc/85450/task/87397/exe (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87397 ms_dispat ceph NOFD /proc/85450/task/87397/fd (opendir: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87398 ms_local ceph cwd unknown /proc/85450/task/87398/cwd (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87398 ms_local ceph rtd unknown /proc/85450/task/87398/root (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87398 ms_local ceph txt unknown /proc/85450/task/87398/exe (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87398 ms_local ceph NOFD /proc/85450/task/87398/fd (opendir: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87400 ms_dispat ceph cwd unknown /proc/85450/task/87400/cwd (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87400 ms_dispat ceph rtd unknown /proc/85450/task/87400/root (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87400 ms_dispat ceph txt unknown /proc/85450/task/87400/exe (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87400 ms_dispat ceph NOFD /proc/85450/task/87400/fd (opendir: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87401 ms_local ceph cwd unknown /proc/85450/task/87401/cwd (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87401 ms_local ceph rtd unknown /proc/85450/task/87401/root (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87401 ms_local ceph txt unknown /proc/85450/task/87401/exe (readlink: Permission denied) 2024-05-10T04:42:31.782 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87401 ms_local ceph NOFD /proc/85450/task/87401/fd (opendir: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87402 ms_dispat ceph cwd unknown /proc/85450/task/87402/cwd (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87402 ms_dispat ceph rtd unknown /proc/85450/task/87402/root (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87402 ms_dispat ceph txt unknown /proc/85450/task/87402/exe (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87402 ms_dispat ceph NOFD /proc/85450/task/87402/fd (opendir: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87404 ms_local ceph cwd unknown /proc/85450/task/87404/cwd (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87404 ms_local ceph rtd unknown /proc/85450/task/87404/root (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87404 ms_local ceph txt unknown /proc/85450/task/87404/exe (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87404 ms_local ceph NOFD /proc/85450/task/87404/fd (opendir: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87405 ms_dispat ceph cwd unknown /proc/85450/task/87405/cwd (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87405 ms_dispat ceph rtd unknown /proc/85450/task/87405/root (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87405 ms_dispat ceph txt unknown /proc/85450/task/87405/exe (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87405 ms_dispat ceph NOFD /proc/85450/task/87405/fd (opendir: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87406 ms_local ceph cwd unknown /proc/85450/task/87406/cwd (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87406 ms_local ceph rtd unknown /proc/85450/task/87406/root (readlink: Permission denied) 2024-05-10T04:42:31.783 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87406 ms_local ceph txt unknown /proc/85450/task/87406/exe (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87406 ms_local ceph NOFD /proc/85450/task/87406/fd (opendir: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87408 ms_dispat ceph cwd unknown /proc/85450/task/87408/cwd (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87408 ms_dispat ceph rtd unknown /proc/85450/task/87408/root (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87408 ms_dispat ceph txt unknown /proc/85450/task/87408/exe (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87408 ms_dispat ceph NOFD /proc/85450/task/87408/fd (opendir: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87409 ms_local ceph cwd unknown /proc/85450/task/87409/cwd (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87409 ms_local ceph rtd unknown /proc/85450/task/87409/root (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87409 ms_local ceph txt unknown /proc/85450/task/87409/exe (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87409 ms_local ceph NOFD /proc/85450/task/87409/fd (opendir: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87410 fn_anonym ceph cwd unknown /proc/85450/task/87410/cwd (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87410 fn_anonym ceph rtd unknown /proc/85450/task/87410/root (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87410 fn_anonym ceph txt unknown /proc/85450/task/87410/exe (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87410 fn_anonym ceph NOFD /proc/85450/task/87410/fd (opendir: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87411 finisher ceph cwd unknown /proc/85450/task/87411/cwd (readlink: Permission denied) 2024-05-10T04:42:31.784 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87411 finisher ceph rtd unknown /proc/85450/task/87411/root (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87411 finisher ceph txt unknown /proc/85450/task/87411/exe (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87411 finisher ceph NOFD /proc/85450/task/87411/fd (opendir: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87412 safe_time ceph cwd unknown /proc/85450/task/87412/cwd (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87412 safe_time ceph rtd unknown /proc/85450/task/87412/root (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87412 safe_time ceph txt unknown /proc/85450/task/87412/exe (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87412 safe_time ceph NOFD /proc/85450/task/87412/fd (opendir: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87413 safe_time ceph cwd unknown /proc/85450/task/87413/cwd (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87413 safe_time ceph rtd unknown /proc/85450/task/87413/root (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87413 safe_time ceph txt unknown /proc/85450/task/87413/exe (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87413 safe_time ceph NOFD /proc/85450/task/87413/fd (opendir: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87414 ceph-osd ceph cwd unknown /proc/85450/task/87414/cwd (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87414 ceph-osd ceph rtd unknown /proc/85450/task/87414/root (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87414 ceph-osd ceph txt unknown /proc/85450/task/87414/exe (readlink: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87414 ceph-osd ceph NOFD /proc/85450/task/87414/fd (opendir: Permission denied) 2024-05-10T04:42:31.785 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87416 osd_srv_a ceph cwd unknown /proc/85450/task/87416/cwd (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87416 osd_srv_a ceph rtd unknown /proc/85450/task/87416/root (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87416 osd_srv_a ceph txt unknown /proc/85450/task/87416/exe (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87416 osd_srv_a ceph NOFD /proc/85450/task/87416/fd (opendir: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87417 tp_osd_tp ceph cwd unknown /proc/85450/task/87417/cwd (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87417 tp_osd_tp ceph rtd unknown /proc/85450/task/87417/root (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87417 tp_osd_tp ceph txt unknown /proc/85450/task/87417/exe (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87417 tp_osd_tp ceph NOFD /proc/85450/task/87417/fd (opendir: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87418 tp_osd_tp ceph cwd unknown /proc/85450/task/87418/cwd (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87418 tp_osd_tp ceph rtd unknown /proc/85450/task/87418/root (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87418 tp_osd_tp ceph txt unknown /proc/85450/task/87418/exe (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87418 tp_osd_tp ceph NOFD /proc/85450/task/87418/fd (opendir: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87419 tp_osd_tp ceph cwd unknown /proc/85450/task/87419/cwd (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87419 tp_osd_tp ceph rtd unknown /proc/85450/task/87419/root (readlink: Permission denied) 2024-05-10T04:42:31.786 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87419 tp_osd_tp ceph txt unknown /proc/85450/task/87419/exe (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87419 tp_osd_tp ceph NOFD /proc/85450/task/87419/fd (opendir: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87421 tp_osd_tp ceph cwd unknown /proc/85450/task/87421/cwd (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87421 tp_osd_tp ceph rtd unknown /proc/85450/task/87421/root (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87421 tp_osd_tp ceph txt unknown /proc/85450/task/87421/exe (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87421 tp_osd_tp ceph NOFD /proc/85450/task/87421/fd (opendir: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87422 tp_osd_tp ceph cwd unknown /proc/85450/task/87422/cwd (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87422 tp_osd_tp ceph rtd unknown /proc/85450/task/87422/root (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87422 tp_osd_tp ceph txt unknown /proc/85450/task/87422/exe (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87422 tp_osd_tp ceph NOFD /proc/85450/task/87422/fd (opendir: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87423 tp_osd_tp ceph cwd unknown /proc/85450/task/87423/cwd (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87423 tp_osd_tp ceph rtd unknown /proc/85450/task/87423/root (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87423 tp_osd_tp ceph txt unknown /proc/85450/task/87423/exe (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87423 tp_osd_tp ceph NOFD /proc/85450/task/87423/fd (opendir: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87424 tp_osd_tp ceph cwd unknown /proc/85450/task/87424/cwd (readlink: Permission denied) 2024-05-10T04:42:31.787 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87424 tp_osd_tp ceph rtd unknown /proc/85450/task/87424/root (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87424 tp_osd_tp ceph txt unknown /proc/85450/task/87424/exe (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87424 tp_osd_tp ceph NOFD /proc/85450/task/87424/fd (opendir: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87425 tp_osd_tp ceph cwd unknown /proc/85450/task/87425/cwd (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87425 tp_osd_tp ceph rtd unknown /proc/85450/task/87425/root (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87425 tp_osd_tp ceph txt unknown /proc/85450/task/87425/exe (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87425 tp_osd_tp ceph NOFD /proc/85450/task/87425/fd (opendir: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87426 tp_osd_tp ceph cwd unknown /proc/85450/task/87426/cwd (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87426 tp_osd_tp ceph rtd unknown /proc/85450/task/87426/root (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87426 tp_osd_tp ceph txt unknown /proc/85450/task/87426/exe (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87426 tp_osd_tp ceph NOFD /proc/85450/task/87426/fd (opendir: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87427 tp_osd_tp ceph cwd unknown /proc/85450/task/87427/cwd (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87427 tp_osd_tp ceph rtd unknown /proc/85450/task/87427/root (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87427 tp_osd_tp ceph txt unknown /proc/85450/task/87427/exe (readlink: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87427 tp_osd_tp ceph NOFD /proc/85450/task/87427/fd (opendir: Permission denied) 2024-05-10T04:42:31.788 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87428 tp_osd_tp ceph cwd unknown /proc/85450/task/87428/cwd (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87428 tp_osd_tp ceph rtd unknown /proc/85450/task/87428/root (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87428 tp_osd_tp ceph txt unknown /proc/85450/task/87428/exe (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87428 tp_osd_tp ceph NOFD /proc/85450/task/87428/fd (opendir: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87429 tp_osd_tp ceph cwd unknown /proc/85450/task/87429/cwd (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87429 tp_osd_tp ceph rtd unknown /proc/85450/task/87429/root (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87429 tp_osd_tp ceph txt unknown /proc/85450/task/87429/exe (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87429 tp_osd_tp ceph NOFD /proc/85450/task/87429/fd (opendir: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87431 tp_osd_tp ceph cwd unknown /proc/85450/task/87431/cwd (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87431 tp_osd_tp ceph rtd unknown /proc/85450/task/87431/root (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87431 tp_osd_tp ceph txt unknown /proc/85450/task/87431/exe (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87431 tp_osd_tp ceph NOFD /proc/85450/task/87431/fd (opendir: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87432 tp_osd_tp ceph cwd unknown /proc/85450/task/87432/cwd (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87432 tp_osd_tp ceph rtd unknown /proc/85450/task/87432/root (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87432 tp_osd_tp ceph txt unknown /proc/85450/task/87432/exe (readlink: Permission denied) 2024-05-10T04:42:31.789 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87432 tp_osd_tp ceph NOFD /proc/85450/task/87432/fd (opendir: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87433 tp_osd_tp ceph cwd unknown /proc/85450/task/87433/cwd (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87433 tp_osd_tp ceph rtd unknown /proc/85450/task/87433/root (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87433 tp_osd_tp ceph txt unknown /proc/85450/task/87433/exe (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87433 tp_osd_tp ceph NOFD /proc/85450/task/87433/fd (opendir: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87434 tp_osd_tp ceph cwd unknown /proc/85450/task/87434/cwd (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87434 tp_osd_tp ceph rtd unknown /proc/85450/task/87434/root (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87434 tp_osd_tp ceph txt unknown /proc/85450/task/87434/exe (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87434 tp_osd_tp ceph NOFD /proc/85450/task/87434/fd (opendir: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87435 osd_srv_h ceph cwd unknown /proc/85450/task/87435/cwd (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87435 osd_srv_h ceph rtd unknown /proc/85450/task/87435/root (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87435 osd_srv_h ceph txt unknown /proc/85450/task/87435/exe (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-osd 85450 87435 osd_srv_h ceph NOFD /proc/85450/task/87435/fd (opendir: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 86184 root cwd unknown /proc/86184/cwd (readlink: Permission denied) 2024-05-10T04:42:31.790 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 86184 root rtd unknown /proc/86184/root (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 86184 root txt unknown /proc/86184/exe (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 86184 root NOFD /proc/86184/fd (opendir: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 86217 root cwd unknown /proc/86217/cwd (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 86217 root rtd unknown /proc/86217/root (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 86217 root txt unknown /proc/86217/exe (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 86217 root NOFD /proc/86217/fd (opendir: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91295 root cwd unknown /proc/91295/cwd (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91295 root rtd unknown /proc/91295/root (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91295 root txt unknown /proc/91295/exe (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91295 root NOFD /proc/91295/fd (opendir: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91315 root cwd unknown /proc/91315/cwd (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91315 root rtd unknown /proc/91315/root (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91315 root txt unknown /proc/91315/exe (readlink: Permission denied) 2024-05-10T04:42:31.791 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91315 root NOFD /proc/91315/fd (opendir: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91316 root cwd unknown /proc/91316/cwd (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91316 root rtd unknown /proc/91316/root (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91316 root txt unknown /proc/91316/exe (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91316 root NOFD /proc/91316/fd (opendir: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91336 root cwd unknown /proc/91336/cwd (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91336 root rtd unknown /proc/91336/root (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91336 root txt unknown /proc/91336/exe (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91336 root NOFD /proc/91336/fd (opendir: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91337 root cwd unknown /proc/91337/cwd (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91337 root rtd unknown /proc/91337/root (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91337 root txt unknown /proc/91337/exe (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91337 root NOFD /proc/91337/fd (opendir: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91357 root cwd unknown /proc/91357/cwd (readlink: Permission denied) 2024-05-10T04:42:31.792 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91357 root rtd unknown /proc/91357/root (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91357 root txt unknown /proc/91357/exe (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91357 root NOFD /proc/91357/fd (opendir: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91358 root cwd unknown /proc/91358/cwd (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91358 root rtd unknown /proc/91358/root (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91358 root txt unknown /proc/91358/exe (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 91358 root NOFD /proc/91358/fd (opendir: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91378 root cwd unknown /proc/91378/cwd (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91378 root rtd unknown /proc/91378/root (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91378 root txt unknown /proc/91378/exe (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:journalct 91378 root NOFD /proc/91378/fd (opendir: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 91855 root cwd unknown /proc/91855/cwd (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 91855 root rtd unknown /proc/91855/root (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 91855 root txt unknown /proc/91855/exe (readlink: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 91855 root NOFD /proc/91855/fd (opendir: Permission denied) 2024-05-10T04:42:31.793 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 91866 root cwd unknown /proc/91866/cwd (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 91866 root rtd unknown /proc/91866/root (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 91866 root txt unknown /proc/91866/exe (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 91866 root NOFD /proc/91866/fd (opendir: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 ceph cwd unknown /proc/91878/cwd (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 ceph rtd unknown /proc/91878/root (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 ceph txt unknown /proc/91878/exe (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 ceph NOFD /proc/91878/fd (opendir: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91879 log ceph cwd unknown /proc/91878/task/91879/cwd (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91879 log ceph rtd unknown /proc/91878/task/91879/root (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91879 log ceph txt unknown /proc/91878/task/91879/exe (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91879 log ceph NOFD /proc/91878/task/91879/fd (opendir: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91881 msgr-work ceph cwd unknown /proc/91878/task/91881/cwd (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91881 msgr-work ceph rtd unknown /proc/91878/task/91881/root (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91881 msgr-work ceph txt unknown /proc/91878/task/91881/exe (readlink: Permission denied) 2024-05-10T04:42:31.794 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91881 msgr-work ceph NOFD /proc/91878/task/91881/fd (opendir: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91882 msgr-work ceph cwd unknown /proc/91878/task/91882/cwd (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91882 msgr-work ceph rtd unknown /proc/91878/task/91882/root (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91882 msgr-work ceph txt unknown /proc/91878/task/91882/exe (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91882 msgr-work ceph NOFD /proc/91878/task/91882/fd (opendir: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91883 msgr-work ceph cwd unknown /proc/91878/task/91883/cwd (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91883 msgr-work ceph rtd unknown /proc/91878/task/91883/root (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91883 msgr-work ceph txt unknown /proc/91878/task/91883/exe (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91883 msgr-work ceph NOFD /proc/91878/task/91883/fd (opendir: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91887 service ceph cwd unknown /proc/91878/task/91887/cwd (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91887 service ceph rtd unknown /proc/91878/task/91887/root (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91887 service ceph txt unknown /proc/91878/task/91887/exe (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91887 service ceph NOFD /proc/91878/task/91887/fd (opendir: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91888 admin_soc ceph cwd unknown /proc/91878/task/91888/cwd (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91888 admin_soc ceph rtd unknown /proc/91878/task/91888/root (readlink: Permission denied) 2024-05-10T04:42:31.795 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91888 admin_soc ceph txt unknown /proc/91878/task/91888/exe (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91888 admin_soc ceph NOFD /proc/91878/task/91888/fd (opendir: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91889 signal_ha ceph cwd unknown /proc/91878/task/91889/cwd (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91889 signal_ha ceph rtd unknown /proc/91878/task/91889/root (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91889 signal_ha ceph txt unknown /proc/91878/task/91889/exe (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91889 signal_ha ceph NOFD /proc/91878/task/91889/fd (opendir: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91890 io_contex ceph cwd unknown /proc/91878/task/91890/cwd (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91890 io_contex ceph rtd unknown /proc/91878/task/91890/root (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91890 io_contex ceph txt unknown /proc/91878/task/91890/exe (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91890 io_contex ceph NOFD /proc/91878/task/91890/fd (opendir: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91891 io_contex ceph cwd unknown /proc/91878/task/91891/cwd (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91891 io_contex ceph rtd unknown /proc/91878/task/91891/root (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91891 io_contex ceph txt unknown /proc/91878/task/91891/exe (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91891 io_contex ceph NOFD /proc/91878/task/91891/fd (opendir: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91892 ms_dispat ceph cwd unknown /proc/91878/task/91892/cwd (readlink: Permission denied) 2024-05-10T04:42:31.796 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91892 ms_dispat ceph rtd unknown /proc/91878/task/91892/root (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91892 ms_dispat ceph txt unknown /proc/91878/task/91892/exe (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91892 ms_dispat ceph NOFD /proc/91878/task/91892/fd (opendir: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91893 ms_local ceph cwd unknown /proc/91878/task/91893/cwd (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91893 ms_local ceph rtd unknown /proc/91878/task/91893/root (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91893 ms_local ceph txt unknown /proc/91878/task/91893/exe (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91893 ms_local ceph NOFD /proc/91878/task/91893/fd (opendir: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91894 safe_time ceph cwd unknown /proc/91878/task/91894/cwd (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91894 safe_time ceph rtd unknown /proc/91878/task/91894/root (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91894 safe_time ceph txt unknown /proc/91878/task/91894/exe (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91894 safe_time ceph NOFD /proc/91878/task/91894/fd (opendir: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91895 safe_time ceph cwd unknown /proc/91878/task/91895/cwd (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91895 safe_time ceph rtd unknown /proc/91878/task/91895/root (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91895 safe_time ceph txt unknown /proc/91878/task/91895/exe (readlink: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91895 safe_time ceph NOFD /proc/91878/task/91895/fd (opendir: Permission denied) 2024-05-10T04:42:31.797 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91896 ceph-mds ceph cwd unknown /proc/91878/task/91896/cwd (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91896 ceph-mds ceph rtd unknown /proc/91878/task/91896/root (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91896 ceph-mds ceph txt unknown /proc/91878/task/91896/exe (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 91896 ceph-mds ceph NOFD /proc/91878/task/91896/fd (opendir: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 92006 safe_time ceph cwd unknown /proc/91878/task/92006/cwd (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 92006 safe_time ceph rtd unknown /proc/91878/task/92006/root (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 92006 safe_time ceph txt unknown /proc/91878/task/92006/exe (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 91878 92006 safe_time ceph NOFD /proc/91878/task/92006/fd (opendir: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92245 root cwd unknown /proc/92245/cwd (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92245 root rtd unknown /proc/92245/root (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92245 root txt unknown /proc/92245/exe (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92245 root NOFD /proc/92245/fd (opendir: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92256 root cwd unknown /proc/92256/cwd (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92256 root rtd unknown /proc/92256/root (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92256 root txt unknown /proc/92256/exe (readlink: Permission denied) 2024-05-10T04:42:31.798 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92256 root NOFD /proc/92256/fd (opendir: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 ceph cwd unknown /proc/92268/cwd (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 ceph rtd unknown /proc/92268/root (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 ceph txt unknown /proc/92268/exe (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 ceph NOFD /proc/92268/fd (opendir: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92269 log ceph cwd unknown /proc/92268/task/92269/cwd (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92269 log ceph rtd unknown /proc/92268/task/92269/root (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92269 log ceph txt unknown /proc/92268/task/92269/exe (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92269 log ceph NOFD /proc/92268/task/92269/fd (opendir: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92271 msgr-work ceph cwd unknown /proc/92268/task/92271/cwd (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92271 msgr-work ceph rtd unknown /proc/92268/task/92271/root (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92271 msgr-work ceph txt unknown /proc/92268/task/92271/exe (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92271 msgr-work ceph NOFD /proc/92268/task/92271/fd (opendir: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92272 msgr-work ceph cwd unknown /proc/92268/task/92272/cwd (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92272 msgr-work ceph rtd unknown /proc/92268/task/92272/root (readlink: Permission denied) 2024-05-10T04:42:31.799 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92272 msgr-work ceph txt unknown /proc/92268/task/92272/exe (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92272 msgr-work ceph NOFD /proc/92268/task/92272/fd (opendir: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92273 msgr-work ceph cwd unknown /proc/92268/task/92273/cwd (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92273 msgr-work ceph rtd unknown /proc/92268/task/92273/root (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92273 msgr-work ceph txt unknown /proc/92268/task/92273/exe (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92273 msgr-work ceph NOFD /proc/92268/task/92273/fd (opendir: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92277 service ceph cwd unknown /proc/92268/task/92277/cwd (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92277 service ceph rtd unknown /proc/92268/task/92277/root (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92277 service ceph txt unknown /proc/92268/task/92277/exe (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92277 service ceph NOFD /proc/92268/task/92277/fd (opendir: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92278 admin_soc ceph cwd unknown /proc/92268/task/92278/cwd (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92278 admin_soc ceph rtd unknown /proc/92268/task/92278/root (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92278 admin_soc ceph txt unknown /proc/92268/task/92278/exe (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92278 admin_soc ceph NOFD /proc/92268/task/92278/fd (opendir: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92279 signal_ha ceph cwd unknown /proc/92268/task/92279/cwd (readlink: Permission denied) 2024-05-10T04:42:31.800 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92279 signal_ha ceph rtd unknown /proc/92268/task/92279/root (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92279 signal_ha ceph txt unknown /proc/92268/task/92279/exe (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92279 signal_ha ceph NOFD /proc/92268/task/92279/fd (opendir: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92280 io_contex ceph cwd unknown /proc/92268/task/92280/cwd (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92280 io_contex ceph rtd unknown /proc/92268/task/92280/root (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92280 io_contex ceph txt unknown /proc/92268/task/92280/exe (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92280 io_contex ceph NOFD /proc/92268/task/92280/fd (opendir: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92281 io_contex ceph cwd unknown /proc/92268/task/92281/cwd (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92281 io_contex ceph rtd unknown /proc/92268/task/92281/root (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92281 io_contex ceph txt unknown /proc/92268/task/92281/exe (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92281 io_contex ceph NOFD /proc/92268/task/92281/fd (opendir: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92282 ms_dispat ceph cwd unknown /proc/92268/task/92282/cwd (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92282 ms_dispat ceph rtd unknown /proc/92268/task/92282/root (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92282 ms_dispat ceph txt unknown /proc/92268/task/92282/exe (readlink: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92282 ms_dispat ceph NOFD /proc/92268/task/92282/fd (opendir: Permission denied) 2024-05-10T04:42:31.801 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92283 ms_local ceph cwd unknown /proc/92268/task/92283/cwd (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92283 ms_local ceph rtd unknown /proc/92268/task/92283/root (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92283 ms_local ceph txt unknown /proc/92268/task/92283/exe (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92283 ms_local ceph NOFD /proc/92268/task/92283/fd (opendir: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92284 safe_time ceph cwd unknown /proc/92268/task/92284/cwd (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92284 safe_time ceph rtd unknown /proc/92268/task/92284/root (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92284 safe_time ceph txt unknown /proc/92268/task/92284/exe (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92284 safe_time ceph NOFD /proc/92268/task/92284/fd (opendir: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92285 safe_time ceph cwd unknown /proc/92268/task/92285/cwd (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92285 safe_time ceph rtd unknown /proc/92268/task/92285/root (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92285 safe_time ceph txt unknown /proc/92268/task/92285/exe (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92285 safe_time ceph NOFD /proc/92268/task/92285/fd (opendir: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92286 ceph-mds ceph cwd unknown /proc/92268/task/92286/cwd (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92286 ceph-mds ceph rtd unknown /proc/92268/task/92286/root (readlink: Permission denied) 2024-05-10T04:42:31.802 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92286 ceph-mds ceph txt unknown /proc/92268/task/92286/exe (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92286 ceph-mds ceph NOFD /proc/92268/task/92286/fd (opendir: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92291 safe_time ceph cwd unknown /proc/92268/task/92291/cwd (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92291 safe_time ceph rtd unknown /proc/92268/task/92291/root (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92291 safe_time ceph txt unknown /proc/92268/task/92291/exe (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92268 92291 safe_time ceph NOFD /proc/92268/task/92291/fd (opendir: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92765 root cwd unknown /proc/92765/cwd (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92765 root rtd unknown /proc/92765/root (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92765 root txt unknown /proc/92765/exe (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 92765 root NOFD /proc/92765/fd (opendir: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92776 root cwd unknown /proc/92776/cwd (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92776 root rtd unknown /proc/92776/root (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92776 root txt unknown /proc/92776/exe (readlink: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 92776 root NOFD /proc/92776/fd (opendir: Permission denied) 2024-05-10T04:42:31.803 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 ceph cwd unknown /proc/92788/cwd (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 ceph rtd unknown /proc/92788/root (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 ceph txt unknown /proc/92788/exe (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 ceph NOFD /proc/92788/fd (opendir: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92789 log ceph cwd unknown /proc/92788/task/92789/cwd (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92789 log ceph rtd unknown /proc/92788/task/92789/root (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92789 log ceph txt unknown /proc/92788/task/92789/exe (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92789 log ceph NOFD /proc/92788/task/92789/fd (opendir: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92792 msgr-work ceph cwd unknown /proc/92788/task/92792/cwd (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92792 msgr-work ceph rtd unknown /proc/92788/task/92792/root (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92792 msgr-work ceph txt unknown /proc/92788/task/92792/exe (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92792 msgr-work ceph NOFD /proc/92788/task/92792/fd (opendir: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92793 msgr-work ceph cwd unknown /proc/92788/task/92793/cwd (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92793 msgr-work ceph rtd unknown /proc/92788/task/92793/root (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92793 msgr-work ceph txt unknown /proc/92788/task/92793/exe (readlink: Permission denied) 2024-05-10T04:42:31.804 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92793 msgr-work ceph NOFD /proc/92788/task/92793/fd (opendir: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92794 msgr-work ceph cwd unknown /proc/92788/task/92794/cwd (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92794 msgr-work ceph rtd unknown /proc/92788/task/92794/root (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92794 msgr-work ceph txt unknown /proc/92788/task/92794/exe (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92794 msgr-work ceph NOFD /proc/92788/task/92794/fd (opendir: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92799 service ceph cwd unknown /proc/92788/task/92799/cwd (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92799 service ceph rtd unknown /proc/92788/task/92799/root (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92799 service ceph txt unknown /proc/92788/task/92799/exe (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92799 service ceph NOFD /proc/92788/task/92799/fd (opendir: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92800 admin_soc ceph cwd unknown /proc/92788/task/92800/cwd (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92800 admin_soc ceph rtd unknown /proc/92788/task/92800/root (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92800 admin_soc ceph txt unknown /proc/92788/task/92800/exe (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92800 admin_soc ceph NOFD /proc/92788/task/92800/fd (opendir: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92801 signal_ha ceph cwd unknown /proc/92788/task/92801/cwd (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92801 signal_ha ceph rtd unknown /proc/92788/task/92801/root (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92801 signal_ha ceph txt unknown /proc/92788/task/92801/exe (readlink: Permission denied) 2024-05-10T04:42:31.805 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92801 signal_ha ceph NOFD /proc/92788/task/92801/fd (opendir: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92802 io_contex ceph cwd unknown /proc/92788/task/92802/cwd (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92802 io_contex ceph rtd unknown /proc/92788/task/92802/root (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92802 io_contex ceph txt unknown /proc/92788/task/92802/exe (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92802 io_contex ceph NOFD /proc/92788/task/92802/fd (opendir: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92803 io_contex ceph cwd unknown /proc/92788/task/92803/cwd (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92803 io_contex ceph rtd unknown /proc/92788/task/92803/root (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92803 io_contex ceph txt unknown /proc/92788/task/92803/exe (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92803 io_contex ceph NOFD /proc/92788/task/92803/fd (opendir: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92804 ms_dispat ceph cwd unknown /proc/92788/task/92804/cwd (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92804 ms_dispat ceph rtd unknown /proc/92788/task/92804/root (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92804 ms_dispat ceph txt unknown /proc/92788/task/92804/exe (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92804 ms_dispat ceph NOFD /proc/92788/task/92804/fd (opendir: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92805 ms_local ceph cwd unknown /proc/92788/task/92805/cwd (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92805 ms_local ceph rtd unknown /proc/92788/task/92805/root (readlink: Permission denied) 2024-05-10T04:42:31.806 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92805 ms_local ceph txt unknown /proc/92788/task/92805/exe (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92805 ms_local ceph NOFD /proc/92788/task/92805/fd (opendir: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92806 safe_time ceph cwd unknown /proc/92788/task/92806/cwd (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92806 safe_time ceph rtd unknown /proc/92788/task/92806/root (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92806 safe_time ceph txt unknown /proc/92788/task/92806/exe (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92806 safe_time ceph NOFD /proc/92788/task/92806/fd (opendir: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92807 safe_time ceph cwd unknown /proc/92788/task/92807/cwd (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92807 safe_time ceph rtd unknown /proc/92788/task/92807/root (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92807 safe_time ceph txt unknown /proc/92788/task/92807/exe (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92807 safe_time ceph NOFD /proc/92788/task/92807/fd (opendir: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92808 ceph-mds ceph cwd unknown /proc/92788/task/92808/cwd (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92808 ceph-mds ceph rtd unknown /proc/92788/task/92808/root (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92808 ceph-mds ceph txt unknown /proc/92788/task/92808/exe (readlink: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 92808 ceph-mds ceph NOFD /proc/92788/task/92808/fd (opendir: Permission denied) 2024-05-10T04:42:31.807 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 93020 safe_time ceph cwd unknown /proc/92788/task/93020/cwd (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 93020 safe_time ceph rtd unknown /proc/92788/task/93020/root (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 93020 safe_time ceph txt unknown /proc/92788/task/93020/exe (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 93020 safe_time ceph NOFD /proc/92788/task/93020/fd (opendir: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96849 ceph_time ceph cwd unknown /proc/92788/task/96849/cwd (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96849 ceph_time ceph rtd unknown /proc/92788/task/96849/root (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96849 ceph_time ceph txt unknown /proc/92788/task/96849/exe (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96849 ceph_time ceph NOFD /proc/92788/task/96849/fd (opendir: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96852 OpHistory ceph cwd unknown /proc/92788/task/96852/cwd (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96852 OpHistory ceph rtd unknown /proc/92788/task/96852/root (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96852 OpHistory ceph txt unknown /proc/92788/task/96852/exe (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96852 OpHistory ceph NOFD /proc/92788/task/96852/fd (opendir: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96854 ms_dispat ceph cwd unknown /proc/92788/task/96854/cwd (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96854 ms_dispat ceph rtd unknown /proc/92788/task/96854/root (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96854 ms_dispat ceph txt unknown /proc/92788/task/96854/exe (readlink: Permission denied) 2024-05-10T04:42:31.808 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96854 ms_dispat ceph NOFD /proc/92788/task/96854/fd (opendir: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96857 mds_rank_ ceph cwd unknown /proc/92788/task/96857/cwd (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96857 mds_rank_ ceph rtd unknown /proc/92788/task/96857/root (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96857 mds_rank_ ceph txt unknown /proc/92788/task/96857/exe (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96857 mds_rank_ ceph NOFD /proc/92788/task/96857/fd (opendir: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96859 PQ_Finish ceph cwd unknown /proc/92788/task/96859/cwd (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96859 PQ_Finish ceph rtd unknown /proc/92788/task/96859/root (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96859 PQ_Finish ceph txt unknown /proc/92788/task/96859/exe (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96859 PQ_Finish ceph NOFD /proc/92788/task/96859/fd (opendir: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96861 safe_time ceph cwd unknown /proc/92788/task/96861/cwd (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96861 safe_time ceph rtd unknown /proc/92788/task/96861/root (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96861 safe_time ceph txt unknown /proc/92788/task/96861/exe (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96861 safe_time ceph NOFD /proc/92788/task/96861/fd (opendir: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96862 MR_Finish ceph cwd unknown /proc/92788/task/96862/cwd (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96862 MR_Finish ceph rtd unknown /proc/92788/task/96862/root (readlink: Permission denied) 2024-05-10T04:42:31.809 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96862 MR_Finish ceph txt unknown /proc/92788/task/96862/exe (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96862 MR_Finish ceph NOFD /proc/92788/task/96862/fd (opendir: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96866 md_submit ceph cwd unknown /proc/92788/task/96866/cwd (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96866 md_submit ceph rtd unknown /proc/92788/task/96866/root (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96866 md_submit ceph txt unknown /proc/92788/task/96866/exe (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 92788 96866 md_submit ceph NOFD /proc/92788/task/96866/fd (opendir: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 92999 root cwd unknown /proc/92999/cwd (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 92999 root rtd unknown /proc/92999/root (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 92999 root txt unknown /proc/92999/exe (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/5 92999 root NOFD /proc/92999/fd (opendir: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 93152 root cwd unknown /proc/93152/cwd (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 93152 root rtd unknown /proc/93152/root (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 93152 root txt unknown /proc/93152/exe (readlink: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:conmon 93152 root NOFD /proc/93152/fd (opendir: Permission denied) 2024-05-10T04:42:31.810 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 93164 root cwd unknown /proc/93164/cwd (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 93164 root rtd unknown /proc/93164/root (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 93164 root txt unknown /proc/93164/exe (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:podman-in 93164 root NOFD /proc/93164/fd (opendir: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 ceph cwd unknown /proc/93176/cwd (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 ceph rtd unknown /proc/93176/root (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 ceph txt unknown /proc/93176/exe (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 ceph NOFD /proc/93176/fd (opendir: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93177 log ceph cwd unknown /proc/93176/task/93177/cwd (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93177 log ceph rtd unknown /proc/93176/task/93177/root (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93177 log ceph txt unknown /proc/93176/task/93177/exe (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93177 log ceph NOFD /proc/93176/task/93177/fd (opendir: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93179 msgr-work ceph cwd unknown /proc/93176/task/93179/cwd (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93179 msgr-work ceph rtd unknown /proc/93176/task/93179/root (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93179 msgr-work ceph txt unknown /proc/93176/task/93179/exe (readlink: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93179 msgr-work ceph NOFD /proc/93176/task/93179/fd (opendir: Permission denied) 2024-05-10T04:42:31.811 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93180 msgr-work ceph cwd unknown /proc/93176/task/93180/cwd (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93180 msgr-work ceph rtd unknown /proc/93176/task/93180/root (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93180 msgr-work ceph txt unknown /proc/93176/task/93180/exe (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93180 msgr-work ceph NOFD /proc/93176/task/93180/fd (opendir: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93181 msgr-work ceph cwd unknown /proc/93176/task/93181/cwd (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93181 msgr-work ceph rtd unknown /proc/93176/task/93181/root (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93181 msgr-work ceph txt unknown /proc/93176/task/93181/exe (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93181 msgr-work ceph NOFD /proc/93176/task/93181/fd (opendir: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93187 service ceph cwd unknown /proc/93176/task/93187/cwd (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93187 service ceph rtd unknown /proc/93176/task/93187/root (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93187 service ceph txt unknown /proc/93176/task/93187/exe (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93187 service ceph NOFD /proc/93176/task/93187/fd (opendir: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93188 admin_soc ceph cwd unknown /proc/93176/task/93188/cwd (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93188 admin_soc ceph rtd unknown /proc/93176/task/93188/root (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93188 admin_soc ceph txt unknown /proc/93176/task/93188/exe (readlink: Permission denied) 2024-05-10T04:42:31.812 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93188 admin_soc ceph NOFD /proc/93176/task/93188/fd (opendir: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93189 signal_ha ceph cwd unknown /proc/93176/task/93189/cwd (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93189 signal_ha ceph rtd unknown /proc/93176/task/93189/root (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93189 signal_ha ceph txt unknown /proc/93176/task/93189/exe (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93189 signal_ha ceph NOFD /proc/93176/task/93189/fd (opendir: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93190 io_contex ceph cwd unknown /proc/93176/task/93190/cwd (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93190 io_contex ceph rtd unknown /proc/93176/task/93190/root (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93190 io_contex ceph txt unknown /proc/93176/task/93190/exe (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93190 io_contex ceph NOFD /proc/93176/task/93190/fd (opendir: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93191 io_contex ceph cwd unknown /proc/93176/task/93191/cwd (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93191 io_contex ceph rtd unknown /proc/93176/task/93191/root (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93191 io_contex ceph txt unknown /proc/93176/task/93191/exe (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93191 io_contex ceph NOFD /proc/93176/task/93191/fd (opendir: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93192 ms_dispat ceph cwd unknown /proc/93176/task/93192/cwd (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93192 ms_dispat ceph rtd unknown /proc/93176/task/93192/root (readlink: Permission denied) 2024-05-10T04:42:31.813 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93192 ms_dispat ceph txt unknown /proc/93176/task/93192/exe (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93192 ms_dispat ceph NOFD /proc/93176/task/93192/fd (opendir: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93193 ms_local ceph cwd unknown /proc/93176/task/93193/cwd (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93193 ms_local ceph rtd unknown /proc/93176/task/93193/root (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93193 ms_local ceph txt unknown /proc/93176/task/93193/exe (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93193 ms_local ceph NOFD /proc/93176/task/93193/fd (opendir: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93194 safe_time ceph cwd unknown /proc/93176/task/93194/cwd (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93194 safe_time ceph rtd unknown /proc/93176/task/93194/root (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93194 safe_time ceph txt unknown /proc/93176/task/93194/exe (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93194 safe_time ceph NOFD /proc/93176/task/93194/fd (opendir: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93195 safe_time ceph cwd unknown /proc/93176/task/93195/cwd (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93195 safe_time ceph rtd unknown /proc/93176/task/93195/root (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93195 safe_time ceph txt unknown /proc/93176/task/93195/exe (readlink: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93195 safe_time ceph NOFD /proc/93176/task/93195/fd (opendir: Permission denied) 2024-05-10T04:42:31.814 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93196 ceph-mds ceph cwd unknown /proc/93176/task/93196/cwd (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93196 ceph-mds ceph rtd unknown /proc/93176/task/93196/root (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93196 ceph-mds ceph txt unknown /proc/93176/task/93196/exe (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93196 ceph-mds ceph NOFD /proc/93176/task/93196/fd (opendir: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93199 safe_time ceph cwd unknown /proc/93176/task/93199/cwd (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93199 safe_time ceph rtd unknown /proc/93176/task/93199/root (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93199 safe_time ceph txt unknown /proc/93176/task/93199/exe (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 93199 safe_time ceph NOFD /proc/93176/task/93199/fd (opendir: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96850 ceph_time ceph cwd unknown /proc/93176/task/96850/cwd (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96850 ceph_time ceph rtd unknown /proc/93176/task/96850/root (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96850 ceph_time ceph txt unknown /proc/93176/task/96850/exe (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96850 ceph_time ceph NOFD /proc/93176/task/96850/fd (opendir: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96851 OpHistory ceph cwd unknown /proc/93176/task/96851/cwd (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96851 OpHistory ceph rtd unknown /proc/93176/task/96851/root (readlink: Permission denied) 2024-05-10T04:42:31.815 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96851 OpHistory ceph txt unknown /proc/93176/task/96851/exe (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96851 OpHistory ceph NOFD /proc/93176/task/96851/fd (opendir: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96853 ms_dispat ceph cwd unknown /proc/93176/task/96853/cwd (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96853 ms_dispat ceph rtd unknown /proc/93176/task/96853/root (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96853 ms_dispat ceph txt unknown /proc/93176/task/96853/exe (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96853 ms_dispat ceph NOFD /proc/93176/task/96853/fd (opendir: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96855 mds_rank_ ceph cwd unknown /proc/93176/task/96855/cwd (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96855 mds_rank_ ceph rtd unknown /proc/93176/task/96855/root (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96855 mds_rank_ ceph txt unknown /proc/93176/task/96855/exe (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96855 mds_rank_ ceph NOFD /proc/93176/task/96855/fd (opendir: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96856 PQ_Finish ceph cwd unknown /proc/93176/task/96856/cwd (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96856 PQ_Finish ceph rtd unknown /proc/93176/task/96856/root (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96856 PQ_Finish ceph txt unknown /proc/93176/task/96856/exe (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96856 PQ_Finish ceph NOFD /proc/93176/task/96856/fd (opendir: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96858 safe_time ceph cwd unknown /proc/93176/task/96858/cwd (readlink: Permission denied) 2024-05-10T04:42:31.816 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96858 safe_time ceph rtd unknown /proc/93176/task/96858/root (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96858 safe_time ceph txt unknown /proc/93176/task/96858/exe (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96858 safe_time ceph NOFD /proc/93176/task/96858/fd (opendir: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96860 MR_Finish ceph cwd unknown /proc/93176/task/96860/cwd (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96860 MR_Finish ceph rtd unknown /proc/93176/task/96860/root (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96860 MR_Finish ceph txt unknown /proc/93176/task/96860/exe (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96860 MR_Finish ceph NOFD /proc/93176/task/96860/fd (opendir: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96864 md_submit ceph cwd unknown /proc/93176/task/96864/cwd (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96864 md_submit ceph rtd unknown /proc/93176/task/96864/root (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96864 md_submit ceph txt unknown /proc/93176/task/96864/exe (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-mds 93176 96864 md_submit ceph NOFD /proc/93176/task/96864/fd (opendir: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 96635 root cwd unknown /proc/96635/cwd (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 96635 root rtd unknown /proc/96635/root (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 96635 root txt unknown /proc/96635/exe (readlink: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/u 96635 root NOFD /proc/96635/fd (opendir: Permission denied) 2024-05-10T04:42:31.817 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 106479 root cwd unknown /proc/106479/cwd (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 106479 root rtd unknown /proc/106479/root (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 106479 root txt unknown /proc/106479/exe (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:kworker/1 106479 root NOFD /proc/106479/fd (opendir: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-msgr 108250 root cwd unknown /proc/108250/cwd (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-msgr 108250 root rtd unknown /proc/108250/root (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-msgr 108250 root txt unknown /proc/108250/exe (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-msgr 108250 root NOFD /proc/108250/fd (opendir: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-watc 108252 root cwd unknown /proc/108252/cwd (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-watc 108252 root rtd unknown /proc/108252/root (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-watc 108252 root txt unknown /proc/108252/exe (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-watc 108252 root NOFD /proc/108252/fd (opendir: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-comp 108253 root cwd unknown /proc/108253/cwd (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-comp 108253 root rtd unknown /proc/108253/root (readlink: Permission denied) 2024-05-10T04:42:31.818 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-comp 108253 root txt unknown /proc/108253/exe (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:ceph-comp 108253 root NOFD /proc/108253/fd (opendir: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 109483 root cwd unknown /proc/109483/cwd (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 109483 root rtd unknown /proc/109483/root (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 109483 root txt unknown /proc/109483/exe (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:sudo 109483 root NOFD /proc/109483/fd (opendir: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:daemon-he 109536 root cwd unknown /proc/109536/cwd (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:daemon-he 109536 root rtd unknown /proc/109536/root (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:daemon-he 109536 root txt unknown /proc/109536/exe (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:daemon-he 109536 root NOFD /proc/109536/fd (opendir: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:bash 109546 root cwd unknown /proc/109546/cwd (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:bash 109546 root rtd unknown /proc/109546/root (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:bash 109546 root txt unknown /proc/109546/exe (readlink: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:bash 109546 root NOFD /proc/109546/fd (opendir: Permission denied) 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu cwd DIR 0,124 4 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T04:42:31.819 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu rtd DIR 8,1 4096 2 / 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu txt REG 8,1 42840 7406 /usr/bin/timeout 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 2586930 18369 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 42744 5185 /usr/lib64/librt-2.28.so 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 54 18374 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 3316 18324 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 286 18408 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 57 18372 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 34 18389 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 77 18373 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 167 18405 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-10T04:42:31.820 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 59 18409 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 23 18407 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu mem REG 8,1 368 18406 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu 0r FIFO 0,13 0t0 460173 pipe 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu 1w FIFO 0,13 0t0 460174 pipe 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:timeout 112549 ubuntu 2w FIFO 0,13 0t0 460175 pipe 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu cwd DIR 0,124 4 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu rtd DIR 8,1 4096 2 / 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu txt REG 8,1 1150584 5206 /usr/bin/bash 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 2586930 18369 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 187552 4894 /usr/lib64/libtinfo.so.6.1 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-05-10T04:42:31.821 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 54 18374 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 3316 18324 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 286 18408 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 57 18372 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 34 18389 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 77 18373 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 167 18405 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 59 18409 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 23 18407 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu mem REG 8,1 368 18406 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu 0r FIFO 0,13 0t0 460173 pipe 2024-05-10T04:42:31.822 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu 1w FIFO 0,13 0t0 460174 pipe 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu 2w FIFO 0,13 0t0 460175 pipe 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:fsync-tes 112570 ubuntu 255r REG 8,1 209 1966761 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 112634 root cwd unknown /proc/112634/cwd (readlink: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 112634 root rtd unknown /proc/112634/root (readlink: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 112634 root txt unknown /proc/112634/exe (readlink: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 112634 root NOFD /proc/112634/fd (opendir: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 113548 root cwd unknown /proc/113548/cwd (readlink: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 113548 root rtd unknown /proc/113548/root (readlink: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 113548 root txt unknown /proc/113548/exe (readlink: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:sleep 113548 root NOFD /proc/113548/fd (opendir: Permission denied) 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu cwd DIR 0,124 4 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu rtd DIR 8,1 4096 2 / 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu txt REG 8,1 179672 11587 /usr/bin/lsof 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 99600 5318 /usr/lib64/libz.so.1.2.11 2024-05-10T04:42:31.823 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 88824 5183 /usr/lib64/libresolv-2.28.so 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 3087848 2384 /usr/lib64/libcrypto.so.1.1.1k 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 16192 5609 /usr/lib64/libkeyutils.so.1.6 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 71464 7213 /usr/lib64/libkrb5support.so.0.1 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 16792 5320 /usr/lib64/libcom_err.so.2.1 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 96272 7207 /usr/lib64/libk5crypto.so.3.1 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 975608 7212 /usr/lib64/libkrb5.so.3.3 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 355960 7204 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 543160 4872 /usr/lib64/libpcre2-8.so.0.7.1 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 213592 7222 /usr/lib64/libtirpc.so.3.0.0 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 168536 4876 /usr/lib64/libselinux.so.1 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-05-10T04:42:31.824 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu 0r FIFO 0,13 0t0 460173 pipe 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu 1w FIFO 0,13 0t0 460174 pipe 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu 2w FIFO 0,13 0t0 460175 pipe 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu 3r DIR 0,5 0 1 /proc 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu 4r DIR 0,5 0 464444 /proc/114009/fd 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu 5w FIFO 0,13 0t0 464449 pipe 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114009 ubuntu 6r FIFO 0,13 0t0 464450 pipe 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu cwd DIR 0,124 4 1099511629283 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu rtd DIR 8,1 4096 2 / 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu txt REG 8,1 179672 11587 /usr/bin/lsof 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 99600 5318 /usr/lib64/libz.so.1.2.11 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 88824 5183 /usr/lib64/libresolv-2.28.so 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 3087848 2384 /usr/lib64/libcrypto.so.1.1.1k 2024-05-10T04:42:31.825 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 16192 5609 /usr/lib64/libkeyutils.so.1.6 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 71464 7213 /usr/lib64/libkrb5support.so.0.1 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 16792 5320 /usr/lib64/libcom_err.so.2.1 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 96272 7207 /usr/lib64/libk5crypto.so.3.1 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 975608 7212 /usr/lib64/libkrb5.so.3.3 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 355960 7204 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 543160 4872 /usr/lib64/libpcre2-8.so.0.7.1 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 213592 7222 /usr/lib64/libtirpc.so.3.0.0 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 168536 4876 /usr/lib64/libselinux.so.1 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-05-10T04:42:31.826 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-05-10T04:42:31.827 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu 4r FIFO 0,13 0t0 464449 pipe 2024-05-10T04:42:31.827 INFO:tasks.workunit.client.0.smithi150.stdout:lsof 114010 ubuntu 7w FIFO 0,13 0t0 464450 pipe 2024-05-10T04:42:31.827 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-10T04:42:31.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-10T04:42:31.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:31 smithi161 ceph-mon[46912]: pgmap v360: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-05-10T04:42:31.853 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:31 smithi150 ceph-mon[42404]: pgmap v360: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-05-10T04:42:31.879 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-05-10T04:42:31.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-10T04:42:31.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:31 smithi181 ceph-mon[47314]: pgmap v360: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-05-10T04:42:32.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-10T04:42:32.100 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-10T04:42:32.212 DEBUG:teuthology.parallel:result is None 2024-05-10T04:42:32.212 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T04:42:32.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 -- 172.21.15.150:0/451770692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654100ac0 msgr2=0x7f1654100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 --2- 172.21.15.150:0/451770692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654100ac0 0x7f1654100ed0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f164400b0a0 tx=0x7f164400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 -- 172.21.15.150:0/451770692 shutdown_connections 2024-05-10T04:42:32.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 --2- 172.21.15.150:0/451770692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1654102580 0x7f1654108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 --2- 172.21.15.150:0/451770692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654100ac0 0x7f1654100ed0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.247 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 --2- 172.21.15.150:0/451770692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1654101be0 0x7f1654101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 -- 172.21.15.150:0/451770692 >> 172.21.15.150:0/451770692 conn(0x7f16540fc0d0 msgr2=0x7f16540fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 -- 172.21.15.150:0/451770692 shutdown_connections 2024-05-10T04:42:32.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 -- 172.21.15.150:0/451770692 wait complete. 2024-05-10T04:42:32.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 Processor -- start 2024-05-10T04:42:32.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.244+0000 7f165a426700 1 -- start start 2024-05-10T04:42:32.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f165a426700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1654100ac0 0x7f1654068e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.248 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f165a426700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1654101be0 0x7f165419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f165a426700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654102580 0x7f165419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f165a426700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165419fb40 con 0x7f1654102580 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f165a426700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165419fcb0 con 0x7f1654100ac0 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f165a426700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165419fe70 con 0x7f1654101be0 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654102580 0x7f165419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654102580 0x7f165419f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37936/0 (socket says 172.21.15.150:37936) 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f1653fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1654100ac0 0x7f1654068e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 -- 172.21.15.150:0/3813334984 learned_addr learned my addr 172.21.15.150:0/3813334984 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f164b7fe700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1654101be0 0x7f165419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 -- 172.21.15.150:0/3813334984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1654101be0 msgr2=0x7f165419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1654101be0 0x7f165419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 -- 172.21.15.150:0/3813334984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1654100ac0 msgr2=0x7f1654068e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1654100ac0 0x7f1654068e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 -- 172.21.15.150:0/3813334984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644009d00 con 0x7f1654102580 2024-05-10T04:42:32.249 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f16589c3700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654102580 0x7f165419f440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f164c00e940 tx=0x7f164c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f1651ffb700 1 -- 172.21.15.150:0/3813334984 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c0097b0 con 0x7f1654102580 2024-05-10T04:42:32.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16541a0150 con 0x7f1654102580 2024-05-10T04:42:32.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f1651ffb700 1 -- 172.21.15.150:0/3813334984 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164c005020 con 0x7f1654102580 2024-05-10T04:42:32.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.245+0000 7f1651ffb700 1 -- 172.21.15.150:0/3813334984 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c00cc40 con 0x7f1654102580 2024-05-10T04:42:32.250 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.246+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16541a0610 con 0x7f1654102580 2024-05-10T04:42:32.252 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.248+0000 7f1651ffb700 1 -- 172.21.15.150:0/3813334984 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f164c005400 con 0x7f1654102580 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.248+0000 7f1651ffb700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16400a11c0 0x7f16400a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.248+0000 7f1653fff700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16400a11c0 0x7f16400a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.248+0000 7f1651ffb700 1 -- 172.21.15.150:0/3813334984 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f164c014070 con 0x7f1654102580 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f1653fff700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16400a11c0 0x7f16400a36d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1644007bc0 tx=0x7f164400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 Processor -- start 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 -- start start 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333d720 0x55dc0333db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0333ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333ef40 0x55dc03345840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc032641d0 con 0x55dc0333d720 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc03261b90 con 0x55dc0333ef40 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f166a76ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc0325d480 con 0x55dc0333e5f0 2024-05-10T04:42:32.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f1653fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333d720 0x55dc0333db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f1653fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333d720 0x55dc0333db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37952/0 (socket says 172.21.15.150:37952) 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f1653fff700 1 -- 172.21.15.150:0/482693906 learned_addr learned my addr 172.21.15.150:0/482693906 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.249+0000 7f164b7fe700 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0333ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f16589c3700 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333ef40 0x55dc03345840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f1653fff700 1 -- 172.21.15.150:0/482693906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 msgr2=0x55dc0333ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f1653fff700 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0333ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f1653fff700 1 -- 172.21.15.150:0/482693906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333ef40 msgr2=0x55dc03345840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f1653fff700 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333ef40 0x55dc03345840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f1653fff700 1 -- 172.21.15.150:0/482693906 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc030de430 con 0x55dc0333d720 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f1653fff700 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333d720 0x55dc0333db70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f164400b040 tx=0x7f164401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dda0d1475e24b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f164a7fc700 1 -- 172.21.15.150:0/482693906 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400e040 con 0x55dc0333d720 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f164a7fc700 1 -- 172.21.15.150:0/482693906 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1644009b80 con 0x55dc0333d720 2024-05-10T04:42:32.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.250+0000 7f164a7fc700 1 -- 172.21.15.150:0/482693906 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644005040 con 0x55dc0333d720 2024-05-10T04:42:32.255 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-10T04:42:32.255 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-10T04:42:32.266 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 -- 172.21.15.150:0/482693906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333d720 msgr2=0x55dc0333db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333d720 0x55dc0333db70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f164400b040 tx=0x7f164401c000 comp rx=0 tx=0).stop 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 -- 172.21.15.150:0/482693906 shutdown_connections 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333ef40 0x55dc03345840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333d720 0x55dc0333db70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 --2- 172.21.15.150:0/482693906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0333ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 -- 172.21.15.150:0/482693906 >> 172.21.15.150:0/482693906 conn(0x55dc0329fae0 msgr2=0x55dc03338060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 -- 172.21.15.150:0/482693906 shutdown_connections 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.251+0000 7f166a76ab80 1 -- 172.21.15.150:0/482693906 wait complete. 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 -- start start 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 Processor -- start 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03347730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc03347c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc031f95e0 con 0x55dc0333e5f0 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc031f9280 con 0x55dc0333d720 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc030e0720 con 0x55dc0333ef40 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f166a76ab80 10 client.0 _ll_register_callbacks cb 0x55dc03202bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f164b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc03347c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f1653fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03347730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f164b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc03347c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37966/0 (socket says 172.21.15.150:37966) 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f1653fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03347730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:50326/0 (socket says 172.21.15.150:50326) 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f164b7fe700 1 -- 172.21.15.150:0/1308040377 learned_addr learned my addr 172.21.15.150:0/1308040377 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.252+0000 7f16589c3700 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f164b7fe700 1 -- 172.21.15.150:0/1308040377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 msgr2=0x55dc0334ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f164b7fe700 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f164b7fe700 1 -- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 msgr2=0x55dc03347730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f164b7fe700 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03347730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f164b7fe700 1 -- 172.21.15.150:0/1308040377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16541098c0 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f1653fff700 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03347730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f164b7fe700 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc03347c70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f163c012820 tx=0x7f163c012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8dd8aad2e035769 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f16337fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f16337fe700 1 -- 172.21.15.150:0/1308040377 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c017070 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f16337fe700 1 -- 172.21.15.150:0/1308040377 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f163c00f540 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc0330c800 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f16337fe700 1 -- 172.21.15.150:0/1308040377 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f163c00f700 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 10 client.16644 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dc0308abe0 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f16337fe700 1 -- 172.21.15.150:0/1308040377 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc0330c800 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f16337fe700 1 -- 172.21.15.150:0/1308040377 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f163c0108a0 con 0x55dc0333e5f0 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 10 client.16644 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:32.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 20 client.16644 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc03353410 0x55dc03355860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 4 client.16644 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.253+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc03255f40 con 0x55dc03353410 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.254+0000 7f1653fff700 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc03353410 0x55dc03355860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.254+0000 7f1653fff700 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc03353410 0x55dc03355860 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74fa4a0c643f8d59 server_cookie=b9d9ff7046bd33d9 in_seq=0 out_seq=0 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.254+0000 7f16337fe700 10 client.16644 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f16337fe700 1 -- 172.21.15.150:0/1308040377 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc03255f40 con 0x55dc03353410 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f16337fe700 10 client.16644 handle_command_reply: tid=0 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 client.16644 shutdown 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 msgr2=0x55dc03347c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc03347c70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f163c012820 tx=0x7f163c012b30 comp rx=0 tx=0).stop 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 shutdown_connections 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc03353410 0x55dc03355860 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f16337fe700 0 client.16644 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03347730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1308040377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc03347c70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 >> 172.21.15.150:0/1308040377 conn(0x55dc0329fae0 msgr2=0x55dc03338030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 shutdown_connections 2024-05-10T04:42:32.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 1 -- 172.21.15.150:0/1308040377 wait complete. 2024-05-10T04:42:32.275 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.255+0000 7f166a76ab80 20 client.16644 trim_cache size 0 max 16384 2024-05-10T04:42:32.276 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-10T04:42:32.287 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-10T04:42:32.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 Processor -- start 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 -- start start 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03342450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0334eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333ef40 0x55dc03342990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc033457f0 con 0x55dc0333ef40 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc03348330 con 0x55dc0333d720 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f166a76ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc03334200 con 0x55dc0333e5f0 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333ef40 0x55dc03342990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333ef40 0x55dc03342990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37972/0 (socket says 172.21.15.150:37972) 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 -- 172.21.15.150:0/3617068007 learned_addr learned my addr 172.21.15.150:0/3617068007 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f1653fff700 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03342450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f164b7fe700 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0334eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.294 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 -- 172.21.15.150:0/3617068007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 msgr2=0x55dc0334eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0334eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 -- 172.21.15.150:0/3617068007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 msgr2=0x55dc03342450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03342450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.273+0000 7f16589c3700 1 -- 172.21.15.150:0/3617068007 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc033344d0 con 0x55dc0333ef40 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.274+0000 7f16589c3700 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333ef40 0x55dc03342990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f164c009590 tx=0x7f164c096260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee527fe19b4d2247 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.274+0000 7f164affd700 1 -- 172.21.15.150:0/3617068007 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c099ea0 con 0x55dc0333ef40 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.274+0000 7f164affd700 1 -- 172.21.15.150:0/3617068007 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164c09dcd0 con 0x55dc0333ef40 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.274+0000 7f164affd700 1 -- 172.21.15.150:0/3617068007 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c099390 con 0x55dc0333ef40 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 -- 172.21.15.150:0/3617068007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333ef40 msgr2=0x55dc03342990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333ef40 0x55dc03342990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f164c009590 tx=0x7f164c096260 comp rx=0 tx=0).stop 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 -- 172.21.15.150:0/3617068007 shutdown_connections 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333ef40 0x55dc03342990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc03342450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 --2- 172.21.15.150:0/3617068007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333e5f0 0x55dc0334eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 -- 172.21.15.150:0/3617068007 >> 172.21.15.150:0/3617068007 conn(0x55dc0329fae0 msgr2=0x55dc033476f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 -- 172.21.15.150:0/3617068007 shutdown_connections 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 -- 172.21.15.150:0/3617068007 wait complete. 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 -- start start 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.275+0000 7f166a76ab80 1 Processor -- start 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc0334d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc0334d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc0334ef00 con 0x55dc0333e5f0 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc033a98e0 con 0x55dc0333d720 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc033a9a20 con 0x55dc0333ef40 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 10 client.0 _ll_register_callbacks cb 0x55dc03202bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc0334d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc0334d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37976/0 (socket says 172.21.15.150:37976) 2024-05-10T04:42:32.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 -- 172.21.15.150:0/1687361931 learned_addr learned my addr 172.21.15.150:0/1687361931 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f1653fff700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc0334d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f16589c3700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 -- 172.21.15.150:0/1687361931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 msgr2=0x55dc0334ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 -- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 msgr2=0x55dc0334d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc0334d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 -- 172.21.15.150:0/1687361931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16541a0610 con 0x55dc0333e5f0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f1653fff700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc0334d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f164b7fe700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc0334d870 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f163c012820 tx=0x7f163c010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164e95a1a9497f5d server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f16337fe700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f16337fe700 1 -- 172.21.15.150:0/1687361931 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c017040 con 0x55dc0333e5f0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f16337fe700 1 -- 172.21.15.150:0/1687361931 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f163c004ae0 con 0x55dc0333e5f0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc031b62f0 con 0x55dc0333e5f0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.276+0000 7f16337fe700 1 -- 172.21.15.150:0/1687361931 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f163c004ca0 con 0x55dc0333e5f0 2024-05-10T04:42:32.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 10 client.16650 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dc032235e0 con 0x55dc0333e5f0 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f16337fe700 1 -- 172.21.15.150:0/1687361931 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc031b62f0 con 0x55dc0333e5f0 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f16337fe700 1 -- 172.21.15.150:0/1687361931 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f163c01c5b0 con 0x55dc0333e5f0 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 20 client.16650 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc033ab320 0x55dc033ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 4 client.16650 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dc030abd70 con 0x55dc033ab320 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f1653fff700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc033ab320 0x55dc033ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f16337fe700 10 client.16650 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.277+0000 7f1653fff700 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc033ab320 0x55dc033ad740 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c47c89968923e2 server_cookie=1a217ffb322d0da8 in_seq=0 out_seq=0 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f16337fe700 1 -- 172.21.15.150:0/1687361931 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dc030abd70 con 0x55dc033ab320 2024-05-10T04:42:32.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f16337fe700 10 client.16650 handle_command_reply: tid=0 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 client.16650 shutdown 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 msgr2=0x55dc0334d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc0334d870 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f163c012820 tx=0x7f163c010b40 comp rx=0 tx=0).stop 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 shutdown_connections 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dc033ab320 0x55dc033ad740 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dc0333ef40 0x55dc0334ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dc0333d720 0x55dc0334d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 --2- 172.21.15.150:0/1687361931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc0333e5f0 0x55dc0334d870 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 >> 172.21.15.150:0/1687361931 conn(0x55dc0329fae0 msgr2=0x55dc033475b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f16337fe700 0 client.16650 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 shutdown_connections 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 1 -- 172.21.15.150:0/1687361931 wait complete. 2024-05-10T04:42:32.299 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.279+0000 7f166a76ab80 20 client.16650 trim_cache size 0 max 16384 2024-05-10T04:42:32.300 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:42:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "return_code": 0, 2024-05-10T04:42:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_tag": "9ee3322a-41c9-433b-a1e2-45cb9c58125c", 2024-05-10T04:42:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": "asynchronous" 2024-05-10T04:42:32.300 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:42:32.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16400a11c0 msgr2=0x7f16400a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16400a11c0 0x7f16400a36d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1644007bc0 tx=0x7f164400b700 comp rx=0 tx=0).stop 2024-05-10T04:42:32.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654102580 msgr2=0x7f165419f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654102580 0x7f165419f440 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f164c00e940 tx=0x7f164c00ec50 comp rx=0 tx=0).stop 2024-05-10T04:42:32.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 shutdown_connections 2024-05-10T04:42:32.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1654102580 0x7f165419f440 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1654100ac0 0x7f1654068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f16400a11c0 0x7f16400a36d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 --2- 172.21.15.150:0/3813334984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1654101be0 0x7f165419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 >> 172.21.15.150:0/3813334984 conn(0x7f16540fc0d0 msgr2=0x7f1654104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 shutdown_connections 2024-05-10T04:42:32.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.280+0000 7f165a426700 1 -- 172.21.15.150:0/3813334984 wait complete. 2024-05-10T04:42:32.302 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:42:32.480 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.478+0000 7f17cadd8700 1 -- 172.21.15.150:0/2819451439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 msgr2=0x7f17c4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.478+0000 7f17cadd8700 1 --2- 172.21.15.150:0/2819451439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c4108e30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00b0a0 tx=0x7f17bc00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.478+0000 7f17cadd8700 1 -- 172.21.15.150:0/2819451439 shutdown_connections 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.478+0000 7f17cadd8700 1 --2- 172.21.15.150:0/2819451439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c4108e30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.479+0000 7f17cadd8700 1 --2- 172.21.15.150:0/2819451439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c4100ac0 0x7f17c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.479+0000 7f17cadd8700 1 --2- 172.21.15.150:0/2819451439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17c4101be0 0x7f17c4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.479+0000 7f17cadd8700 1 -- 172.21.15.150:0/2819451439 >> 172.21.15.150:0/2819451439 conn(0x7f17c40fc0d0 msgr2=0x7f17c40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.479+0000 7f17cadd8700 1 -- 172.21.15.150:0/2819451439 shutdown_connections 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.479+0000 7f17cadd8700 1 -- 172.21.15.150:0/2819451439 wait complete. 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.479+0000 7f17cadd8700 1 Processor -- start 2024-05-10T04:42:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.479+0000 7f17cadd8700 1 -- start start 2024-05-10T04:42:32.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17cadd8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c4100ac0 0x7f17c4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17cadd8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17c4101be0 0x7f17c4197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17cadd8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17cadd8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c419ea00 con 0x7f17c4102580 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17cadd8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c419eb70 con 0x7f17c4101be0 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17cadd8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c419ed30 con 0x7f17c4100ac0 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37984/0 (socket says 172.21.15.150:37984) 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 -- 172.21.15.150:0/4245986902 learned_addr learned my addr 172.21.15.150:0/4245986902 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c3fff700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17c4101be0 0x7f17c4197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c8b74700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c4100ac0 0x7f17c4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 -- 172.21.15.150:0/4245986902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c4100ac0 msgr2=0x7f17c4196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c4100ac0 0x7f17c4196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 -- 172.21.15.150:0/4245986902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17c4101be0 msgr2=0x7f17c4197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17c4101be0 0x7f17c4197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 -- 172.21.15.150:0/4245986902 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc009d00 con 0x7f17c4102580 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c9375700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c419e3f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f17bc004c80 tx=0x7f17bc004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.480+0000 7f17c1ffb700 1 -- 172.21.15.150:0/4245986902 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00e040 con 0x7f17c4102580 2024-05-10T04:42:32.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.481+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c419efb0 con 0x7f17c4102580 2024-05-10T04:42:32.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.481+0000 7f17c1ffb700 1 -- 172.21.15.150:0/4245986902 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17bc00ce50 con 0x7f17c4102580 2024-05-10T04:42:32.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.481+0000 7f17c1ffb700 1 -- 172.21.15.150:0/4245986902 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc01b9b0 con 0x7f17c4102580 2024-05-10T04:42:32.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.481+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c419f380 con 0x7f17c4102580 2024-05-10T04:42:32.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.483+0000 7f17c1ffb700 1 -- 172.21.15.150:0/4245986902 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f17bc01dc50 con 0x7f17c4102580 2024-05-10T04:42:32.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.484+0000 7f17c1ffb700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17b40a11c0 0x7f17b40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.484+0000 7f17c8b74700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17b40a11c0 0x7f17b40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.484+0000 7f17c1ffb700 1 -- 172.21.15.150:0/4245986902 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f17bc0ce8e0 con 0x7f17c4102580 2024-05-10T04:42:32.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17b40a11c0 0x7f17b40a36d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005950 tx=0x7f17b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 Processor -- start 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 -- start start 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fbdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fbe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbeec0 0x55ba67fc57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67ee4150 con 0x55ba67fbd6a0 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67ee1b10 con 0x55ba67fbeec0 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17db11cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67edd400 con 0x55ba67fbe570 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fbdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fbdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37994/0 (socket says 172.21.15.150:37994) 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 -- 172.21.15.150:0/1089027412 learned_addr learned my addr 172.21.15.150:0/1089027412 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c3fff700 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fbe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 -- 172.21.15.150:0/1089027412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 msgr2=0x55ba67fbe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fbe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 -- 172.21.15.150:0/1089027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbeec0 msgr2=0x55ba67fc57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c9375700 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbeec0 0x55ba67fc57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbeec0 0x55ba67fc57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.485+0000 7f17c8b74700 1 -- 172.21.15.150:0/1089027412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba67d5e3b0 con 0x55ba67fbd6a0 2024-05-10T04:42:32.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.486+0000 7f17c8b74700 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fbdaf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f17b800fd00 tx=0x7f17b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8722e03168d078f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.486+0000 7f17aeffd700 1 -- 172.21.15.150:0/1089027412 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8011c10 con 0x55ba67fbd6a0 2024-05-10T04:42:32.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.486+0000 7f17aeffd700 1 -- 172.21.15.150:0/1089027412 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b8011d70 con 0x55ba67fbd6a0 2024-05-10T04:42:32.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.486+0000 7f17aeffd700 1 -- 172.21.15.150:0/1089027412 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801d610 con 0x55ba67fbd6a0 2024-05-10T04:42:32.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 -- 172.21.15.150:0/1089027412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 msgr2=0x55ba67fbdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fbdaf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f17b800fd00 tx=0x7f17b8012000 comp rx=0 tx=0).stop 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 -- 172.21.15.150:0/1089027412 shutdown_connections 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbeec0 0x55ba67fc57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fbdaf0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 --2- 172.21.15.150:0/1089027412 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fbe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 -- 172.21.15.150:0/1089027412 >> 172.21.15.150:0/1089027412 conn(0x55ba67f1fa60 msgr2=0x55ba67fb7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 -- 172.21.15.150:0/1089027412 shutdown_connections 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 -- 172.21.15.150:0/1089027412 wait complete. 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 -- start start 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.487+0000 7f17db11cb80 1 Processor -- start 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fc9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbeec0 0x55ba67fd0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17db11cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67e79560 con 0x55ba67fbeec0 2024-05-10T04:42:32.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17db11cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67e79200 con 0x55ba67fbd6a0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17db11cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67d606a0 con 0x55ba67fbe570 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17db11cb80 10 client.0 _ll_register_callbacks cb 0x55ba67e82b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:50380/0 (socket says 172.21.15.150:50380) 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 -- 172.21.15.150:0/2207599377 learned_addr learned my addr 172.21.15.150:0/2207599377 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c3fff700 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fc9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c9375700 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbeec0 0x55ba67fd0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 -- 172.21.15.150:0/2207599377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 msgr2=0x55ba67fc9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fc9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 -- 172.21.15.150:0/2207599377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbeec0 msgr2=0x55ba67fd0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbeec0 0x55ba67fd0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.488+0000 7f17c8b74700 1 -- 172.21.15.150:0/2207599377 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b800f9b0 con 0x55ba67fbd6a0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17c8b74700 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc98d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b9e0 tx=0x7f17b8005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7101290d42598290 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-05-10T04:42:32.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17a3fff700 1 -- 172.21.15.150:0/2207599377 <== mon.2 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b802e070 con 0x55ba67fbd6a0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba67e36270 con 0x55ba67fbd6a0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17a3fff700 1 -- 172.21.15.150:0/2207599377 <== mon.2 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f17b8004710 con 0x55ba67fbd6a0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17a3fff700 1 -- 172.21.15.150:0/2207599377 <== mon.2 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b80056a0 con 0x55ba67fbd6a0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17db11cb80 10 client.26267 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ba67d0ab60 con 0x55ba67fbd6a0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17a3fff700 1 -- 172.21.15.150:0/2207599377 <== mon.2 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba67e36270 con 0x55ba67fbd6a0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17a3fff700 1 -- 172.21.15.150:0/2207599377 <== mon.2 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17b8027450 con 0x55ba67fbd6a0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17db11cb80 10 client.26267 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.489+0000 7f17db11cb80 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.490+0000 7f17db11cb80 20 client.26267 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.490+0000 7f17db11cb80 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba67fc8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.490+0000 7f17db11cb80 4 client.26267 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.490+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba67f8c780 con 0x55ba67fc8b10 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.490+0000 7f17c3fff700 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba67fc8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.490+0000 7f17c3fff700 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba67fc8f50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d23aefc2ede3738 server_cookie=f9456bc014383753 in_seq=0 out_seq=0 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.490+0000 7f17a3fff700 10 client.26267 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17a3fff700 1 -- 172.21.15.150:0/2207599377 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba67f8c780 con 0x55ba67fc8b10 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17a3fff700 10 client.26267 handle_command_reply: tid=0 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 client.26267 shutdown 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 msgr2=0x55ba67fc98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc98d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b9e0 tx=0x7f17b8005270 comp rx=0 tx=0).stop 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 shutdown_connections 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba67fc8f50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbeec0 0x55ba67fd0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17a3fff700 0 client.26267 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc98d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 --2- 172.21.15.150:0/2207599377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbe570 0x55ba67fc9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 >> 172.21.15.150:0/2207599377 conn(0x55ba67f1fa60 msgr2=0x55ba67fb7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 shutdown_connections 2024-05-10T04:42:32.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 1 -- 172.21.15.150:0/2207599377 wait complete. 2024-05-10T04:42:32.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.491+0000 7f17db11cb80 20 client.26267 trim_cache size 0 max 16384 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 Processor -- start 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 -- start start 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbe570 0x55ba67fd0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fc59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67fd1eb0 con 0x55ba67fbd6a0 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67fda920 con 0x55ba67fbe570 2024-05-10T04:42:32.512 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17db11cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67fc2d30 con 0x55ba67fbeec0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38020/0 (socket says 172.21.15.150:38020) 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 -- 172.21.15.150:0/4202980085 learned_addr learned my addr 172.21.15.150:0/4202980085 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c9375700 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fc59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c3fff700 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbe570 0x55ba67fd0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 -- 172.21.15.150:0/4202980085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 msgr2=0x55ba67fc59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fc59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 -- 172.21.15.150:0/4202980085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbe570 msgr2=0x55ba67fd0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbe570 0x55ba67fd0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 -- 172.21.15.150:0/4202980085 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba67ea3560 con 0x55ba67fbd6a0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17c8b74700 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc17a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f17b80277b0 tx=0x7f17b8005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fe02ad8cccec6e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.509+0000 7f17af7fe700 1 -- 172.21.15.150:0/4202980085 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b802e040 con 0x55ba67fbd6a0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17af7fe700 1 -- 172.21.15.150:0/4202980085 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b80117d0 con 0x55ba67fbd6a0 2024-05-10T04:42:32.513 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17af7fe700 1 -- 172.21.15.150:0/4202980085 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801e450 con 0x55ba67fbd6a0 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 -- 172.21.15.150:0/4202980085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 msgr2=0x55ba67fc17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc17a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f17b80277b0 tx=0x7f17b8005390 comp rx=0 tx=0).stop 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 -- 172.21.15.150:0/4202980085 shutdown_connections 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fc59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbd6a0 0x55ba67fc17a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 --2- 172.21.15.150:0/4202980085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbe570 0x55ba67fd0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 -- 172.21.15.150:0/4202980085 >> 172.21.15.150:0/4202980085 conn(0x55ba67f1fa60 msgr2=0x55ba67fb8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 -- 172.21.15.150:0/4202980085 shutdown_connections 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.510+0000 7f17db11cb80 1 -- 172.21.15.150:0/4202980085 wait complete. 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 -- start start 2024-05-10T04:42:32.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 Processor -- start 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbe570 0x55ba67fcec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fcfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67fcffe0 con 0x55ba67fbe570 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67fd0150 con 0x55ba67fbd6a0 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67fd0310 con 0x55ba67fbeec0 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.511+0000 7f17db11cb80 10 client.0 _ll_register_callbacks cb 0x55ba67e82b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbe570 0x55ba67fcec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbe570 0x55ba67fcec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:38024/0 (socket says 172.21.15.150:38024) 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 -- 172.21.15.150:0/560258229 learned_addr learned my addr 172.21.15.150:0/560258229 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c8b74700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.518 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c9375700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fcfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 -- 172.21.15.150:0/560258229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 msgr2=0x55ba67fcfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fcfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 -- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 msgr2=0x55ba67fce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 -- 172.21.15.150:0/560258229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba67d0ab60 con 0x55ba67fbe570 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c9375700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fcfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17c3fff700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbe570 0x55ba67fcec10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f17b00046e0 tx=0x7f17b001d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34ef08599e88bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17a3fff700 1 -- 172.21.15.150:0/560258229 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0003680 con 0x55ba67fbe570 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17a3fff700 1 -- 172.21.15.150:0/560258229 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f17b0018040 con 0x55ba67fbe570 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba67ed5ec0 con 0x55ba67fbe570 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17a3fff700 1 -- 172.21.15.150:0/560258229 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b00037e0 con 0x55ba67fbe570 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17db11cb80 10 client.16668 fetch_fsmap learned FSMap version 29 2024-05-10T04:42:32.519 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.512+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ba67d62cd0 con 0x55ba67fbe570 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17a3fff700 1 -- 172.21.15.150:0/560258229 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba67ed5ec0 con 0x55ba67fbe570 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17a3fff700 1 -- 172.21.15.150:0/560258229 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f17b0012b80 con 0x55ba67fbe570 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17db11cb80 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17db11cb80 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17db11cb80 20 client.16668 populate_metadata read hostname 'smithi150' 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17db11cb80 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba6802c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17db11cb80 4 client.16668 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba67d2bcf0 con 0x55ba67fc8b10 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17c8b74700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba6802c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17a3fff700 10 client.16668 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.513+0000 7f17c8b74700 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba6802c170 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974a9df6ea6c475 server_cookie=10e61bfe22d76468 in_seq=0 out_seq=0 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17a3fff700 1 -- 172.21.15.150:0/560258229 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ba67d2bcf0 con 0x55ba67fc8b10 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17a3fff700 10 client.16668 handle_command_reply: tid=0 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 client.16668 shutdown 2024-05-10T04:42:32.520 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbe570 msgr2=0x55ba67fcec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbe570 0x55ba67fcec10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f17b00046e0 tx=0x7f17b001d650 comp rx=0 tx=0).stop 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 shutdown_connections 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55ba67fc8b10 0x55ba6802c170 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ba67fbeec0 0x55ba67fcfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ba67fbd6a0 0x55ba67fce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 --2- 172.21.15.150:0/560258229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ba67fbe570 0x55ba67fcec10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17a3fff700 0 client.16668 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 >> 172.21.15.150:0/560258229 conn(0x55ba67f1fa60 msgr2=0x55ba67fb8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 shutdown_connections 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 1 -- 172.21.15.150:0/560258229 wait complete. 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.514+0000 7f17db11cb80 20 client.16668 trim_cache size 0 max 16384 2024-05-10T04:42:32.521 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": { 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: "9ee3322a-41c9-433b-a1e2-45cb9c58125c": { 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: "path": "/", 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: "tag": "9ee3322a-41c9-433b-a1e2-45cb9c58125c", 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: "options": "recursive,force" 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:42:32.522 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17b40a11c0 msgr2=0x7f17b40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17b40a11c0 0x7f17b40a36d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005950 tx=0x7f17b80058e0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 msgr2=0x7f17c419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c419e3f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f17bc004c80 tx=0x7f17bc004cb0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 shutdown_connections 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4102580 0x7f17c419e3f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f17b40a11c0 0x7f17b40a36d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c4100ac0 0x7f17c4196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.523 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 --2- 172.21.15.150:0/4245986902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17c4101be0 0x7f17c4197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:42:32.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 >> 172.21.15.150:0/4245986902 conn(0x7f17c40fc0d0 msgr2=0x7f17c4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:42:32.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 shutdown_connections 2024-05-10T04:42:32.524 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:42:32.515+0000 7f17cadd8700 1 -- 172.21.15.150:0/4245986902 wait complete. 2024-05-10T04:42:32.529 INFO:tasks.cephfs.filesystem:scrub status for tag:9ee3322a-41c9-433b-a1e2-45cb9c58125c - {'path': '/', 'tag': '9ee3322a-41c9-433b-a1e2-45cb9c58125c', 'options': 'recursive,force'} 2024-05-10T04:42:33.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:32 smithi181 ceph-mon[47314]: scrub queued for path: / 2024-05-10T04:42:33.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:32 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:33.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:32 smithi181 ceph-mon[47314]: scrub summary: active paths [/] 2024-05-10T04:42:33.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:32 smithi181 ceph-mon[47314]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:33.155 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:32 smithi181 ceph-mon[47314]: scrub summary: idle 2024-05-10T04:42:33.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:32 smithi150 ceph-mon[42404]: scrub queued for path: / 2024-05-10T04:42:33.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:32 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:33.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:32 smithi150 ceph-mon[42404]: scrub summary: active paths [/] 2024-05-10T04:42:33.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:32 smithi150 ceph-mon[42404]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:33.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:32 smithi150 ceph-mon[42404]: scrub summary: idle 2024-05-10T04:42:33.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:32 smithi161 ceph-mon[46912]: scrub queued for path: / 2024-05-10T04:42:33.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:32 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:33.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:32 smithi161 ceph-mon[46912]: scrub summary: active paths [/] 2024-05-10T04:42:33.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:32 smithi161 ceph-mon[46912]: scrub summary: idle+waiting paths [/] 2024-05-10T04:42:33.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:32 smithi161 ceph-mon[46912]: scrub summary: idle 2024-05-10T04:42:34.154 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:33 smithi181 ceph-mon[47314]: pgmap v361: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.2 MiB/s wr, 16 op/s 2024-05-10T04:42:34.161 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:33 smithi150 ceph-mon[42404]: pgmap v361: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.2 MiB/s wr, 16 op/s 2024-05-10T04:42:34.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:33 smithi161 ceph-mon[46912]: pgmap v361: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.2 MiB/s wr, 16 op/s 2024-05-10T04:42:35.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:35 smithi161 ceph-mon[46912]: pgmap v362: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-10T04:42:35.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:35 smithi181 ceph-mon[47314]: pgmap v362: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-10T04:42:35.918 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:35 smithi150 ceph-mon[42404]: pgmap v362: 129 pgs: 129 active+clean; 258 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-10T04:42:37.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:37 smithi161 ceph-mon[46912]: pgmap v363: 129 pgs: 129 active+clean; 202 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-05-10T04:42:37.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:37 smithi181 ceph-mon[47314]: pgmap v363: 129 pgs: 129 active+clean; 202 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-05-10T04:42:37.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:37 smithi150 ceph-mon[42404]: pgmap v363: 129 pgs: 129 active+clean; 202 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 22 op/s 2024-05-10T04:42:39.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:39 smithi161 ceph-mon[46912]: pgmap v364: 129 pgs: 129 active+clean; 122 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:39.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:39 smithi181 ceph-mon[47314]: pgmap v364: 129 pgs: 129 active+clean; 122 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:39.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:39 smithi150 ceph-mon[42404]: pgmap v364: 129 pgs: 129 active+clean; 122 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 21 op/s 2024-05-10T04:42:41.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:41 smithi161 ceph-mon[46912]: pgmap v365: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-10T04:42:41.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:41 smithi181 ceph-mon[47314]: pgmap v365: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-10T04:42:41.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:41 smithi150 ceph-mon[42404]: pgmap v365: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-05-10T04:42:43.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:43 smithi161 ceph-mon[46912]: pgmap v366: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 26 op/s 2024-05-10T04:42:43.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:43 smithi181 ceph-mon[47314]: pgmap v366: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 26 op/s 2024-05-10T04:42:43.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:43 smithi150 ceph-mon[42404]: pgmap v366: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 26 op/s 2024-05-10T04:42:45.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:45 smithi161 ceph-mon[46912]: pgmap v367: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 MiB/s wr, 24 op/s 2024-05-10T04:42:45.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:45 smithi181 ceph-mon[47314]: pgmap v367: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 MiB/s wr, 24 op/s 2024-05-10T04:42:45.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:45 smithi150 ceph-mon[42404]: pgmap v367: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 MiB/s wr, 24 op/s 2024-05-10T04:42:47.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:47 smithi161 ceph-mon[46912]: pgmap v368: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 20 op/s 2024-05-10T04:42:47.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:47 smithi181 ceph-mon[47314]: pgmap v368: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 20 op/s 2024-05-10T04:42:47.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:47 smithi150 ceph-mon[42404]: pgmap v368: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 20 op/s 2024-05-10T04:42:49.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:49 smithi161 ceph-mon[46912]: pgmap v369: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 MiB/s wr, 15 op/s 2024-05-10T04:42:49.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:49 smithi181 ceph-mon[47314]: pgmap v369: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 MiB/s wr, 15 op/s 2024-05-10T04:42:49.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:49 smithi150 ceph-mon[42404]: pgmap v369: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 MiB/s wr, 15 op/s 2024-05-10T04:42:51.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:51 smithi161 ceph-mon[46912]: pgmap v370: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-10T04:42:51.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:51 smithi181 ceph-mon[47314]: pgmap v370: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-10T04:42:51.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:51 smithi150 ceph-mon[42404]: pgmap v370: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-10T04:42:53.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:53 smithi161 ceph-mon[46912]: pgmap v371: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T04:42:53.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:53 smithi181 ceph-mon[47314]: pgmap v371: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T04:42:53.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:53 smithi150 ceph-mon[42404]: pgmap v371: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-10T04:42:55.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:55 smithi161 ceph-mon[46912]: pgmap v372: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:42:55.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:55 smithi181 ceph-mon[47314]: pgmap v372: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:42:55.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:55 smithi150 ceph-mon[42404]: pgmap v372: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-10T04:42:57.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:57 smithi161 ceph-mon[46912]: pgmap v373: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:42:57.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:57 smithi181 ceph-mon[47314]: pgmap v373: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:42:57.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:57 smithi150 ceph-mon[42404]: pgmap v373: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-10T04:42:59.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:42:59 smithi161 ceph-mon[46912]: pgmap v374: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:42:59.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:42:59 smithi181 ceph-mon[47314]: pgmap v374: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:42:59.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:42:59 smithi150 ceph-mon[42404]: pgmap v374: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:43:01.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:01 smithi161 ceph-mon[46912]: pgmap v375: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:01.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:01 smithi181 ceph-mon[47314]: pgmap v375: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:01.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:01 smithi150 ceph-mon[42404]: pgmap v375: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:02.530 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-10T04:43:02.683 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 -- 172.21.15.150:0/1117939935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11081025a0 msgr2=0x7f1108108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 --2- 172.21.15.150:0/1117939935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11081025a0 0x7f1108108e50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f110000b0a0 tx=0x7f110000b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 -- 172.21.15.150:0/1117939935 shutdown_connections 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 --2- 172.21.15.150:0/1117939935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11081025a0 0x7f1108108e50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 --2- 172.21.15.150:0/1117939935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1108100ae0 0x7f1108100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 --2- 172.21.15.150:0/1117939935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1108101c00 0x7f1108102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 -- 172.21.15.150:0/1117939935 >> 172.21.15.150:0/1117939935 conn(0x7f11080fc0f0 msgr2=0x7f11080fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 -- 172.21.15.150:0/1117939935 shutdown_connections 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.680+0000 7f110f39c700 1 -- 172.21.15.150:0/1117939935 wait complete. 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 Processor -- start 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 -- start start 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1108100ae0 0x7f110806a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.684 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1108101c00 0x7f110819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11081025a0 0x7f110819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110819fa50 con 0x7f1108100ae0 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110819fbc0 con 0x7f11081025a0 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110f39c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110819fd80 con 0x7f1108101c00 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1108100ae0 0x7f110806a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1108100ae0 0x7f110806a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45276/0 (socket says 172.21.15.150:45276) 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 -- 172.21.15.150:0/2056216282 learned_addr learned my addr 172.21.15.150:0/2056216282 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110c937700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1108101c00 0x7f110819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d939700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11081025a0 0x7f110819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 -- 172.21.15.150:0/2056216282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1108101c00 msgr2=0x7f110819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1108101c00 0x7f110819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 -- 172.21.15.150:0/2056216282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11081025a0 msgr2=0x7f110819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11081025a0 0x7f110819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.685 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.681+0000 7f110d138700 1 -- 172.21.15.150:0/2056216282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100009d00 con 0x7f1108100ae0 2024-05-10T04:43:02.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.682+0000 7f110d138700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1108100ae0 0x7f110806a570 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c970 tx=0x7f10f800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.682+0000 7f10fe7fc700 1 -- 172.21.15.150:0/2056216282 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8007b00 con 0x7f1108100ae0 2024-05-10T04:43:02.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.682+0000 7f10fe7fc700 1 -- 172.21.15.150:0/2056216282 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10f8007c60 con 0x7f1108100ae0 2024-05-10T04:43:02.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.682+0000 7f10fe7fc700 1 -- 172.21.15.150:0/2056216282 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8018660 con 0x7f1108100ae0 2024-05-10T04:43:02.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.682+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11081a0060 con 0x7f1108100ae0 2024-05-10T04:43:02.686 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.682+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11081a0520 con 0x7f1108100ae0 2024-05-10T04:43:02.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.685+0000 7f10fe7fc700 1 -- 172.21.15.150:0/2056216282 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f10f800f450 con 0x7f1108100ae0 2024-05-10T04:43:02.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.685+0000 7f10fe7fc700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10f40a11c0 0x7f10f40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.685+0000 7f110c937700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10f40a11c0 0x7f10f40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.685+0000 7f10fe7fc700 1 -- 172.21.15.150:0/2056216282 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f10f80ccca0 con 0x7f1108100ae0 2024-05-10T04:43:02.687 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f110c937700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10f40a11c0 0x7f10f40a36d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1104005950 tx=0x7f11040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 Processor -- start 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 -- start start 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb89506a0 0x55dcb8950af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951570 0x55dcb8951980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951ec0 0x55dcb89587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb8877150 con 0x55dcb8951570 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb8874b10 con 0x55dcb89506a0 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.686+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb8870400 con 0x55dcb8951ec0 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951570 0x55dcb8951980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951570 0x55dcb8951980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45286/0 (socket says 172.21.15.150:45286) 2024-05-10T04:43:02.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 -- 172.21.15.150:0/1811050908 learned_addr learned my addr 172.21.15.150:0/1811050908 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110d138700 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb89506a0 0x55dcb8950af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110d939700 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951ec0 0x55dcb89587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 -- 172.21.15.150:0/1811050908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951ec0 msgr2=0x55dcb89587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951ec0 0x55dcb89587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 -- 172.21.15.150:0/1811050908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb89506a0 msgr2=0x55dcb8950af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb89506a0 0x55dcb8950af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 -- 172.21.15.150:0/1811050908 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcb86f13b0 con 0x55dcb8951570 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f110c937700 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951570 0x55dcb8951980 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f110400fd40 tx=0x7f1104012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e3d60df467d2d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f10f37fe700 1 -- 172.21.15.150:0/1811050908 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104011c60 con 0x55dcb8951570 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f10f37fe700 1 -- 172.21.15.150:0/1811050908 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1104011dc0 con 0x55dcb8951570 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.687+0000 7f10f37fe700 1 -- 172.21.15.150:0/1811050908 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110401d7d0 con 0x55dcb8951570 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1811050908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951570 msgr2=0x55dcb8951980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951570 0x55dcb8951980 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f110400fd40 tx=0x7f1104012000 comp rx=0 tx=0).stop 2024-05-10T04:43:02.696 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1811050908 shutdown_connections 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951ec0 0x55dcb89587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb89506a0 0x55dcb8950af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1811050908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951570 0x55dcb8951980 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1811050908 >> 172.21.15.150:0/1811050908 conn(0x55dcb88b2a60 msgr2=0x55dcb894afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1811050908 shutdown_connections 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1811050908 wait complete. 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.688+0000 7f111f6e0b80 1 -- start start 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 1 Processor -- start 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb895a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb895abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951ec0 0x55dcb8961b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb880c560 con 0x55dcb8951ec0 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb880c200 con 0x55dcb8951570 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb86f36a0 con 0x55dcb89506a0 2024-05-10T04:43:02.697 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f111f6e0b80 10 client.0 _ll_register_callbacks cb 0x55dcb8815b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d939700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951ec0 0x55dcb8961b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d939700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951ec0 0x55dcb8961b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45302/0 (socket says 172.21.15.150:45302) 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d939700 1 -- 172.21.15.150:0/1254412881 learned_addr learned my addr 172.21.15.150:0/1254412881 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d138700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb895a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110c937700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb895abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d939700 1 -- 172.21.15.150:0/1254412881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 msgr2=0x55dcb895a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d939700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb895a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d939700 1 -- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 msgr2=0x55dcb895abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.689+0000 7f110d939700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb895abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f110d939700 1 -- 172.21.15.150:0/1254412881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110400f9f0 con 0x55dcb8951ec0 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f110c937700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb895abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f110d939700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951ec0 0x55dcb8961b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1100015000 tx=0x7f11000092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3492a9083f1a87c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f10f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:02.698 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1254412881 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110000e040 con 0x55dcb8951ec0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1254412881 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1100005040 con 0x55dcb8951ec0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcb87c9270 con 0x55dcb8951ec0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1254412881 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1100004e60 con 0x55dcb8951ec0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 10 client.16680 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dcb869db60 con 0x55dcb8951ec0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1254412881 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcb87c9270 con 0x55dcb8951ec0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1254412881 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f110000b0d0 con 0x55dcb8951ec0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 10 client.16680 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 20 client.16680 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb89662f0 0x55dcb8968740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 4 client.16680 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.690+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcb891f780 con 0x55dcb89662f0 2024-05-10T04:43:02.699 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.691+0000 7f110d138700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb89662f0 0x55dcb8968740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.691+0000 7f110d138700 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb89662f0 0x55dcb8968740 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142593250d2a165 server_cookie=ad8f0d81df462ccc in_seq=0 out_seq=0 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.691+0000 7f10f0ff9700 10 client.16680 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1254412881 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dcb891f780 con 0x55dcb89662f0 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f10f0ff9700 10 client.16680 handle_command_reply: tid=0 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 client.16680 shutdown 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951ec0 msgr2=0x55dcb8961b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951ec0 0x55dcb8961b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1100015000 tx=0x7f11000092e0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 shutdown_connections 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb89662f0 0x55dcb8968740 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb8951ec0 0x55dcb8961b40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f10f0ff9700 0 client.16680 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb895a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1254412881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb895abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.700 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 >> 172.21.15.150:0/1254412881 conn(0x55dcb88b2a60 msgr2=0x55dcb894afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:02.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 shutdown_connections 2024-05-10T04:43:02.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1254412881 wait complete. 2024-05-10T04:43:02.701 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.692+0000 7f111f6e0b80 20 client.16680 trim_cache size 0 max 16384 2024-05-10T04:43:02.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 Processor -- start 2024-05-10T04:43:02.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 -- start start 2024-05-10T04:43:02.716 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb88b2a60 0x55dcb895a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb89506a0 0x55dcb895abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951570 0x55dcb8949c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb8950f50 con 0x55dcb89506a0 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb8961ef0 con 0x55dcb88b2a60 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.709+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb8958670 con 0x55dcb8951570 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb89506a0 0x55dcb895abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb89506a0 0x55dcb895abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45318/0 (socket says 172.21.15.150:45318) 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 -- 172.21.15.150:0/1197333069 learned_addr learned my addr 172.21.15.150:0/1197333069 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110d138700 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb88b2a60 0x55dcb895a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110d939700 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951570 0x55dcb8949c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 -- 172.21.15.150:0/1197333069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951570 msgr2=0x55dcb8949c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951570 0x55dcb8949c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 -- 172.21.15.150:0/1197333069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb88b2a60 msgr2=0x55dcb895a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb88b2a60 0x55dcb895a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 -- 172.21.15.150:0/1197333069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcb8836560 con 0x55dcb89506a0 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110d138700 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb88b2a60 0x55dcb895a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f110c937700 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb89506a0 0x55dcb895abe0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f110400b860 tx=0x7f1104011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed75d915f244ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f10f3fff700 1 -- 172.21.15.150:0/1197333069 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104005040 con 0x55dcb89506a0 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f10f3fff700 1 -- 172.21.15.150:0/1197333069 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110402f040 con 0x55dcb89506a0 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.710+0000 7f10f3fff700 1 -- 172.21.15.150:0/1197333069 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110401e450 con 0x55dcb89506a0 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1197333069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb89506a0 msgr2=0x55dcb895abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb89506a0 0x55dcb895abe0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f110400b860 tx=0x7f1104011d50 comp rx=0 tx=0).stop 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1197333069 shutdown_connections 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb8951570 0x55dcb8949c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb88b2a60 0x55dcb895a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1197333069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb89506a0 0x55dcb895abe0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1197333069 >> 172.21.15.150:0/1197333069 conn(0x55dcb8947450 msgr2=0x55dcb8955350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1197333069 shutdown_connections 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1197333069 wait complete. 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 -- start start 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.711+0000 7f111f6e0b80 1 Processor -- start 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb88b2a60 0x55dcb89b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb89b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f111f6e0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb89bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb89c0170 con 0x55dcb88b2a60 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb89c02b0 con 0x55dcb8951570 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f111f6e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcb89c0440 con 0x55dcb89506a0 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f111f6e0b80 10 client.0 _ll_register_callbacks cb 0x55dcb8815b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb88b2a60 0x55dcb89b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb88b2a60 0x55dcb89b8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45328/0 (socket says 172.21.15.150:45328) 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 -- 172.21.15.150:0/1138525170 learned_addr learned my addr 172.21.15.150:0/1138525170 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:02.719 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110c937700 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb89b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d939700 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb89bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 -- 172.21.15.150:0/1138525170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 msgr2=0x55dcb89b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb89b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 -- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 msgr2=0x55dcb89bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb89bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 -- 172.21.15.150:0/1138525170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcb86f13b0 con 0x55dcb88b2a60 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f110d138700 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb88b2a60 0x55dcb89b8870 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f10f8096250 tx=0x7f10f800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594045373000cab1 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f10f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.712+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1138525170 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f80a21d0 con 0x55dcb88b2a60 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1138525170 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f10f80980f0 con 0x55dcb88b2a60 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1104011dc0 con 0x55dcb88b2a60 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1138525170 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10f80b8150 con 0x55dcb88b2a60 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 10 client.16686 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:02.720 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dcb86f5cd0 con 0x55dcb88b2a60 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1138525170 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1104011dc0 con 0x55dcb88b2a60 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1138525170 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f10f80a1d40 con 0x55dcb88b2a60 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 10 client.16686 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 20 client.16686 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb8951ec0 0x55dcb89c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 4 client.16686 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dcb8868ec0 con 0x55dcb8951ec0 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.713+0000 7f110c937700 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb8951ec0 0x55dcb89c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f10f0ff9700 10 client.16686 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f110c937700 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb8951ec0 0x55dcb89c3de0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f78b6e8fe7fd4f0 server_cookie=6946d59c2dca4753 in_seq=0 out_seq=0 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f10f0ff9700 1 -- 172.21.15.150:0/1138525170 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dcb8868ec0 con 0x55dcb8951ec0 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f10f0ff9700 10 client.16686 handle_command_reply: tid=0 2024-05-10T04:43:02.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f111f6e0b80 1 client.16686 shutdown 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb88b2a60 msgr2=0x55dcb89b8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb88b2a60 0x55dcb89b8870 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f10f8096250 tx=0x7f10f800cf70 comp rx=0 tx=0).stop 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.714+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 shutdown_connections 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x55dcb8951ec0 0x55dcb89c3de0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dcb8951570 0x55dcb89bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f10f0ff9700 0 client.16686 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcb88b2a60 0x55dcb89b8870 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 1 --2- 172.21.15.150:0/1138525170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55dcb89506a0 0x55dcb89b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 >> 172.21.15.150:0/1138525170 conn(0x55dcb8947450 msgr2=0x55dcb894f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 shutdown_connections 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 1 -- 172.21.15.150:0/1138525170 wait complete. 2024-05-10T04:43:02.722 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f111f6e0b80 20 client.16686 trim_cache size 0 max 16384 2024-05-10T04:43:02.723 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:43:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "status": "no active scrubs running", 2024-05-10T04:43:02.723 INFO:teuthology.orchestra.run.smithi150.stdout: "scrubs": {} 2024-05-10T04:43:02.723 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:43:02.723 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10f40a11c0 msgr2=0x7f10f40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10f40a11c0 0x7f10f40a36d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1104005950 tx=0x7f11040058e0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1108100ae0 msgr2=0x7f110806a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1108100ae0 0x7f110806a570 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c970 tx=0x7f10f800cd30 comp rx=0 tx=0).stop 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 shutdown_connections 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11081025a0 0x7f110819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1108100ae0 0x7f110806a570 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f10f40a11c0 0x7f10f40a36d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 --2- 172.21.15.150:0/2056216282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1108101c00 0x7f110819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 >> 172.21.15.150:0/2056216282 conn(0x7f11080fc0f0 msgr2=0x7f1108104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 shutdown_connections 2024-05-10T04:43:02.724 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.715+0000 7f110f39c700 1 -- 172.21.15.150:0/2056216282 wait complete. 2024-05-10T04:43:02.729 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-10T04:43:02.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 -- 172.21.15.150:0/349636587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 msgr2=0x7f8ccc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/349636587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc100030 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009990 tx=0x7f8cbc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 -- 172.21.15.150:0/349636587 shutdown_connections 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/349636587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc100570 0x7f8ccc10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/349636587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc1031e0 0x7f8ccc1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/349636587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc100030 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 -- 172.21.15.150:0/349636587 >> 172.21.15.150:0/349636587 conn(0x7f8ccc0fb8f0 msgr2=0x7f8ccc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 -- 172.21.15.150:0/349636587 shutdown_connections 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.862+0000 7f8cd29f1700 1 -- 172.21.15.150:0/349636587 wait complete. 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 Processor -- start 2024-05-10T04:43:02.866 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 -- start start 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc100570 0x7f8ccc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc1031e0 0x7f8ccc19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc19fa70 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc19fbb0 con 0x7f8ccc1031e0 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd29f1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc19fd40 con 0x7f8ccc100570 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45348/0 (socket says 172.21.15.150:45348) 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 -- 172.21.15.150:0/939899549 learned_addr learned my addr 172.21.15.150:0/939899549 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccb7fe700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc100570 0x7f8ccc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd0f8e700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc1031e0 0x7f8ccc19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 -- 172.21.15.150:0/939899549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc100570 msgr2=0x7f8ccc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc100570 0x7f8ccc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.867 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 -- 172.21.15.150:0/939899549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc1031e0 msgr2=0x7f8ccc19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc1031e0 0x7f8ccc19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8ccbfff700 1 -- 172.21.15.150:0/939899549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0095f0 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.863+0000 7f8cd0f8e700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc1031e0 0x7f8ccc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.864+0000 7f8ccbfff700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc197fd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c9a0 tx=0x7f8cc000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.864+0000 7f8cc97fa700 1 -- 172.21.15.150:0/939899549 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0007ac0 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.864+0000 7f8cc97fa700 1 -- 172.21.15.150:0/939899549 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cc0007c20 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.864+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc19ffc0 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.864+0000 7f8cc97fa700 1 -- 172.21.15.150:0/939899549 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0018660 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.864+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1a0470 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.865+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc109030 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.866+0000 7f8cc97fa700 1 -- 172.21.15.150:0/939899549 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f8cc000f450 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.867+0000 7f8cc97fa700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8cb40a1120 0x7f8cb40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:02.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.867+0000 7f8ccb7fe700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8cb40a1120 0x7f8cb40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:02.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.867+0000 7f8cc97fa700 1 -- 172.21.15.150:0/939899549 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f8cc00ccc50 con 0x7f8ccc0ffc20 2024-05-10T04:43:02.869 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.867+0000 7f8ccb7fe700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8cb40a1120 0x7f8cb40a3630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00b5c0 tx=0x7f8cbc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:02.872 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:02.872+0000 7f8cc97fa700 1 -- 172.21.15.150:0/939899549 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8cc0095fe0 con 0x7f8ccc0ffc20 2024-05-10T04:43:03.116 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.115+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ccc0678b0 con 0x7f8ccc0ffc20 2024-05-10T04:43:03.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.116+0000 7f8cc97fa700 1 -- 172.21.15.150:0/939899549 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f8cc0099600 con 0x7f8ccc0ffc20 2024-05-10T04:43:03.117 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:43:03.118 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8cb40a1120 msgr2=0x7f8cb40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8cb40a1120 0x7f8cb40a3630 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00b5c0 tx=0x7f8cbc01a040 comp rx=0 tx=0).stop 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 msgr2=0x7f8ccc197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc197fd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c9a0 tx=0x7f8cc000cd60 comp rx=0 tx=0).stop 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 shutdown_connections 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ccc1031e0 0x7f8ccc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ccc0ffc20 0x7f8ccc197fd0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f8cb40a1120 0x7f8cb40a3630 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 --2- 172.21.15.150:0/939899549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc100570 0x7f8ccc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 >> 172.21.15.150:0/939899549 conn(0x7f8ccc0fb8f0 msgr2=0x7f8ccc106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.120 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 shutdown_connections 2024-05-10T04:43:03.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.118+0000 7f8cd29f1700 1 -- 172.21.15.150:0/939899549 wait complete. 2024-05-10T04:43:03.121 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:43:03.133 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 -- 172.21.15.150:0/4169011177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90103690 msgr2=0x7efc90109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 --2- 172.21.15.150:0/4169011177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90103690 0x7efc90109f40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efc8800b0a0 tx=0x7efc8800b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 -- 172.21.15.150:0/4169011177 shutdown_connections 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 --2- 172.21.15.150:0/4169011177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90103690 0x7efc90109f40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 --2- 172.21.15.150:0/4169011177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc90101bd0 0x7efc90101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 --2- 172.21.15.150:0/4169011177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc90102cf0 0x7efc90103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 -- 172.21.15.150:0/4169011177 >> 172.21.15.150:0/4169011177 conn(0x7efc900fd1e0 msgr2=0x7efc900ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 -- 172.21.15.150:0/4169011177 shutdown_connections 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.267+0000 7efc97161700 1 -- 172.21.15.150:0/4169011177 wait complete. 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 Processor -- start 2024-05-10T04:43:03.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 -- start start 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90101bd0 0x7efc90074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc90102cf0 0x7efc90073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc90103690 0x7efc90073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc90074180 con 0x7efc90101bd0 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900742f0 con 0x7efc90102cf0 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc97161700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc901a2080 con 0x7efc90103690 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90101bd0 0x7efc90074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90101bd0 0x7efc90074ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45366/0 (socket says 172.21.15.150:45366) 2024-05-10T04:43:03.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 -- 172.21.15.150:0/1347953204 learned_addr learned my addr 172.21.15.150:0/1347953204 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc956fe700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc90103690 0x7efc90073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc8ffff700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc90102cf0 0x7efc90073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 -- 172.21.15.150:0/1347953204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc90103690 msgr2=0x7efc90073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc90103690 0x7efc90073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 -- 172.21.15.150:0/1347953204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc90102cf0 msgr2=0x7efc90073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc90102cf0 0x7efc90073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.268+0000 7efc94efd700 1 -- 172.21.15.150:0/1347953204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88009d00 con 0x7efc90101bd0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.269+0000 7efc94efd700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90101bd0 0x7efc90074ec0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efc8400d950 tx=0x7efc8400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.269+0000 7efc8dffb700 1 -- 172.21.15.150:0/1347953204 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84004220 con 0x7efc90101bd0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.269+0000 7efc8dffb700 1 -- 172.21.15.150:0/1347953204 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc84004380 con 0x7efc90101bd0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.269+0000 7efc8dffb700 1 -- 172.21.15.150:0/1347953204 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84003be0 con 0x7efc90101bd0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.269+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc901a22e0 con 0x7efc90101bd0 2024-05-10T04:43:03.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.269+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc901a27c0 con 0x7efc90101bd0 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.271+0000 7efc8dffb700 1 -- 172.21.15.150:0/1347953204 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7efc84003d40 con 0x7efc90101bd0 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.272+0000 7efc8dffb700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc800a1240 0x7efc800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.272+0000 7efc8ffff700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc800a1240 0x7efc800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.272+0000 7efc8dffb700 1 -- 172.21.15.150:0/1347953204 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7efc840cc6a0 con 0x7efc90101bd0 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efc8ffff700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc800a1240 0x7efc800a3750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc90074860 tx=0x7efc7c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 Processor -- start 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 -- start start 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.277 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561839011150 con 0x5618390ebec0 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56183900eb10 con 0x5618390ea6a0 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efca74a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56183900a400 con 0x5618390eb570 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efc956fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efc956fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45380/0 (socket says 172.21.15.150:45380) 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.273+0000 7efc956fe700 1 -- 172.21.15.150:0/693511768 learned_addr learned my addr 172.21.15.150:0/693511768 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc94efd700 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc8ffff700 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc956fe700 1 -- 172.21.15.150:0/693511768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 msgr2=0x5618390eb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc956fe700 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390eb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc956fe700 1 -- 172.21.15.150:0/693511768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 msgr2=0x5618390eaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc956fe700 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390eaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc956fe700 1 -- 172.21.15.150:0/693511768 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561838e8b3b0 con 0x5618390ebec0 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc8ffff700 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390eb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc94efd700 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc956fe700 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390f27c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7efc8800b070 tx=0x7efc880098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b1b1985f412518 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc7affd700 1 -- 172.21.15.150:0/693511768 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800e040 con 0x5618390ebec0 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc7affd700 1 -- 172.21.15.150:0/693511768 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc88009b80 con 0x5618390ebec0 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.274+0000 7efc7affd700 1 -- 172.21.15.150:0/693511768 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc880054e0 con 0x5618390ebec0 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 -- 172.21.15.150:0/693511768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 msgr2=0x5618390f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390f27c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7efc8800b070 tx=0x7efc880098b0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 -- 172.21.15.150:0/693511768 shutdown_connections 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390f27c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 --2- 172.21.15.150:0/693511768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390eb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 -- 172.21.15.150:0/693511768 >> 172.21.15.150:0/693511768 conn(0x56183904ca60 msgr2=0x5618390e4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 -- 172.21.15.150:0/693511768 shutdown_connections 2024-05-10T04:43:03.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 -- 172.21.15.150:0/693511768 wait complete. 2024-05-10T04:43:03.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.275+0000 7efca74a5b80 1 -- start start 2024-05-10T04:43:03.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 1 Processor -- start 2024-05-10T04:43:03.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ea6a0 0x5618390f4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ebec0 0x5618390fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x561838fa6560 con 0x5618390eb570 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561838fa6200 con 0x5618390ebec0 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x561838e8d6a0 con 0x5618390ea6a0 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efca74a5b80 10 client.0 _ll_register_callbacks cb 0x561838fafb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45384/0 (socket says 172.21.15.150:45384) 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 -- 172.21.15.150:0/702683764 learned_addr learned my addr 172.21.15.150:0/702683764 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc94efd700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ea6a0 0x5618390f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 -- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ea6a0 msgr2=0x5618390f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ea6a0 0x5618390f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc956fe700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ebec0 0x5618390fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 -- 172.21.15.150:0/702683764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ebec0 msgr2=0x5618390fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ebec0 0x5618390fbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc8ffff700 1 -- 172.21.15.150:0/702683764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9010a9d0 con 0x5618390eb570 2024-05-10T04:43:03.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.276+0000 7efc94efd700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ea6a0 0x5618390f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efc8ffff700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f4b50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0116f0 tx=0x7efc7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d677a8513b3c30 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efc6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efc6ffff700 1 -- 172.21.15.150:0/702683764 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c01e830 con 0x5618390eb570 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efc6ffff700 1 -- 172.21.15.150:0/702683764 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7efc7c017070 con 0x5618390eb570 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618390b9780 con 0x5618390eb570 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efc6ffff700 1 -- 172.21.15.150:0/702683764 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc7c01e990 con 0x5618390eb570 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 10 client.16704 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561838e37b60 con 0x5618390eb570 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efc6ffff700 1 -- 172.21.15.150:0/702683764 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618390b9780 con 0x5618390eb570 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efc6ffff700 1 -- 172.21.15.150:0/702683764 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc7c027cc0 con 0x5618390eb570 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 10 client.16704 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 10 client.16704 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:43:03.286 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 20 client.16704 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839100170 0x5618391025c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 4 client.16704 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.277+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561839002ec0 con 0x561839100170 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.278+0000 7efc94efd700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839100170 0x5618391025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.278+0000 7efc6ffff700 10 client.16704 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.278+0000 7efc94efd700 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839100170 0x5618391025c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c31e495e664fa7cd server_cookie=77de58696d854e32 in_seq=0 out_seq=0 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efc6ffff700 1 -- 172.21.15.150:0/702683764 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561839002ec0 con 0x561839100170 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efc6ffff700 10 client.16704 handle_command_reply: tid=0 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 client.16704 shutdown 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 msgr2=0x5618390f4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f4b50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0116f0 tx=0x7efc7c0047c0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 shutdown_connections 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839100170 0x5618391025c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.287 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ebec0 0x5618390fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ea6a0 0x5618390f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efc6ffff700 0 client.16704 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:43:03.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 --2- 172.21.15.150:0/702683764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f4b50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 >> 172.21.15.150:0/702683764 conn(0x56183904ca60 msgr2=0x5618390e4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 shutdown_connections 2024-05-10T04:43:03.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 1 -- 172.21.15.150:0/702683764 wait complete. 2024-05-10T04:43:03.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.279+0000 7efca74a5b80 20 client.16704 trim_cache size 0 max 16384 2024-05-10T04:43:03.295 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 Processor -- start 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 -- start start 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618390e5ad0 con 0x5618390ebec0 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618390e16c0 con 0x5618390ea6a0 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efca74a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618390f56b0 con 0x5618390eb570 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efc956fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efc956fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45388/0 (socket says 172.21.15.150:45388) 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.292+0000 7efc956fe700 1 -- 172.21.15.150:0/2930900199 learned_addr learned my addr 172.21.15.150:0/2930900199 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc8ffff700 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc94efd700 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc956fe700 1 -- 172.21.15.150:0/2930900199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 msgr2=0x5618390fba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.296 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc956fe700 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390fba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc956fe700 1 -- 172.21.15.150:0/2930900199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 msgr2=0x5618390fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc956fe700 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc956fe700 1 -- 172.21.15.150:0/2930900199 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561838fd0560 con 0x5618390ebec0 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc94efd700 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc956fe700 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390e1180 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7efc88000f80 tx=0x7efc88007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7882706395664d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc7b7fe700 1 -- 172.21.15.150:0/2930900199 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800e040 con 0x5618390ebec0 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc7b7fe700 1 -- 172.21.15.150:0/2930900199 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc88003d40 con 0x5618390ebec0 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.293+0000 7efc7b7fe700 1 -- 172.21.15.150:0/2930900199 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88024e60 con 0x5618390ebec0 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 -- 172.21.15.150:0/2930900199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 msgr2=0x5618390e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390e1180 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7efc88000f80 tx=0x7efc88007830 comp rx=0 tx=0).stop 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 -- 172.21.15.150:0/2930900199 shutdown_connections 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390ebec0 0x5618390e1180 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.297 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 --2- 172.21.15.150:0/2930900199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390eb570 0x5618390fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 -- 172.21.15.150:0/2930900199 >> 172.21.15.150:0/2930900199 conn(0x56183904ca60 msgr2=0x5618390e4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 -- 172.21.15.150:0/2930900199 shutdown_connections 2024-05-10T04:43:03.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 -- 172.21.15.150:0/2930900199 wait complete. 2024-05-10T04:43:03.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 -- start start 2024-05-10T04:43:03.298 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.294+0000 7efca74a5b80 1 Processor -- start 2024-05-10T04:43:03.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efca74a5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ebec0 0x5618390fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.301 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efca74a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618390fa7d0 con 0x5618390eb570 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efca74a5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618390fa940 con 0x5618390ea6a0 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efca74a5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618390fab00 con 0x5618390ebec0 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efca74a5b80 10 client.0 _ll_register_callbacks cb 0x561838fafb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45400/0 (socket says 172.21.15.150:45400) 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 -- 172.21.15.150:0/4075161069 learned_addr learned my addr 172.21.15.150:0/4075161069 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc94efd700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc956fe700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ebec0 0x5618390fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 -- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ebec0 msgr2=0x5618390fa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ebec0 0x5618390fa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 -- 172.21.15.150:0/4075161069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 msgr2=0x5618390f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.302 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 -- 172.21.15.150:0/4075161069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561838e8b3b0 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc94efd700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc8ffff700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f9360 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0116c0 tx=0x7efc7c004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e317e357de4138de server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.295+0000 7efc6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efc6ffff700 1 -- 172.21.15.150:0/4075161069 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c01ea40 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efc6ffff700 1 -- 172.21.15.150:0/4075161069 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7efc7c017070 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561838f63270 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efc6ffff700 1 -- 172.21.15.150:0/4075161069 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc7c00be80 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 10 client.16710 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561838e8fcd0 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efc6ffff700 1 -- 172.21.15.150:0/4075161069 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561838f63270 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efc6ffff700 1 -- 172.21.15.150:0/4075161069 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efc7c027b70 con 0x5618390eb570 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 10 client.16710 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-10T04:43:03.303 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 20 client.16710 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839156af0 0x561839158f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 4 client.16710 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 --> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561838e58cf0 con 0x561839156af0 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.296+0000 7efc94efd700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839156af0 0x561839158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efc6ffff700 10 client.16710 ms_handle_connect on v2:172.21.15.181:6832/1504167586 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efc94efd700 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839156af0 0x561839158f10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e72e6cb7951112d0 server_cookie=cea84797594a9cc3 in_seq=0 out_seq=0 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efc6ffff700 1 -- 172.21.15.150:0/4075161069 <== mds.4 v2:172.21.15.181:6832/1504167586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561838e58cf0 con 0x561839156af0 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efc6ffff700 10 client.16710 handle_command_reply: tid=0 2024-05-10T04:43:03.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 client.16710 shutdown 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 msgr2=0x5618390f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f9360 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0116c0 tx=0x7efc7c004690 comp rx=0 tx=0).stop 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 shutdown_connections 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:6832/1504167586,v1:172.21.15.181:6833/1504167586] conn(0x561839156af0 0x561839158f10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618390ebec0 0x5618390fa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5618390ea6a0 0x5618390f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 --2- 172.21.15.150:0/4075161069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618390eb570 0x5618390f9360 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efc6ffff700 0 client.16710 ms_handle_reset on v2:172.21.15.181:6832/1504167586 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 >> 172.21.15.150:0/4075161069 conn(0x56183904ca60 msgr2=0x5618390ee7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 shutdown_connections 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 1 -- 172.21.15.150:0/4075161069 wait complete. 2024-05-10T04:43:03.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.297+0000 7efca74a5b80 20 client.16710 trim_cache size 0 max 16384 2024-05-10T04:43:03.306 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:43:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc800a1240 msgr2=0x7efc800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc800a1240 0x7efc800a3750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc90074860 tx=0x7efc7c00b410 comp rx=0 tx=0).stop 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90101bd0 msgr2=0x7efc90074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90101bd0 0x7efc90074ec0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efc8400d950 tx=0x7efc8400dc60 comp rx=0 tx=0).stop 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 shutdown_connections 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7efc800a1240 0x7efc800a3750 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc90103690 0x7efc90073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc90101bd0 0x7efc90074ec0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 --2- 172.21.15.150:0/1347953204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc90102cf0 0x7efc90073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 >> 172.21.15.150:0/1347953204 conn(0x7efc900fd1e0 msgr2=0x7efc90105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 shutdown_connections 2024-05-10T04:43:03.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.298+0000 7efc97161700 1 -- 172.21.15.150:0/1347953204 wait complete. 2024-05-10T04:43:03.312 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-10T04:43:03.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:03 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/939899549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:03.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 -- 172.21.15.150:0/3710646259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464101bf0 msgr2=0x7f1464102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 --2- 172.21.15.150:0/3710646259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464101bf0 0x7f1464102000 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1458009990 tx=0x7f1458009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 -- 172.21.15.150:0/3710646259 shutdown_connections 2024-05-10T04:43:03.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 --2- 172.21.15.150:0/3710646259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464102590 0x7f1464108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 --2- 172.21.15.150:0/3710646259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1464100ad0 0x7f1464100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 --2- 172.21.15.150:0/3710646259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464101bf0 0x7f1464102000 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 -- 172.21.15.150:0/3710646259 >> 172.21.15.150:0/3710646259 conn(0x7f14640fc0e0 msgr2=0x7f14640fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 -- 172.21.15.150:0/3710646259 shutdown_connections 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.447+0000 7f14694c9700 1 -- 172.21.15.150:0/3710646259 wait complete. 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.448+0000 7f14694c9700 1 Processor -- start 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.448+0000 7f14694c9700 1 -- start start 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.448+0000 7f14694c9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464100ad0 0x7f1464197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.448+0000 7f14694c9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1464101bf0 0x7f14641984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.448+0000 7f14694c9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464102590 0x7f146419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.448+0000 7f14694c9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146419fa60 con 0x7f1464100ad0 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f14694c9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146419fbd0 con 0x7f1464101bf0 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f14694c9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146419fd90 con 0x7f1464102590 2024-05-10T04:43:03.453 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464100ad0 0x7f1464197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464100ad0 0x7f1464197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45412/0 (socket says 172.21.15.150:45412) 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 -- 172.21.15.150:0/1609345465 learned_addr learned my addr 172.21.15.150:0/1609345465 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f14627fc700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1464101bf0 0x7f14641984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f14637fe700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464102590 0x7f146419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 -- 172.21.15.150:0/1609345465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464102590 msgr2=0x7f146419f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464102590 0x7f146419f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 -- 172.21.15.150:0/1609345465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1464101bf0 msgr2=0x7f14641984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1464101bf0 0x7f14641984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 -- 172.21.15.150:0/1609345465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14580095f0 con 0x7f1464100ad0 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f1462ffd700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464100ad0 0x7f1464197f80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f145400c970 tx=0x7f145400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f144bfff700 1 -- 172.21.15.150:0/1609345465 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454004220 con 0x7f1464100ad0 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14641a0070 con 0x7f1464100ad0 2024-05-10T04:43:03.454 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f144bfff700 1 -- 172.21.15.150:0/1609345465 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1454004380 con 0x7f1464100ad0 2024-05-10T04:43:03.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f144bfff700 1 -- 172.21.15.150:0/1609345465 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454003b20 con 0x7f1464100ad0 2024-05-10T04:43:03.455 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.449+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14641a05d0 con 0x7f1464100ad0 2024-05-10T04:43:03.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.452+0000 7f144bfff700 1 -- 172.21.15.150:0/1609345465 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1454003cc0 con 0x7f1464100ad0 2024-05-10T04:43:03.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.452+0000 7f144bfff700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f144c0a11c0 0x7f144c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f14627fc700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f144c0a11c0 0x7f144c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f144bfff700 1 -- 172.21.15.150:0/1609345465 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f14540cc7d0 con 0x7f1464100ad0 2024-05-10T04:43:03.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 Processor -- start 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 -- start start 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f14627fc700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f144c0a11c0 0x7f144c0a36d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f145800b5c0 tx=0x7f145801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c0969ab0 0x55c8c0969f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c096ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096b2d0 0x55c8c0971bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c088c150 con 0x55c8c0969ab0 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c0889b10 con 0x55c8c096b2d0 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f147980db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c0885400 con 0x55c8c096a980 2024-05-10T04:43:03.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f1462ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c0969ab0 0x55c8c0969f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f1462ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c0969ab0 0x55c8c0969f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45414/0 (socket says 172.21.15.150:45414) 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f1462ffd700 1 -- 172.21.15.150:0/3555549810 learned_addr learned my addr 172.21.15.150:0/3555549810 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.453+0000 7f14627fc700 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c096ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f14637fe700 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096b2d0 0x55c8c0971bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f1462ffd700 1 -- 172.21.15.150:0/3555549810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 msgr2=0x55c8c096ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f1462ffd700 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c096ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f1462ffd700 1 -- 172.21.15.150:0/3555549810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096b2d0 msgr2=0x55c8c0971bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f1462ffd700 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096b2d0 0x55c8c0971bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f1462ffd700 1 -- 172.21.15.150:0/3555549810 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c07063b0 con 0x55c8c0969ab0 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f14637fe700 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096b2d0 0x55c8c0971bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f1462ffd700 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c0969ab0 0x55c8c0969f00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1454003ff0 tx=0x7f1454099460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e719b8def7101ed server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f14497fa700 1 -- 172.21.15.150:0/3555549810 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145409d650 con 0x55c8c0969ab0 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f14497fa700 1 -- 172.21.15.150:0/3555549810 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14540a1ca0 con 0x55c8c0969ab0 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.454+0000 7f14497fa700 1 -- 172.21.15.150:0/3555549810 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454099a80 con 0x55c8c0969ab0 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 -- 172.21.15.150:0/3555549810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c0969ab0 msgr2=0x55c8c0969f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c0969ab0 0x55c8c0969f00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1454003ff0 tx=0x7f1454099460 comp rx=0 tx=0).stop 2024-05-10T04:43:03.469 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 -- 172.21.15.150:0/3555549810 shutdown_connections 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096b2d0 0x55c8c0971bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c0969ab0 0x55c8c0969f00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 --2- 172.21.15.150:0/3555549810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c096ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 -- 172.21.15.150:0/3555549810 >> 172.21.15.150:0/3555549810 conn(0x55c8c08c7a60 msgr2=0x55c8c095ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 -- 172.21.15.150:0/3555549810 shutdown_connections 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 -- 172.21.15.150:0/3555549810 wait complete. 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 -- start start 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.455+0000 7f147980db80 1 Processor -- start 2024-05-10T04:43:03.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0973af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c0974030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c097af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c0821560 con 0x55c8c096b2d0 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c0821200 con 0x55c8c0969ab0 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c07086a0 con 0x55c8c096a980 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 10 client.0 _ll_register_callbacks cb 0x55c8c082ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c097af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c097af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45416/0 (socket says 172.21.15.150:45416) 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 -- 172.21.15.150:0/3122184884 learned_addr learned my addr 172.21.15.150:0/3122184884 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f1462ffd700 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0973af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14627fc700 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c0974030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 -- 172.21.15.150:0/3122184884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 msgr2=0x55c8c0974030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.471 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c0974030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 -- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 msgr2=0x55c8c0973af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0973af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 -- 172.21.15.150:0/3122184884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14641a05d0 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f14637fe700 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c097af90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f145c008e90 tx=0x7f145c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1178730deac23 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f1446ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f1446ff5700 1 -- 172.21.15.150:0/3122184884 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c010d50 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f1446ff5700 1 -- 172.21.15.150:0/3122184884 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f145c007e10 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.456+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8c07de270 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f1446ff5700 1 -- 172.21.15.150:0/3122184884 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145c010eb0 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 10 client.16722 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c8c06b2b60 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f1446ff5700 1 -- 172.21.15.150:0/3122184884 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8c07de270 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f1446ff5700 1 -- 172.21.15.150:0/3122184884 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f145c014570 con 0x55c8c096b2d0 2024-05-10T04:43:03.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 10 client.16722 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 10 client.16722 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 20 client.16722 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c097f750 0x55c8c0981ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 4 client.16722 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8c0934780 con 0x55c8c097f750 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.457+0000 7f1462ffd700 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c097f750 0x55c8c0981ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.458+0000 7f1446ff5700 10 client.16722 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.458+0000 7f1462ffd700 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c097f750 0x55c8c0981ba0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=893a5bb307cd730c server_cookie=c601b2319ab03267 in_seq=0 out_seq=0 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.458+0000 7f1446ff5700 1 -- 172.21.15.150:0/3122184884 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8c0934780 con 0x55c8c097f750 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.458+0000 7f1446ff5700 10 client.16722 handle_command_reply: tid=0 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.458+0000 7f147980db80 1 client.16722 shutdown 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 msgr2=0x55c8c097af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c097af90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f145c008e90 tx=0x7f145c010430 comp rx=0 tx=0).stop 2024-05-10T04:43:03.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 shutdown_connections 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c097f750 0x55c8c0981ba0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c097af90 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0973af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 --2- 172.21.15.150:0/3122184884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096a980 0x55c8c0974030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f1446ff5700 0 client.16722 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 >> 172.21.15.150:0/3122184884 conn(0x55c8c08c7a60 msgr2=0x55c8c095ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 shutdown_connections 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 1 -- 172.21.15.150:0/3122184884 wait complete. 2024-05-10T04:43:03.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.459+0000 7f147980db80 20 client.16722 trim_cache size 0 max 16384 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 Processor -- start 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 -- start start 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0986450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096a980 0x55c8c097ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096b2d0 0x55c8c096e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c097b3e0 con 0x55c8c096a980 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c097f2c0 con 0x55c8c0969ab0 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f147980db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c097f480 con 0x55c8c096b2d0 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096a980 0x55c8c097ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096a980 0x55c8c097ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45426/0 (socket says 172.21.15.150:45426) 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 -- 172.21.15.150:0/4161920360 learned_addr learned my addr 172.21.15.150:0/4161920360 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14637fe700 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096b2d0 0x55c8c096e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.481 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 -- 172.21.15.150:0/4161920360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096b2d0 msgr2=0x55c8c096e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096b2d0 0x55c8c096e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f1462ffd700 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0986450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 -- 172.21.15.150:0/4161920360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 msgr2=0x55c8c0986450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0986450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 -- 172.21.15.150:0/4161920360 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c095c450 con 0x55c8c096a980 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.472+0000 7f14627fc700 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096a980 0x55c8c097ade0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1458015000 tx=0x7f1458022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8040a63c27ef6da server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f1449ffb700 1 -- 172.21.15.150:0/4161920360 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458024ad0 con 0x55c8c096a980 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f1449ffb700 1 -- 172.21.15.150:0/4161920360 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145802c070 con 0x55c8c096a980 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f1449ffb700 1 -- 172.21.15.150:0/4161920360 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580173f0 con 0x55c8c096a980 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 -- 172.21.15.150:0/4161920360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096a980 msgr2=0x55c8c097ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096a980 0x55c8c097ade0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1458015000 tx=0x7f1458022000 comp rx=0 tx=0).stop 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 -- 172.21.15.150:0/4161920360 shutdown_connections 2024-05-10T04:43:03.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c096b2d0 0x55c8c096e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c0969ab0 0x55c8c0986450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 --2- 172.21.15.150:0/4161920360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096a980 0x55c8c097ade0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 -- 172.21.15.150:0/4161920360 >> 172.21.15.150:0/4161920360 conn(0x55c8c08c7a60 msgr2=0x55c8c096a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 -- 172.21.15.150:0/4161920360 shutdown_connections 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.473+0000 7f147980db80 1 -- 172.21.15.150:0/4161920360 wait complete. 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 -- start start 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 Processor -- start 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c0969ab0 0x55c8c0978000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096a980 0x55c8c0978540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c09d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c09d40f0 con 0x55c8c096b2d0 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c09d4230 con 0x55c8c096a980 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c09d4370 con 0x55c8c0969ab0 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f147980db80 10 client.0 _ll_register_callbacks cb 0x55c8c082ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f14637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c09d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f1462ffd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c0969ab0 0x55c8c0978000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f14637fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c09d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45432/0 (socket says 172.21.15.150:45432) 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f14637fe700 1 -- 172.21.15.150:0/502057734 learned_addr learned my addr 172.21.15.150:0/502057734 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f14627fc700 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096a980 0x55c8c0978540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f14637fe700 1 -- 172.21.15.150:0/502057734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c0969ab0 msgr2=0x55c8c0978000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.474+0000 7f14637fe700 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c0969ab0 0x55c8c0978000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f14637fe700 1 -- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096a980 msgr2=0x55c8c0978540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f14637fe700 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096a980 0x55c8c0978540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f14637fe700 1 -- 172.21.15.150:0/502057734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14641098d0 con 0x55c8c096b2d0 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f1462ffd700 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c0969ab0 0x55c8c0978000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f14637fe700 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c09d3bb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f145c008e90 tx=0x7f145c006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad223a298faacb47 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f1446ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f1446ff5700 1 -- 172.21.15.150:0/502057734 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c010b50 con 0x55c8c096b2d0 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f1446ff5700 1 -- 172.21.15.150:0/502057734 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f145c010cb0 con 0x55c8c096b2d0 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14540a1ca0 con 0x55c8c096b2d0 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f1446ff5700 1 -- 172.21.15.150:0/502057734 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145c014ca0 con 0x55c8c096b2d0 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f147980db80 10 client.16728 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c8c084b560 con 0x55c8c096b2d0 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f1446ff5700 1 -- 172.21.15.150:0/502057734 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14540a1ca0 con 0x55c8c096b2d0 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f1446ff5700 1 -- 172.21.15.150:0/502057734 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f145c02e060 con 0x55c8c096b2d0 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f147980db80 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f147980db80 10 client.16728 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f147980db80 20 client.16728 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.475+0000 7f147980db80 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c09d5d30 0x55c8c09d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 4 client.16728 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:43:03.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 --> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8c087dec0 con 0x55c8c09d5d30 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f1462ffd700 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c09d5d30 0x55c8c09d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f1446ff5700 10 client.16728 ms_handle_connect on v2:172.21.15.161:6832/825127258 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f1462ffd700 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c09d5d30 0x55c8c09d8150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b89ac4f88f92d70 server_cookie=25bf95a9f2f7eb29 in_seq=0 out_seq=0 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f1446ff5700 1 -- 172.21.15.150:0/502057734 <== mds.3 v2:172.21.15.161:6832/825127258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8c087dec0 con 0x55c8c09d5d30 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f1446ff5700 10 client.16728 handle_command_reply: tid=0 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 client.16728 shutdown 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 msgr2=0x55c8c09d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c09d3bb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f145c008e90 tx=0x7f145c006070 comp rx=0 tx=0).stop 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 shutdown_connections 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:6832/825127258,v1:172.21.15.161:6833/825127258] conn(0x55c8c09d5d30 0x55c8c09d8150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c8c096b2d0 0x55c8c09d3bb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f1446ff5700 0 client.16728 ms_handle_reset on v2:172.21.15.161:6832/825127258 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c8c0969ab0 0x55c8c0978000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 --2- 172.21.15.150:0/502057734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c096a980 0x55c8c0978540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.476+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 >> 172.21.15.150:0/502057734 conn(0x55c8c08c7a60 msgr2=0x55c8c0981760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 shutdown_connections 2024-05-10T04:43:03.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f147980db80 1 -- 172.21.15.150:0/502057734 wait complete. 2024-05-10T04:43:03.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f147980db80 20 client.16728 trim_cache size 0 max 16384 2024-05-10T04:43:03.487 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f144c0a11c0 msgr2=0x7f144c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f144c0a11c0 0x7f144c0a36d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f145800b5c0 tx=0x7f145801a040 comp rx=0 tx=0).stop 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464100ad0 msgr2=0x7f1464197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464100ad0 0x7f1464197f80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f145400c970 tx=0x7f145400cc80 comp rx=0 tx=0).stop 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 shutdown_connections 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f144c0a11c0 0x7f144c0a36d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1464102590 0x7f146419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1464100ad0 0x7f1464197f80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 --2- 172.21.15.150:0/1609345465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1464101bf0 0x7f14641984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 >> 172.21.15.150:0/1609345465 conn(0x7f14640fc0e0 msgr2=0x7f14640fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 shutdown_connections 2024-05-10T04:43:03.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.477+0000 7f14694c9700 1 -- 172.21.15.150:0/1609345465 wait complete. 2024-05-10T04:43:03.491 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-10T04:43:03.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:03 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/939899549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:03.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 -- 172.21.15.150:0/311125502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54102590 msgr2=0x7f1c54108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 --2- 172.21.15.150:0/311125502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54102590 0x7f1c54108e40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00b0a0 tx=0x7f1c4c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 -- 172.21.15.150:0/311125502 shutdown_connections 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 --2- 172.21.15.150:0/311125502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54102590 0x7f1c54108e40 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 --2- 172.21.15.150:0/311125502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c54100ad0 0x7f1c54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 --2- 172.21.15.150:0/311125502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c54101bf0 0x7f1c54102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 -- 172.21.15.150:0/311125502 >> 172.21.15.150:0/311125502 conn(0x7f1c540fc0e0 msgr2=0x7f1c540fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 -- 172.21.15.150:0/311125502 shutdown_connections 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 -- 172.21.15.150:0/311125502 wait complete. 2024-05-10T04:43:03.629 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.625+0000 7f1c5991b700 1 Processor -- start 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c5991b700 1 -- start start 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c5991b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54100ad0 0x7f1c54196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c5991b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c54101bf0 0x7f1c541974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c5991b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c54102590 0x7f1c5419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c5991b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5419ea70 con 0x7f1c54100ad0 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c5991b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5419ebe0 con 0x7f1c54101bf0 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c5991b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5419eda0 con 0x7f1c54102590 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54100ad0 0x7f1c54196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54100ad0 0x7f1c54196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45446/0 (socket says 172.21.15.150:45446) 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 -- 172.21.15.150:0/3556763561 learned_addr learned my addr 172.21.15.150:0/3556763561 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.630 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c537fe700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c54102590 0x7f1c5419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c527fc700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c54101bf0 0x7f1c541974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 -- 172.21.15.150:0/3556763561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c54102590 msgr2=0x7f1c5419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c54102590 0x7f1c5419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 -- 172.21.15.150:0/3556763561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c54101bf0 msgr2=0x7f1c541974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c54101bf0 0x7f1c541974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 -- 172.21.15.150:0/3556763561 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c009d00 con 0x7f1c54100ad0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c527fc700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c54101bf0 0x7f1c541974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.626+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54100ad0 0x7f1c54196f90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400d950 tx=0x7f1c4400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.627+0000 7f1c58919700 1 -- 172.21.15.150:0/3556763561 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44004220 con 0x7f1c54100ad0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.627+0000 7f1c58919700 1 -- 172.21.15.150:0/3556763561 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4400de30 con 0x7f1c54100ad0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.627+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5419f080 con 0x7f1c54100ad0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.627+0000 7f1c58919700 1 -- 172.21.15.150:0/3556763561 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44003cb0 con 0x7f1c54100ad0 2024-05-10T04:43:03.631 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.627+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5419f590 con 0x7f1c54100ad0 2024-05-10T04:43:03.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.630+0000 7f1c69c5fb80 1 Processor -- start 2024-05-10T04:43:03.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.630+0000 7f1c69c5fb80 1 -- start start 2024-05-10T04:43:03.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5623af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5624980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa562b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa554a150 con 0x564fa5624570 2024-05-10T04:43:03.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa5547b10 con 0x564fa5624ec0 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa5543400 con 0x564fa56236a0 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5623af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c527fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa562b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c527fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa562b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.150:38996/0 (socket says 172.21.15.150:38996) 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5623af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:59286/0 (socket says 172.21.15.150:59286) 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c527fc700 1 -- 172.21.15.150:0/2862026788 learned_addr learned my addr 172.21.15.150:0/2862026788 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 -- 172.21.15.150:0/2862026788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 msgr2=0x564fa562b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 --2- 172.21.15.150:0/2862026788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa562b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 -- 172.21.15.150:0/2862026788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 msgr2=0x564fa5624980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 --2- 172.21.15.150:0/2862026788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5624980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 -- 172.21.15.150:0/2862026788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa53c43b0 con 0x564fa56236a0 2024-05-10T04:43:03.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.631+0000 7f1c537fe700 1 --2- 172.21.15.150:0/2862026788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5623af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00b070 tx=0x7f1c4c0097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50688c903e27aab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c39ffb700 1 -- 172.21.15.150:0/2862026788 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00e040 con 0x564fa56236a0 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c39ffb700 1 -- 172.21.15.150:0/2862026788 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4c0075c0 con 0x564fa56236a0 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c39ffb700 1 -- 172.21.15.150:0/2862026788 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c01b440 con 0x564fa56236a0 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c58919700 1 -- 172.21.15.150:0/3556763561 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1c44004380 con 0x7f1c54100ad0 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/2862026788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 msgr2=0x564fa5623af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/2862026788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5623af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00b070 tx=0x7f1c4c0097f0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/2862026788 shutdown_connections 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/2862026788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa562b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/2862026788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5623af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/2862026788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5624980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.632+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/2862026788 >> 172.21.15.150:0/2862026788 conn(0x564fa5585a60 msgr2=0x564fa561dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/2862026788 shutdown_connections 2024-05-10T04:43:03.641 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/2862026788 wait complete. 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c58919700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1c3c0a10d0 0x7f1c3c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c527fc700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1c3c0a10d0 0x7f1c3c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c69c5fb80 1 -- start start 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c58919700 1 -- 172.21.15.150:0/3556763561 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1c440cea20 con 0x7f1c54100ad0 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c69c5fb80 1 Processor -- start 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.633+0000 7f1c527fc700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1c3c0a10d0 0x7f1c3c0a35e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009cf0 tx=0x7f1c48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa562d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa562dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa5634ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa54df560 con 0x564fa5624570 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa54df200 con 0x564fa5624ec0 2024-05-10T04:43:03.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa53c66a0 con 0x564fa56236a0 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa562dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c69c5fb80 10 client.0 _ll_register_callbacks cb 0x564fa54e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa562dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45472/0 (socket says 172.21.15.150:45472) 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 -- 172.21.15.150:0/585195849 learned_addr learned my addr 172.21.15.150:0/585195849 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa562d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c537fe700 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa5634ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 -- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 msgr2=0x564fa562d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa562d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 -- 172.21.15.150:0/585195849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 msgr2=0x564fa5634ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa5634ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 -- 172.21.15.150:0/585195849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c541098d0 con 0x564fa5624570 2024-05-10T04:43:03.643 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c527fc700 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa562dbe0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1c4801e8e0 tx=0x7f1c48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893388a087d1f82c server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c32ffd700 1 -- 172.21.15.150:0/585195849 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48011770 con 0x564fa5624570 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.634+0000 7f1c32ffd700 1 -- 172.21.15.150:0/585195849 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1c480118d0 con 0x564fa5624570 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fa549c270 con 0x564fa5624570 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c32ffd700 1 -- 172.21.15.150:0/585195849 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c48011a90 con 0x564fa5624570 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 10 client.16737 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564fa5370b60 con 0x564fa5624570 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c32ffd700 1 -- 172.21.15.150:0/585195849 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564fa549c270 con 0x564fa5624570 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c32ffd700 1 -- 172.21.15.150:0/585195849 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c48026410 con 0x564fa5624570 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 10 client.16737 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 10 client.16737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:43:03.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 20 client.16737 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5639320 0x564fa563b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 4 client.16737 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fa55f2780 con 0x564fa5639320 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.635+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5639320 0x564fa563b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.636+0000 7f1c32ffd700 10 client.16737 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.636+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5639320 0x564fa563b770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c4a4d23d2f46923 server_cookie=dcd91cb2f07557c4 in_seq=0 out_seq=0 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.636+0000 7f1c32ffd700 1 -- 172.21.15.150:0/585195849 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564fa55f2780 con 0x564fa5639320 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.636+0000 7f1c32ffd700 10 client.16737 handle_command_reply: tid=0 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.636+0000 7f1c69c5fb80 1 client.16737 shutdown 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 msgr2=0x564fa562dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa562dbe0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1c4801e8e0 tx=0x7f1c48004750 comp rx=0 tx=0).stop 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 shutdown_connections 2024-05-10T04:43:03.645 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5639320 0x564fa563b770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624ec0 0x564fa5634ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa562d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c32ffd700 0 client.16737 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/585195849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa562dbe0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 >> 172.21.15.150:0/585195849 conn(0x564fa5585a60 msgr2=0x564fa561dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 shutdown_connections 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/585195849 wait complete. 2024-05-10T04:43:03.646 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.637+0000 7f1c69c5fb80 20 client.16737 trim_cache size 0 max 16384 2024-05-10T04:43:03.659 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:03 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/939899549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:03.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 Processor -- start 2024-05-10T04:43:03.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 -- start start 2024-05-10T04:43:03.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5585a60 0x564fa562d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5634a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.660 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624570 0x564fa562dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa562e490 con 0x564fa5585a60 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa561a180 con 0x564fa5624570 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa561a2f0 con 0x564fa56236a0 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5585a60 0x564fa562d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5585a60 0x564fa562d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45488/0 (socket says 172.21.15.150:45488) 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 -- 172.21.15.150:0/1115897721 learned_addr learned my addr 172.21.15.150:0/1115897721 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c527fc700 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5634a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 -- 172.21.15.150:0/1115897721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 msgr2=0x564fa5634a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5634a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 -- 172.21.15.150:0/1115897721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624570 msgr2=0x564fa562dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624570 0x564fa562dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.661 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c52ffd700 1 -- 172.21.15.150:0/1115897721 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fa5509560 con 0x564fa5585a60 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.650+0000 7f1c527fc700 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5634a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5585a60 0x564fa562d6a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1c44097fd0 tx=0x7f1c44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e66f75d5a6e77 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c3a7fc700 1 -- 172.21.15.150:0/1115897721 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44014070 con 0x564fa5585a60 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c3a7fc700 1 -- 172.21.15.150:0/1115897721 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4409f900 con 0x564fa5585a60 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c3a7fc700 1 -- 172.21.15.150:0/1115897721 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440b44c0 con 0x564fa5585a60 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/1115897721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5585a60 msgr2=0x564fa562d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5585a60 0x564fa562d6a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1c44097fd0 tx=0x7f1c44000bc0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/1115897721 shutdown_connections 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5624570 0x564fa562dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5585a60 0x564fa562d6a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/1115897721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa5634a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.651+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/1115897721 >> 172.21.15.150:0/1115897721 conn(0x564fa561a450 msgr2=0x564fa561de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/1115897721 shutdown_connections 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/1115897721 wait complete. 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 -- start start 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 Processor -- start 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5585a60 0x564fa568b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa568dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5690410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa5690ab0 con 0x564fa5624570 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa5690c20 con 0x564fa5585a60 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fa5690de0 con 0x564fa56236a0 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c69c5fb80 10 client.0 _ll_register_callbacks cb 0x564fa54e8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c537fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5690410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.663 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c537fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5690410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45504/0 (socket says 172.21.15.150:45504) 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.652+0000 7f1c537fe700 1 -- 172.21.15.150:0/3224871156 learned_addr learned my addr 172.21.15.150:0/3224871156 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c527fc700 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa568dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5585a60 0x564fa568b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c537fe700 1 -- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 msgr2=0x564fa568dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c537fe700 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa568dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c537fe700 1 -- 172.21.15.150:0/3224871156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5585a60 msgr2=0x564fa568b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c537fe700 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5585a60 0x564fa568b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c537fe700 1 -- 172.21.15.150:0/3224871156 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5419f590 con 0x564fa5624570 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c537fe700 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5690410 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x564fa5623380 tx=0x7f1c4c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee69531ecffeaaa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c32ffd700 1 -- 172.21.15.150:0/3224871156 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00e070 con 0x564fa5624570 2024-05-10T04:43:03.664 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c32ffd700 1 -- 172.21.15.150:0/3224871156 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1c4c00bd30 con 0x564fa5624570 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c4c0075c0 con 0x564fa5624570 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c32ffd700 1 -- 172.21.15.150:0/3224871156 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c4c004a50 con 0x564fa5624570 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c69c5fb80 10 client.16743 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564fa53c8cd0 con 0x564fa5624570 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.653+0000 7f1c32ffd700 1 -- 172.21.15.150:0/3224871156 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4c0075c0 con 0x564fa5624570 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c32ffd700 1 -- 172.21.15.150:0/3224871156 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1c4c019040 con 0x564fa5624570 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c69c5fb80 10 client.16743 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c69c5fb80 10 client.16743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c69c5fb80 20 client.16743 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5624ec0 0x564fa5694230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c69c5fb80 4 client.16743 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 --> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564fa553bec0 con 0x564fa5624ec0 2024-05-10T04:43:03.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5624ec0 0x564fa5694230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c32ffd700 10 client.16743 ms_handle_connect on v2:172.21.15.181:6834/3090072990 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c52ffd700 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5624ec0 0x564fa5694230 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81d10ce8800b0569 server_cookie=2ea93c6d16d7f1ab in_seq=0 out_seq=0 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c32ffd700 1 -- 172.21.15.150:0/3224871156 <== mds.2 v2:172.21.15.181:6834/3090072990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564fa553bec0 con 0x564fa5624ec0 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c32ffd700 10 client.16743 handle_command_reply: tid=0 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.654+0000 7f1c69c5fb80 1 client.16743 shutdown 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 msgr2=0x564fa5690410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5690410 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x564fa5623380 tx=0x7f1c4c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 shutdown_connections 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:6834/3090072990,v1:172.21.15.181:6835/3090072990] conn(0x564fa5624ec0 0x564fa5694230 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564fa5624570 0x564fa5690410 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c32ffd700 0 client.16743 ms_handle_reset on v2:172.21.15.181:6834/3090072990 2024-05-10T04:43:03.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fa5585a60 0x564fa568b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 --2- 172.21.15.150:0/3224871156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564fa56236a0 0x564fa568dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 >> 172.21.15.150:0/3224871156 conn(0x564fa561a450 msgr2=0x564fa562b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 shutdown_connections 2024-05-10T04:43:03.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 1 -- 172.21.15.150:0/3224871156 wait complete. 2024-05-10T04:43:03.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c69c5fb80 20 client.16743 trim_cache size 0 max 16384 2024-05-10T04:43:03.667 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:43:03.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1c3c0a10d0 msgr2=0x7f1c3c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1c3c0a10d0 0x7f1c3c0a35e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009cf0 tx=0x7f1c48009260 comp rx=0 tx=0).stop 2024-05-10T04:43:03.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54100ad0 msgr2=0x7f1c54196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54100ad0 0x7f1c54196f90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400d950 tx=0x7f1c4400dd10 comp rx=0 tx=0).stop 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 shutdown_connections 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f1c3c0a10d0 0x7f1c3c0a35e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c54102590 0x7f1c5419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c54100ad0 0x7f1c54196f90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 --2- 172.21.15.150:0/3556763561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c54101bf0 0x7f1c541974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 >> 172.21.15.150:0/3556763561 conn(0x7f1c540fc0e0 msgr2=0x7f1c54104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 shutdown_connections 2024-05-10T04:43:03.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.655+0000 7f1c5991b700 1 -- 172.21.15.150:0/3556763561 wait complete. 2024-05-10T04:43:03.670 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 -- 172.21.15.150:0/3048813109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7094104de0 msgr2=0x7f70941072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 --2- 172.21.15.150:0/3048813109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7094104de0 0x7f70941072c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7088009990 tx=0x7f7088009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 -- 172.21.15.150:0/3048813109 shutdown_connections 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 --2- 172.21.15.150:0/3048813109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094107950 0x7f7094109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 --2- 172.21.15.150:0/3048813109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f709406a430 0x7f7094104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 --2- 172.21.15.150:0/3048813109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7094104de0 0x7f70941072c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 -- 172.21.15.150:0/3048813109 >> 172.21.15.150:0/3048813109 conn(0x7f70940fc180 msgr2=0x7f70940fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.809 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 -- 172.21.15.150:0/3048813109 shutdown_connections 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 -- 172.21.15.150:0/3048813109 wait complete. 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.805+0000 7f709a1e0700 1 Processor -- start 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f709a1e0700 1 -- start start 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f709a1e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709406a430 0x7f7094197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f709a1e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7094104de0 0x7f7094198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f709a1e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094107950 0x7f709419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f709a1e0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709419f9c0 con 0x7f709406a430 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f709a1e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709419fb30 con 0x7f7094107950 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f709a1e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709419fcf0 con 0x7f7094104de0 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709406a430 0x7f7094197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709406a430 0x7f7094197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45520/0 (socket says 172.21.15.150:45520) 2024-05-10T04:43:03.810 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 -- 172.21.15.150:0/58358551 learned_addr learned my addr 172.21.15.150:0/58358551 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f7093fff700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094107950 0x7f709419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f7092ffd700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7094104de0 0x7f7094198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 -- 172.21.15.150:0/58358551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7094104de0 msgr2=0x7f7094198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7094104de0 0x7f7094198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 -- 172.21.15.150:0/58358551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094107950 msgr2=0x7f709419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094107950 0x7f709419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 -- 172.21.15.150:0/58358551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880095f0 con 0x7f709406a430 2024-05-10T04:43:03.811 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.806+0000 7f70937fe700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709406a430 0x7f7094197ee0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f708400e970 tx=0x7f708400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.807+0000 7f7090ff9700 1 -- 172.21.15.150:0/58358551 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084009790 con 0x7f709406a430 2024-05-10T04:43:03.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.807+0000 7f7090ff9700 1 -- 172.21.15.150:0/58358551 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7084004cf0 con 0x7f709406a430 2024-05-10T04:43:03.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.807+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709419ffd0 con 0x7f709406a430 2024-05-10T04:43:03.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.807+0000 7f7090ff9700 1 -- 172.21.15.150:0/58358551 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084005680 con 0x7f709406a430 2024-05-10T04:43:03.812 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.807+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70941a03f0 con 0x7f709406a430 2024-05-10T04:43:03.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.809+0000 7f7090ff9700 1 -- 172.21.15.150:0/58358551 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f7084004750 con 0x7f709406a430 2024-05-10T04:43:03.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.810+0000 7f7090ff9700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f707c0a11c0 0x7f707c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.810+0000 7f7092ffd700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f707c0a11c0 0x7f707c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.810+0000 7f7090ff9700 1 -- 172.21.15.150:0/58358551 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7084014070 con 0x7f709406a430 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f7092ffd700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f707c0a11c0 0x7f707c0a36d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f708800b2e0 tx=0x7f708801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 Processor -- start 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 -- start start 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e494511150 con 0x55e4945eb570 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49450eb10 con 0x55e4945ea6a0 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70aa524b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49450a400 con 0x55e4945ebec0 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f7092ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.821 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f7092ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945eb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45524/0 (socket says 172.21.15.150:45524) 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f7092ffd700 1 -- 172.21.15.150:0/1996463417 learned_addr learned my addr 172.21.15.150:0/1996463417 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f70937fe700 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f7093fff700 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.811+0000 7f7092ffd700 1 -- 172.21.15.150:0/1996463417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 msgr2=0x55e4945f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7092ffd700 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7092ffd700 1 -- 172.21.15.150:0/1996463417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 msgr2=0x55e4945eaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7092ffd700 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945eaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7092ffd700 1 -- 172.21.15.150:0/1996463417 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e49438b3b0 con 0x55e4945eb570 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f70937fe700 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7092ffd700 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945eb980 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7088009990 tx=0x7f708801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c8dfa1aa81eb79 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7079ffb700 1 -- 172.21.15.150:0/1996463417 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708802a070 con 0x55e4945eb570 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7079ffb700 1 -- 172.21.15.150:0/1996463417 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708800fd00 con 0x55e4945eb570 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.812+0000 7f7079ffb700 1 -- 172.21.15.150:0/1996463417 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708801d040 con 0x55e4945eb570 2024-05-10T04:43:03.822 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 -- 172.21.15.150:0/1996463417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 msgr2=0x55e4945eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945eb980 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7088009990 tx=0x7f708801e000 comp rx=0 tx=0).stop 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 -- 172.21.15.150:0/1996463417 shutdown_connections 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1996463417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945eb980 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 -- 172.21.15.150:0/1996463417 >> 172.21.15.150:0/1996463417 conn(0x55e49454ca60 msgr2=0x55e4945e4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 -- 172.21.15.150:0/1996463417 shutdown_connections 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 -- 172.21.15.150:0/1996463417 wait complete. 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 -- start start 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.813+0000 7f70aa524b80 1 Processor -- start 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945eb570 0x55e4945f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70aa524b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4944a6560 con 0x55e4945ebec0 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70aa524b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4944a6200 con 0x55e4945ea6a0 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70aa524b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e49438d6a0 con 0x55e4945eb570 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70aa524b80 10 client.0 _ll_register_callbacks cb 0x55e4944afb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70937fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945fbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45538/0 (socket says 172.21.15.150:45538) 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 -- 172.21.15.150:0/4170149589 learned_addr learned my addr 172.21.15.150:0/4170149589 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7092ffd700 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945eb570 0x55e4945f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 -- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945eb570 msgr2=0x55e4945f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945eb570 0x55e4945f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 -- 172.21.15.150:0/4170149589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 msgr2=0x55e4945f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.824 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 -- 172.21.15.150:0/4170149589 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709410aa50 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f70937fe700 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7093fff700 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945fbab0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f708c007970 tx=0x7f708c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74616c037ea92dd server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7072ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.814+0000 7f7072ffd700 1 -- 172.21.15.150:0/4170149589 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c015610 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f7072ffd700 1 -- 172.21.15.150:0/4170149589 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f708c015770 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e494463270 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f7072ffd700 1 -- 172.21.15.150:0/4170149589 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f708c015930 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 10 client.16755 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e494337b60 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f7072ffd700 1 -- 172.21.15.150:0/4170149589 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e494463270 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f7072ffd700 1 -- 172.21.15.150:0/4170149589 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f708c01dc70 con 0x55e4945ebec0 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 10 client.16755 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 10 client.16755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:43:03.825 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 20 client.16755 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e494600310 0x55e494602760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 4 client.16755 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4945b9780 con 0x55e494600310 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.815+0000 7f70937fe700 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e494600310 0x55e494602760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.816+0000 7f7072ffd700 10 client.16755 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.816+0000 7f70937fe700 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e494600310 0x55e494602760 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7695bbb2e6569ca0 server_cookie=4e2ba4b36de6cd2b in_seq=0 out_seq=0 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.816+0000 7f7072ffd700 1 -- 172.21.15.150:0/4170149589 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4945b9780 con 0x55e494600310 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.816+0000 7f7072ffd700 10 client.16755 handle_command_reply: tid=0 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.816+0000 7f70aa524b80 1 client.16755 shutdown 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 msgr2=0x55e4945fbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945fbab0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f708c007970 tx=0x7f708c007c80 comp rx=0 tx=0).stop 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 shutdown_connections 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e494600310 0x55e494602760 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.826 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945fbab0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f7072ffd700 0 client.16755 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:43:03.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 --2- 172.21.15.150:0/4170149589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945eb570 0x55e4945f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 >> 172.21.15.150:0/4170149589 conn(0x55e49454ca60 msgr2=0x55e4945e4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 shutdown_connections 2024-05-10T04:43:03.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 1 -- 172.21.15.150:0/4170149589 wait complete. 2024-05-10T04:43:03.827 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.817+0000 7f70aa524b80 20 client.16755 trim_cache size 0 max 16384 2024-05-10T04:43:03.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 Processor -- start 2024-05-10T04:43:03.833 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 -- start start 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ea6a0 0x55e4945f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945eb570 0x55e4945fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4945e6770 con 0x55e4945ebec0 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4945e61e0 con 0x55e4945eb570 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f70aa524b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4945f2670 con 0x55e4945ea6a0 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f7093fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f7093fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45544/0 (socket says 172.21.15.150:45544) 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.830+0000 7f7093fff700 1 -- 172.21.15.150:0/1140344806 learned_addr learned my addr 172.21.15.150:0/1140344806 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f70937fe700 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ea6a0 0x55e4945f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7092ffd700 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945eb570 0x55e4945fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.834 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7093fff700 1 -- 172.21.15.150:0/1140344806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ea6a0 msgr2=0x55e4945f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7093fff700 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ea6a0 0x55e4945f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7093fff700 1 -- 172.21.15.150:0/1140344806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945eb570 msgr2=0x55e4945fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7093fff700 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945eb570 0x55e4945fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7093fff700 1 -- 172.21.15.150:0/1140344806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4944d0560 con 0x55e4945ebec0 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f70937fe700 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ea6a0 0x55e4945f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7092ffd700 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945eb570 0x55e4945fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f7093fff700 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945ef350 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f708c007940 tx=0x7f708c0093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736ab0d9f5586ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f707a7fc700 1 -- 172.21.15.150:0/1140344806 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c014520 con 0x55e4945ebec0 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f707a7fc700 1 -- 172.21.15.150:0/1140344806 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708c014830 con 0x55e4945ebec0 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.831+0000 7f707a7fc700 1 -- 172.21.15.150:0/1140344806 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c01bb50 con 0x55e4945ebec0 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 -- 172.21.15.150:0/1140344806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 msgr2=0x55e4945ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945ef350 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f708c007940 tx=0x7f708c0093b0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 -- 172.21.15.150:0/1140344806 shutdown_connections 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945ebec0 0x55e4945ef350 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ea6a0 0x55e4945f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 --2- 172.21.15.150:0/1140344806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945eb570 0x55e4945fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 -- 172.21.15.150:0/1140344806 >> 172.21.15.150:0/1140344806 conn(0x55e49454ca60 msgr2=0x55e4945e5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 -- 172.21.15.150:0/1140344806 shutdown_connections 2024-05-10T04:43:03.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 -- 172.21.15.150:0/1140344806 wait complete. 2024-05-10T04:43:03.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 -- start start 2024-05-10T04:43:03.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.832+0000 7f70aa524b80 1 Processor -- start 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945f9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70aa524b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70aa524b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4945fa880 con 0x55e4945eb570 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70aa524b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4945fa9f0 con 0x55e4945ea6a0 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70aa524b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4945fabb0 con 0x55e4945ebec0 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70aa524b80 10 client.0 _ll_register_callbacks cb 0x55e4944afb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45556/0 (socket says 172.21.15.150:45556) 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 -- 172.21.15.150:0/2613450733 learned_addr learned my addr 172.21.15.150:0/2613450733 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f70937fe700 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.840 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7093fff700 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 -- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 msgr2=0x55e4945fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 -- 172.21.15.150:0/2613450733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 msgr2=0x55e4945f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 -- 172.21.15.150:0/2613450733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e494337b60 con 0x55e4945eb570 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7092ffd700 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945f9410 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f708800c010 tx=0x7f7088004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a159f6dbad5cc9e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.833+0000 7f7072ffd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f7072ffd700 1 -- 172.21.15.150:0/2613450733 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708802a070 con 0x55e4945eb570 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f7072ffd700 1 -- 172.21.15.150:0/2613450733 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f7088004e40 con 0x55e4945eb570 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f708800fd00 con 0x55e4945eb570 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f7072ffd700 1 -- 172.21.15.150:0/2613450733 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f708801d050 con 0x55e4945eb570 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 10 client.16761 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e49438fcd0 con 0x55e4945eb570 2024-05-10T04:43:03.841 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f7072ffd700 1 -- 172.21.15.150:0/2613450733 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7088040080 con 0x55e4945eb570 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f7072ffd700 1 -- 172.21.15.150:0/2613450733 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f708802e950 con 0x55e4945eb570 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 10 client.16761 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 10 client.16761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 20 client.16761 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e4946568f0 0x55e494658d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 4 client.16761 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 --> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e494502ec0 con 0x55e4946568f0 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.834+0000 7f70937fe700 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e4946568f0 0x55e494658d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f7072ffd700 10 client.16761 ms_handle_connect on v2:172.21.15.181:6838/4287682926 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70937fe700 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e4946568f0 0x55e494658d10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=810b4f8c96fc3406 server_cookie=1e77f755b23ee1c4 in_seq=0 out_seq=0 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f7072ffd700 1 -- 172.21.15.150:0/2613450733 <== mds.1 v2:172.21.15.181:6838/4287682926 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e494502ec0 con 0x55e4946568f0 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f7072ffd700 10 client.16761 handle_command_reply: tid=0 2024-05-10T04:43:03.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 client.16761 shutdown 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 msgr2=0x55e4945f9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945f9410 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f708800c010 tx=0x7f7088004a90 comp rx=0 tx=0).stop 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 shutdown_connections 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:6838/4287682926,v1:172.21.15.181:6839/4287682926] conn(0x55e4946568f0 0x55e494658d10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4945ebec0 0x55e4945fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e4945ea6a0 0x55e4945f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f7072ffd700 0 client.16761 ms_handle_reset on v2:172.21.15.181:6838/4287682926 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 --2- 172.21.15.150:0/2613450733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e4945eb570 0x55e4945f9410 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 >> 172.21.15.150:0/2613450733 conn(0x55e49454ca60 msgr2=0x55e4945e5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 shutdown_connections 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 1 -- 172.21.15.150:0/2613450733 wait complete. 2024-05-10T04:43:03.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.835+0000 7f70aa524b80 20 client.16761 trim_cache size 0 max 16384 2024-05-10T04:43:03.844 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:43:03.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f707c0a11c0 msgr2=0x7f707c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.844 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f707c0a11c0 0x7f707c0a36d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f708800b2e0 tx=0x7f708801a040 comp rx=0 tx=0).stop 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709406a430 msgr2=0x7f7094197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709406a430 0x7f7094197ee0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f708400e970 tx=0x7f708400ed30 comp rx=0 tx=0).stop 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 shutdown_connections 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7094107950 0x7f709419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709406a430 0x7f7094197ee0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f707c0a11c0 0x7f707c0a36d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 --2- 172.21.15.150:0/58358551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7094104de0 0x7f7094198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 >> 172.21.15.150:0/58358551 conn(0x7f70940fc180 msgr2=0x7f7094102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 shutdown_connections 2024-05-10T04:43:03.845 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.836+0000 7f709a1e0700 1 -- 172.21.15.150:0/58358551 wait complete. 2024-05-10T04:43:03.850 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 -- 172.21.15.150:0/1906339401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641013e0 msgr2=0x7f12641017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 --2- 172.21.15.150:0/1906339401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641013e0 0x7f12641017f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1250009990 tx=0x7f1250009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 -- 172.21.15.150:0/1906339401 shutdown_connections 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 --2- 172.21.15.150:0/1906339401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1264101d80 0x7f1264108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 --2- 172.21.15.150:0/1906339401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12641002c0 0x7f12641006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 --2- 172.21.15.150:0/1906339401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641013e0 0x7f12641017f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 -- 172.21.15.150:0/1906339401 >> 172.21.15.150:0/1906339401 conn(0x7f12640fb8f0 msgr2=0x7f12640fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 -- 172.21.15.150:0/1906339401 shutdown_connections 2024-05-10T04:43:03.988 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.984+0000 7f126afb8700 1 -- 172.21.15.150:0/1906339401 wait complete. 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 Processor -- start 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 -- start start 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641002c0 0x7f1264197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12641013e0 0x7f12641984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1264101d80 0x7f126419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126419fa20 con 0x7f12641002c0 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126419fb60 con 0x7f1264101d80 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f126afb8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126419fcf0 con 0x7f12641013e0 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641002c0 0x7f1264197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641002c0 0x7f1264197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45574/0 (socket says 172.21.15.150:45574) 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 -- 172.21.15.150:0/83484902 learned_addr learned my addr 172.21.15.150:0/83484902 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1263fff700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12641013e0 0x7f12641984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1269555700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1264101d80 0x7f126419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 -- 172.21.15.150:0/83484902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12641013e0 msgr2=0x7f12641984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.989 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12641013e0 0x7f12641984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 -- 172.21.15.150:0/83484902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1264101d80 msgr2=0x7f126419f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1264101d80 0x7f126419f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1268d54700 1 -- 172.21.15.150:0/83484902 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500095f0 con 0x7f12641002c0 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1263fff700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12641013e0 0x7f12641984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.985+0000 7f1269555700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1264101d80 0x7f126419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.986+0000 7f1268d54700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641002c0 0x7f1264197f80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f125800eac0 tx=0x7f125800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.986+0000 7f1261ffb700 1 -- 172.21.15.150:0/83484902 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125800cb90 con 0x7f12641002c0 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.986+0000 7f1261ffb700 1 -- 172.21.15.150:0/83484902 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12580044e0 con 0x7f12641002c0 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.986+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126419ffd0 con 0x7f12641002c0 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.986+0000 7f1261ffb700 1 -- 172.21.15.150:0/83484902 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258003e30 con 0x7f12641002c0 2024-05-10T04:43:03.990 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.986+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12641a0400 con 0x7f12641002c0 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.989+0000 7f1261ffb700 1 -- 172.21.15.150:0/83484902 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1258004000 con 0x7f12641002c0 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.989+0000 7f1261ffb700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12540a11c0 0x7f12540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f1263fff700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12540a11c0 0x7f12540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f1261ffb700 1 -- 172.21.15.150:0/83484902 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f12580ccb50 con 0x7f12641002c0 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 Processor -- start 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f1263fff700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12540a11c0 0x7f12540a36d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f125000b2e0 tx=0x7f125001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 -- start start 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cd570 0x5608162cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:03.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608161f3150 con 0x5608162cdec0 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608161f0b10 con 0x5608162cc6a0 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.990+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608161ec400 con 0x5608162cd570 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162d47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45576/0 (socket says 172.21.15.150:45576) 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 -- 172.21.15.150:0/3932394018 learned_addr learned my addr 172.21.15.150:0/3932394018 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1268d54700 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1263fff700 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cd570 0x5608162cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 -- 172.21.15.150:0/3932394018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cd570 msgr2=0x5608162cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cd570 0x5608162cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 -- 172.21.15.150:0/3932394018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 msgr2=0x5608162ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 -- 172.21.15.150:0/3932394018 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56081606d3b0 con 0x5608162cdec0 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f1269555700 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162d47c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d300 tx=0x7f125c00d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f844720ccc886e server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:03.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f124effd700 1 -- 172.21.15.150:0/3932394018 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c0125f0 con 0x5608162cdec0 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f124effd700 1 -- 172.21.15.150:0/3932394018 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125c012c10 con 0x5608162cdec0 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.991+0000 7f124effd700 1 -- 172.21.15.150:0/3932394018 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c011b40 con 0x5608162cdec0 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3932394018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 msgr2=0x5608162d47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162d47c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d300 tx=0x7f125c00d610 comp rx=0 tx=0).stop 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3932394018 shutdown_connections 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162d47c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3932394018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cd570 0x5608162cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3932394018 >> 172.21.15.150:0/3932394018 conn(0x56081622ea60 msgr2=0x5608162c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3932394018 shutdown_connections 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3932394018 wait complete. 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.992+0000 7f127b2fcb80 1 -- start start 2024-05-10T04:43:03.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 1 Processor -- start 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cd570 0x5608162d6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cdec0 0x5608162ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560816188560 con 0x5608162cd570 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560816188200 con 0x5608162cc6a0 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081606f6a0 con 0x5608162cdec0 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f127b2fcb80 10 client.0 _ll_register_callbacks cb 0x560816191b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cd570 0x5608162d6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cd570 0x5608162d6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45584/0 (socket says 172.21.15.150:45584) 2024-05-10T04:43:04.000 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 -- 172.21.15.150:0/547460362 learned_addr learned my addr 172.21.15.150:0/547460362 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1269555700 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cdec0 0x5608162ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1268d54700 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 -- 172.21.15.150:0/547460362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cdec0 msgr2=0x5608162ddb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cdec0 0x5608162ddb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 -- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 msgr2=0x5608162d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.993+0000 7f1263fff700 1 -- 172.21.15.150:0/547460362 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c007d50 con 0x5608162cd570 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f1263fff700 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cd570 0x5608162d6be0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f125000fbb0 tx=0x7f1250009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56342f0b9f67a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f1243fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f1243fff700 1 -- 172.21.15.150:0/547460362 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125001d040 con 0x5608162cd570 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f1243fff700 1 -- 172.21.15.150:0/547460362 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f1250004a30 con 0x5608162cd570 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560816145270 con 0x5608162cd570 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f1243fff700 1 -- 172.21.15.150:0/547460362 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1250004bf0 con 0x5608162cd570 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 10 client.16773 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:04.001 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560816019b60 con 0x5608162cd570 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f1243fff700 1 -- 172.21.15.150:0/547460362 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1250038020 con 0x5608162cd570 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f1243fff700 1 -- 172.21.15.150:0/547460362 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1250005250 con 0x5608162cd570 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 10 client.16773 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 20 client.16773 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5608162e23f0 0x5608162e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 4 client.16773 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.994+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56081629b780 con 0x5608162e23f0 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.995+0000 7f1268d54700 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5608162e23f0 0x5608162e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.995+0000 7f1243fff700 10 client.16773 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.995+0000 7f1268d54700 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5608162e23f0 0x5608162e4840 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9fc069cfa75076 server_cookie=c86b42fad24ae9fa in_seq=0 out_seq=0 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f1243fff700 1 -- 172.21.15.150:0/547460362 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56081629b780 con 0x5608162e23f0 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f1243fff700 10 client.16773 handle_command_reply: tid=0 2024-05-10T04:43:04.002 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 client.16773 shutdown 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cd570 msgr2=0x5608162d6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cd570 0x5608162d6be0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f125000fbb0 tx=0x7f1250009310 comp rx=0 tx=0).stop 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 shutdown_connections 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x5608162e23f0 0x5608162e4840 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cdec0 0x5608162ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cc6a0 0x5608162d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f1243fff700 0 client.16773 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/547460362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cd570 0x5608162d6be0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 >> 172.21.15.150:0/547460362 conn(0x56081622ea60 msgr2=0x5608162c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 shutdown_connections 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 1 -- 172.21.15.150:0/547460362 wait complete. 2024-05-10T04:43:04.003 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:03.996+0000 7f127b2fcb80 20 client.16773 trim_cache size 0 max 16384 2024-05-10T04:43:04.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 Processor -- start 2024-05-10T04:43:04.012 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 -- start start 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608162c87b0 con 0x5608162cdec0 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608162cd340 con 0x5608162cd570 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.009+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608162e4810 con 0x5608162cc6a0 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45586/0 (socket says 172.21.15.150:45586) 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 -- 172.21.15.150:0/1988645985 learned_addr learned my addr 172.21.15.150:0/1988645985 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1268d54700 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1263fff700 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 -- 172.21.15.150:0/1988645985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 msgr2=0x5608162de120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:04.013 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162de120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 -- 172.21.15.150:0/1988645985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 msgr2=0x5608162ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162ddb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 -- 172.21.15.150:0/1988645985 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608161b2560 con 0x5608162cdec0 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f1269555700 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162c7ad0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d220 tx=0x7f125c007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdab51667e2fcadf server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f124f7fe700 1 -- 172.21.15.150:0/1988645985 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c022070 con 0x5608162cdec0 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f124f7fe700 1 -- 172.21.15.150:0/1988645985 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125c0123f0 con 0x5608162cdec0 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.010+0000 7f124f7fe700 1 -- 172.21.15.150:0/1988645985 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c0118f0 con 0x5608162cdec0 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 -- 172.21.15.150:0/1988645985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 msgr2=0x5608162c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162c7ad0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d220 tx=0x7f125c007ce0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 -- 172.21.15.150:0/1988645985 shutdown_connections 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162c7ad0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162de120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.014 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/1988645985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 -- 172.21.15.150:0/1988645985 >> 172.21.15.150:0/1988645985 conn(0x56081622ea60 msgr2=0x5608162c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:04.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 -- 172.21.15.150:0/1988645985 shutdown_connections 2024-05-10T04:43:04.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 -- 172.21.15.150:0/1988645985 wait complete. 2024-05-10T04:43:04.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 -- start start 2024-05-10T04:43:04.015 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.011+0000 7f127b2fcb80 1 Processor -- start 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162db280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f127b2fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608162dcc30 con 0x5608162cdec0 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608162dcda0 con 0x5608162cd570 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f127b2fcb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608162dcf60 con 0x5608162cc6a0 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f127b2fcb80 10 client.0 _ll_register_callbacks cb 0x560816191b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162dc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45598/0 (socket says 172.21.15.150:45598) 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 -- 172.21.15.150:0/3495552658 learned_addr learned my addr 172.21.15.150:0/3495552658 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1268d54700 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162db280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1263fff700 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 -- 172.21.15.150:0/3495552658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 msgr2=0x5608162db280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162db280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 -- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 msgr2=0x5608162db7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162db7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 -- 172.21.15.150:0/3495552658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56081606d3b0 con 0x5608162cdec0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1269555700 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162dc6f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f125c029040 tx=0x7f125c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932ee3b38a15924 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.012+0000 7f1243fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f1243fff700 1 -- 172.21.15.150:0/3495552658 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c022050 con 0x5608162cdec0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f1243fff700 1 -- 172.21.15.150:0/3495552658 <== mon.0 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f125c0117e0 con 0x5608162cdec0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f125c012c10 con 0x5608162cdec0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f1243fff700 1 -- 172.21.15.150:0/3495552658 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f125c0119a0 con 0x5608162cdec0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 10 client.16779 fetch_fsmap learned FSMap version 29 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560816071cd0 con 0x5608162cdec0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f1243fff700 1 -- 172.21.15.150:0/3495552658 <== mon.0 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125c012c10 con 0x5608162cdec0 2024-05-10T04:43:04.020 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f1243fff700 1 -- 172.21.15.150:0/3495552658 <== mon.0 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f125c039070 con 0x5608162cdec0 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 10 client.16779 fetch_fsmap finished waiting for FSMap version 29 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 20 client.16779 populate_metadata read hostname 'smithi150' 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560816338ba0 0x56081633afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 4 client.16779 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 --> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5608161e4ec0 con 0x560816338ba0 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.013+0000 7f1268d54700 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560816338ba0 0x56081633afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f1268d54700 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560816338ba0 0x56081633afc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97511e67c00468d2 server_cookie=4003fecf8c8d1ebe in_seq=0 out_seq=0 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f1243fff700 10 client.16779 ms_handle_connect on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f1243fff700 1 -- 172.21.15.150:0/3495552658 <== mds.0 v2:172.21.15.161:6838/3001288873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5608161e4ec0 con 0x560816338ba0 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f1243fff700 10 client.16779 handle_command_reply: tid=0 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 client.16779 shutdown 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 msgr2=0x5608162dc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:04.021 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162dc6f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f125c029040 tx=0x7f125c004cc0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 shutdown_connections 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:6838/3001288873,v1:172.21.15.161:6839/3001288873] conn(0x560816338ba0 0x56081633afc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608162cdec0 0x5608162dc6f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f1243fff700 0 client.16779 ms_handle_reset on v2:172.21.15.161:6838/3001288873 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5608162cc6a0 0x5608162db280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 --2- 172.21.15.150:0/3495552658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608162cd570 0x5608162db7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 >> 172.21.15.150:0/3495552658 conn(0x56081622ea60 msgr2=0x5608162c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 shutdown_connections 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 1 -- 172.21.15.150:0/3495552658 wait complete. 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.014+0000 7f127b2fcb80 20 client.16779 trim_cache size 0 max 16384 2024-05-10T04:43:04.022 INFO:teuthology.orchestra.run.smithi150.stdout:[] 2024-05-10T04:43:04.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12540a11c0 msgr2=0x7f12540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:04.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12540a11c0 0x7f12540a36d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f125000b2e0 tx=0x7f125001a040 comp rx=0 tx=0).stop 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641002c0 msgr2=0x7f1264197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641002c0 0x7f1264197f80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f125800eac0 tx=0x7f125800ee80 comp rx=0 tx=0).stop 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 shutdown_connections 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1264101d80 0x7f126419f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12641002c0 0x7f1264197f80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f12540a11c0 0x7f12540a36d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 --2- 172.21.15.150:0/83484902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12641013e0 0x7f12641984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 >> 172.21.15.150:0/83484902 conn(0x7f12640fb8f0 msgr2=0x7f1264106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 shutdown_connections 2024-05-10T04:43:04.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:04.015+0000 7f126afb8700 1 -- 172.21.15.150:0/83484902 wait complete. 2024-05-10T04:43:04.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:04 smithi161 ceph-mon[46912]: pgmap v376: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:04.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:04 smithi181 ceph-mon[47314]: pgmap v376: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:04.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:04 smithi150 ceph-mon[42404]: pgmap v376: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:05.030 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-10T04:43:05.030 INFO:tasks.fwd_scrub:done joining 2024-05-10T04:43:05.031 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-10T04:43:05.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:43:05.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 -- 172.21.15.150:0/3749828128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 msgr2=0x7f9d24102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3749828128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009990 tx=0x7f9d18009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 -- 172.21.15.150:0/3749828128 shutdown_connections 2024-05-10T04:43:05.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3749828128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d241025b0 0x7f9d24108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3749828128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d24100af0 0x7f9d24100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.191 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3749828128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102020 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 -- 172.21.15.150:0/3749828128 >> 172.21.15.150:0/3749828128 conn(0x7f9d240fc0e0 msgr2=0x7f9d240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 -- 172.21.15.150:0/3749828128 shutdown_connections 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.187+0000 7f9d2944e700 1 -- 172.21.15.150:0/3749828128 wait complete. 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.188+0000 7f9d2944e700 1 Processor -- start 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.188+0000 7f9d2944e700 1 -- start start 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.188+0000 7f9d2944e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d24100af0 0x7f9d241048b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.188+0000 7f9d2944e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.188+0000 7f9d2944e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d241025b0 0x7f9d24103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.188+0000 7f9d2944e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24103b70 con 0x7f9d24101c10 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d2944e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24103ce0 con 0x7f9d241025b0 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d2944e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241a2050 con 0x7f9d24100af0 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45606/0 (socket says 172.21.15.150:45606) 2024-05-10T04:43:05.192 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 -- 172.21.15.150:0/3792830716 learned_addr learned my addr 172.21.15.150:0/3792830716 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d22ffd700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d24100af0 0x7f9d241048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d237fe700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d241025b0 0x7f9d24103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 -- 172.21.15.150:0/3792830716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d24100af0 msgr2=0x7f9d241048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d24100af0 0x7f9d241048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 -- 172.21.15.150:0/3792830716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d241025b0 msgr2=0x7f9d24103490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d241025b0 0x7f9d24103490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 -- 172.21.15.150:0/3792830716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180095f0 con 0x7f9d24101c10 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d237fe700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d241025b0 0x7f9d24103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d227fc700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102f50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9d180098e0 tx=0x7f9d18009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d0bfff700 1 -- 172.21.15.150:0/3792830716 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1801d070 con 0x7f9d24101c10 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d241a2250 con 0x7f9d24101c10 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d0bfff700 1 -- 172.21.15.150:0/3792830716 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d18021410 con 0x7f9d24101c10 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d0bfff700 1 -- 172.21.15.150:0/3792830716 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18017650 con 0x7f9d24101c10 2024-05-10T04:43:05.193 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.189+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d241a26d0 con 0x7f9d24101c10 2024-05-10T04:43:05.194 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.191+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24068730 con 0x7f9d24101c10 2024-05-10T04:43:05.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.191+0000 7f9d0bfff700 1 -- 172.21.15.150:0/3792830716 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f9d1800fb10 con 0x7f9d24101c10 2024-05-10T04:43:05.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.192+0000 7f9d0bfff700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d0c0a1120 0x7f9d0c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.192+0000 7f9d0bfff700 1 -- 172.21.15.150:0/3792830716 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f9d180ce100 con 0x7f9d24101c10 2024-05-10T04:43:05.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.192+0000 7f9d22ffd700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d0c0a1120 0x7f9d0c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.195 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.193+0000 7f9d22ffd700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d0c0a1120 0x7f9d0c0a3630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9d24101b10 tx=0x7f9d14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:05.198 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.197+0000 7f9d0bfff700 1 -- 172.21.15.150:0/3792830716 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d18097580 con 0x7f9d24101c10 2024-05-10T04:43:05.400 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.398+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d241a28c0 con 0x7f9d24101c10 2024-05-10T04:43:05.401 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.400+0000 7f9d0bfff700 1 -- 172.21.15.150:0/3792830716 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24811 (secure 0 0 0) 0x7f9d1809aba0 con 0x7f9d24101c10 2024-05-10T04:43:05.402 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:43:05.402 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":90,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:40:13.400103+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:43:05.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d0c0a1120 msgr2=0x7f9d0c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d0c0a1120 0x7f9d0c0a3630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9d24101b10 tx=0x7f9d14009260 comp rx=0 tx=0).stop 2024-05-10T04:43:05.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 msgr2=0x7f9d24102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102f50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9d180098e0 tx=0x7f9d18009580 comp rx=0 tx=0).stop 2024-05-10T04:43:05.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 shutdown_connections 2024-05-10T04:43:05.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d241025b0 0x7f9d24103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.404 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f9d0c0a1120 0x7f9d0c0a3630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d24100af0 0x7f9d241048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 --2- 172.21.15.150:0/3792830716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d24101c10 0x7f9d24102f50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.402+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 >> 172.21.15.150:0/3792830716 conn(0x7f9d240fc0e0 msgr2=0x7f9d24073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:05.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.403+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 shutdown_connections 2024-05-10T04:43:05.405 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.403+0000 7f9d2944e700 1 -- 172.21.15.150:0/3792830716 wait complete. 2024-05-10T04:43:05.417 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:43:05.418 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:43:05.555 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 -- 172.21.15.150:0/1849291662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa80ffc20 msgr2=0x7fdaa8100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 --2- 172.21.15.150:0/1849291662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa80ffc20 0x7fdaa8100030 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009990 tx=0x7fda9c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 -- 172.21.15.150:0/1849291662 shutdown_connections 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 --2- 172.21.15.150:0/1849291662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaa8100570 0x7fdaa810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 --2- 172.21.15.150:0/1849291662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa81031e0 0x7fdaa81035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 --2- 172.21.15.150:0/1849291662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa80ffc20 0x7fdaa8100030 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 -- 172.21.15.150:0/1849291662 >> 172.21.15.150:0/1849291662 conn(0x7fdaa80fb8f0 msgr2=0x7fdaa80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 -- 172.21.15.150:0/1849291662 shutdown_connections 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 -- 172.21.15.150:0/1849291662 wait complete. 2024-05-10T04:43:05.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.554+0000 7fdaad862700 1 Processor -- start 2024-05-10T04:43:05.558 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaad862700 1 -- start start 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaad862700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa80ffc20 0x7fdaa81027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaad862700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8100570 0x7fdaa8100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaad862700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaa81031e0 0x7fdaa8101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaad862700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa8101a70 con 0x7fdaa8100570 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaad862700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa8101be0 con 0x7fdaa80ffc20 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaad862700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81a2050 con 0x7fdaa81031e0 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8100570 0x7fdaa8100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8100570 0x7fdaa8100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45622/0 (socket says 172.21.15.150:45622) 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 -- 172.21.15.150:0/3407815691 learned_addr learned my addr 172.21.15.150:0/3407815691 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa77fe700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaa81031e0 0x7fdaa8101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa6ffd700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa80ffc20 0x7fdaa81027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 -- 172.21.15.150:0/3407815691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaa81031e0 msgr2=0x7fdaa8101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaa81031e0 0x7fdaa8101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 -- 172.21.15.150:0/3407815691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa80ffc20 msgr2=0x7fdaa81027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa80ffc20 0x7fdaa81027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 -- 172.21.15.150:0/3407815691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0095f0 con 0x7fdaa8100570 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa6ffd700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa80ffc20 0x7fdaa81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.555+0000 7fdaa67fc700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8100570 0x7fdaa8100e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0095c0 tx=0x7fda9c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.556+0000 7fdaac860700 1 -- 172.21.15.150:0/3407815691 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c01d070 con 0x7fdaa8100570 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.556+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa81a2250 con 0x7fdaa8100570 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.556+0000 7fdaac860700 1 -- 172.21.15.150:0/3407815691 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda9c022470 con 0x7fdaa8100570 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.556+0000 7fdaac860700 1 -- 172.21.15.150:0/3407815691 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c017650 con 0x7fdaa8100570 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.556+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa81a2670 con 0x7fdaa8100570 2024-05-10T04:43:05.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.557+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa8109030 con 0x7fdaa8100570 2024-05-10T04:43:05.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.558+0000 7fdaac860700 1 -- 172.21.15.150:0/3407815691 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fda9c0225e0 con 0x7fdaa8100570 2024-05-10T04:43:05.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.558+0000 7fdaac860700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fda900a11c0 0x7fda900a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.561 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.559+0000 7fdaac860700 1 -- 172.21.15.150:0/3407815691 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fda9c0ce350 con 0x7fdaa8100570 2024-05-10T04:43:05.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.559+0000 7fdaa6ffd700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fda900a11c0 0x7fda900a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.562 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.559+0000 7fdaa6ffd700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fda900a11c0 0x7fda900a36d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda98005eb0 tx=0x7fda98005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:05.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.562+0000 7fdaac860700 1 -- 172.21.15.150:0/3407815691 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fda9c097800 con 0x7fdaa8100570 2024-05-10T04:43:05.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.759+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdaa810d530 con 0x7fdaa8100570 2024-05-10T04:43:05.761 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.760+0000 7fdaac860700 1 -- 172.21.15.150:0/3407815691 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda9c09ae20 con 0x7fdaa8100570 2024-05-10T04:43:05.761 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:43:05.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fda900a11c0 msgr2=0x7fda900a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fda900a11c0 0x7fda900a36d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda98005eb0 tx=0x7fda98005e40 comp rx=0 tx=0).stop 2024-05-10T04:43:05.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8100570 msgr2=0x7fdaa8100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8100570 0x7fdaa8100e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0095c0 tx=0x7fda9c00f690 comp rx=0 tx=0).stop 2024-05-10T04:43:05.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 shutdown_connections 2024-05-10T04:43:05.763 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fda900a11c0 0x7fda900a36d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaa81031e0 0x7fdaa8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa80ffc20 0x7fdaa81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 --2- 172.21.15.150:0/3407815691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa8100570 0x7fdaa8100e50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 >> 172.21.15.150:0/3407815691 conn(0x7fdaa80fb8f0 msgr2=0x7fdaa8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:05.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 shutdown_connections 2024-05-10T04:43:05.764 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.762+0000 7fdaad862700 1 -- 172.21.15.150:0/3407815691 wait complete. 2024-05-10T04:43:05.775 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:05 smithi150 ceph-mon[42404]: pgmap v377: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:05.775 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:05 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3792830716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:05.776 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:43:05.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:05 smithi161 ceph-mon[46912]: pgmap v377: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:05.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:05 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3792830716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:05.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:05 smithi181 ceph-mon[47314]: pgmap v377: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:05.905 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:05 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3792830716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 -- 172.21.15.150:0/1578421806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac107980 msgr2=0x7fa4ac109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 --2- 172.21.15.150:0/1578421806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac107980 0x7fa4ac109da0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400b0a0 tx=0x7fa4a400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 -- 172.21.15.150:0/1578421806 shutdown_connections 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 --2- 172.21.15.150:0/1578421806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac107980 0x7fa4ac109da0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 --2- 172.21.15.150:0/1578421806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac1026c0 0x7fa4ac104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 --2- 172.21.15.150:0/1578421806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac105020 0x7fa4ac107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 -- 172.21.15.150:0/1578421806 >> 172.21.15.150:0/1578421806 conn(0x7fa4ac0fc0e0 msgr2=0x7fa4ac0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:05.914 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 -- 172.21.15.150:0/1578421806 shutdown_connections 2024-05-10T04:43:05.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.912+0000 7fa4b1310700 1 -- 172.21.15.150:0/1578421806 wait complete. 2024-05-10T04:43:05.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 Processor -- start 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 -- start start 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac1026c0 0x7fa4ac101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac105020 0x7fa4ac1005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac107980 0x7fa4ac100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1011f0 con 0x7fa4ac105020 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac101360 con 0x7fa4ac107980 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.913+0000 7fa4b1310700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1a2150 con 0x7fa4ac1026c0 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac105020 0x7fa4ac1005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac105020 0x7fa4ac1005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45652/0 (socket says 172.21.15.150:45652) 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 -- 172.21.15.150:0/2703342123 learned_addr learned my addr 172.21.15.150:0/2703342123 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aaffd700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac1026c0 0x7fa4ac101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4ab7fe700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac107980 0x7fa4ac100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.917 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 -- 172.21.15.150:0/2703342123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac1026c0 msgr2=0x7fa4ac101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac1026c0 0x7fa4ac101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 -- 172.21.15.150:0/2703342123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac107980 msgr2=0x7fa4ac100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac107980 0x7fa4ac100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 -- 172.21.15.150:0/2703342123 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4009d00 con 0x7fa4ac105020 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4aa7fc700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac105020 0x7fa4ac1005d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00d950 tx=0x7fa49c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa493fff700 1 -- 172.21.15.150:0/2703342123 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c009940 con 0x7fa4ac105020 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa493fff700 1 -- 172.21.15.150:0/2703342123 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa49c010460 con 0x7fa4ac105020 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac1a23b0 con 0x7fa4ac105020 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa493fff700 1 -- 172.21.15.150:0/2703342123 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00f620 con 0x7fa4ac105020 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.914+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac1a2890 con 0x7fa4ac105020 2024-05-10T04:43:05.918 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.915+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ac0fdc90 con 0x7fa4ac105020 2024-05-10T04:43:05.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.918+0000 7fa493fff700 1 -- 172.21.15.150:0/2703342123 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa49c0108d0 con 0x7fa4ac105020 2024-05-10T04:43:05.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.918+0000 7fa493fff700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4940a11c0 0x7fa4940a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:05.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.918+0000 7fa493fff700 1 -- 172.21.15.150:0/2703342123 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fa49c0ccad0 con 0x7fa4ac105020 2024-05-10T04:43:05.919 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.918+0000 7fa4aaffd700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4940a11c0 0x7fa4940a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:05.920 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.919+0000 7fa4aaffd700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4940a11c0 0x7fa4940a36d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0005950 tx=0x7fa4a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:05.923 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:05.922+0000 7fa493fff700 1 -- 172.21.15.150:0/2703342123 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa49c095ec0 con 0x7fa4ac105020 2024-05-10T04:43:06.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.121+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4ac10d3d0 con 0x7fa4ac105020 2024-05-10T04:43:06.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.122+0000 7fa493fff700 1 -- 172.21.15.150:0/2703342123 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa49c016020 con 0x7fa4ac105020 2024-05-10T04:43:06.123 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4940a11c0 msgr2=0x7fa4940a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4940a11c0 0x7fa4940a36d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0005950 tx=0x7fa4a000b410 comp rx=0 tx=0).stop 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac105020 msgr2=0x7fa4ac1005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac105020 0x7fa4ac1005d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00d950 tx=0x7fa49c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 shutdown_connections 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4ac107980 0x7fa4ac100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa4940a11c0 0x7fa4940a36d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4ac1026c0 0x7fa4ac101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 --2- 172.21.15.150:0/2703342123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4ac105020 0x7fa4ac1005d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 >> 172.21.15.150:0/2703342123 conn(0x7fa4ac0fc0e0 msgr2=0x7fa4ac103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:06.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 shutdown_connections 2024-05-10T04:43:06.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.124+0000 7fa4b1310700 1 -- 172.21.15.150:0/2703342123 wait complete. 2024-05-10T04:43:06.139 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 -- 172.21.15.150:0/3261428265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77381031e0 msgr2=0x7f77381035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 --2- 172.21.15.150:0/3261428265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77381031e0 0x7f77381035f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f772c009990 tx=0x7f772c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 -- 172.21.15.150:0/3261428265 shutdown_connections 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 --2- 172.21.15.150:0/3261428265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7738100570 0x7f773810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 --2- 172.21.15.150:0/3261428265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77381031e0 0x7f77381035f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 --2- 172.21.15.150:0/3261428265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77380ffc20 0x7f7738100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 -- 172.21.15.150:0/3261428265 >> 172.21.15.150:0/3261428265 conn(0x7f77380fb8f0 msgr2=0x7f77380fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:06.278 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 -- 172.21.15.150:0/3261428265 shutdown_connections 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.274+0000 7f773f118700 1 -- 172.21.15.150:0/3261428265 wait complete. 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 Processor -- start 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 -- start start 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77380ffc20 0x7f77381027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7738100570 0x7f7738100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77381031e0 0x7f7738101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738101a70 con 0x7f7738100570 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738101be0 con 0x7f77381031e0 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773f118700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77381a1ff0 con 0x7f77380ffc20 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f7737fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7738100570 0x7f7738100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f7737fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7738100570 0x7f7738100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45656/0 (socket says 172.21.15.150:45656) 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f7737fff700 1 -- 172.21.15.150:0/982863059 learned_addr learned my addr 172.21.15.150:0/982863059 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773ceb4700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77380ffc20 0x7f77381027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.275+0000 7f773d6b5700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77381031e0 0x7f7738101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7737fff700 1 -- 172.21.15.150:0/982863059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77380ffc20 msgr2=0x7f77381027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7737fff700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77380ffc20 0x7f77381027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7737fff700 1 -- 172.21.15.150:0/982863059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77381031e0 msgr2=0x7f7738101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7737fff700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77381031e0 0x7f7738101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7737fff700 1 -- 172.21.15.150:0/982863059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0095f0 con 0x7f7738100570 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f773d6b5700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77381031e0 0x7f7738101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:06.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7737fff700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7738100570 0x7f7738100e50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f772400d950 tx=0x7f772400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:06.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7735ffb700 1 -- 172.21.15.150:0/982863059 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724009940 con 0x7f7738100570 2024-05-10T04:43:06.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77381a2250 con 0x7f7738100570 2024-05-10T04:43:06.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7735ffb700 1 -- 172.21.15.150:0/982863059 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7724010460 con 0x7f7738100570 2024-05-10T04:43:06.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f7735ffb700 1 -- 172.21.15.150:0/982863059 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400f620 con 0x7f7738100570 2024-05-10T04:43:06.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.276+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77381a2780 con 0x7f7738100570 2024-05-10T04:43:06.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.277+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7738109030 con 0x7f7738100570 2024-05-10T04:43:06.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.278+0000 7f7735ffb700 1 -- 172.21.15.150:0/982863059 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f77240108d0 con 0x7f7738100570 2024-05-10T04:43:06.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.279+0000 7f7735ffb700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77280a11c0 0x7f77280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.279+0000 7f7735ffb700 1 -- 172.21.15.150:0/982863059 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f77240cce30 con 0x7f7738100570 2024-05-10T04:43:06.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.279+0000 7f773ceb4700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77280a11c0 0x7f77280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.279+0000 7f773ceb4700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77280a11c0 0x7f77280a36d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f772c00b5c0 tx=0x7f772c011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:06.284 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.283+0000 7f7735ffb700 1 -- 172.21.15.150:0/982863059 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7724096220 con 0x7f7738100570 2024-05-10T04:43:06.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.481+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f773810d3f0 con 0x7f7738100570 2024-05-10T04:43:06.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.482+0000 7f7735ffb700 1 -- 172.21.15.150:0/982863059 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77240160a0 con 0x7f7738100570 2024-05-10T04:43:06.483 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:43:06.485 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77280a11c0 msgr2=0x7f77280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77280a11c0 0x7f77280a36d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f772c00b5c0 tx=0x7f772c011040 comp rx=0 tx=0).stop 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7738100570 msgr2=0x7f7738100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7738100570 0x7f7738100e50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f772400d950 tx=0x7f772400dc60 comp rx=0 tx=0).stop 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 shutdown_connections 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77381031e0 0x7f7738101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f77280a11c0 0x7f77280a36d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77380ffc20 0x7f77381027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 --2- 172.21.15.150:0/982863059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7738100570 0x7f7738100e50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 >> 172.21.15.150:0/982863059 conn(0x7f77380fb8f0 msgr2=0x7f7738106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 shutdown_connections 2024-05-10T04:43:06.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.484+0000 7f773f118700 1 -- 172.21.15.150:0/982863059 wait complete. 2024-05-10T04:43:06.498 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:43:06.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.632+0000 7f72d7927700 1 -- 172.21.15.150:0/1434685101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d01004a0 msgr2=0x7f72d01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.632+0000 7f72d7927700 1 --2- 172.21.15.150:0/1434685101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d01004a0 0x7f72d01028c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009990 tx=0x7f72cc009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.632+0000 7f72d7927700 1 -- 172.21.15.150:0/1434685101 shutdown_connections 2024-05-10T04:43:06.636 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.632+0000 7f72d7927700 1 --2- 172.21.15.150:0/1434685101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72d0105760 0x7f72d0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.632+0000 7f72d7927700 1 --2- 172.21.15.150:0/1434685101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d01004a0 0x7f72d01028c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.632+0000 7f72d7927700 1 --2- 172.21.15.150:0/1434685101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d0102e00 0x7f72d0105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.632+0000 7f72d7927700 1 -- 172.21.15.150:0/1434685101 >> 172.21.15.150:0/1434685101 conn(0x7f72d00fc0d0 msgr2=0x7f72d00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 -- 172.21.15.150:0/1434685101 shutdown_connections 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 -- 172.21.15.150:0/1434685101 wait complete. 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 Processor -- start 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 -- start start 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d01004a0 0x7f72d019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72d0102e00 0x7f72d019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d0105760 0x7f72d01a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d01a5ff0 con 0x7f72d0105760 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d01a6160 con 0x7f72d0102e00 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.633+0000 7f72d7927700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d01a6320 con 0x7f72d01004a0 2024-05-10T04:43:06.637 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d0105760 0x7f72d01a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d0105760 0x7f72d01a59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45670/0 (socket says 172.21.15.150:45670) 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 -- 172.21.15.150:0/1027158921 learned_addr learned my addr 172.21.15.150:0/1027158921 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d4ec2700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72d0102e00 0x7f72d019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d56c3700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d01004a0 0x7f72d019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 -- 172.21.15.150:0/1027158921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d01004a0 msgr2=0x7f72d019e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d01004a0 0x7f72d019e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 -- 172.21.15.150:0/1027158921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72d0102e00 msgr2=0x7f72d019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72d0102e00 0x7f72d019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 -- 172.21.15.150:0/1027158921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0095f0 con 0x7f72d0105760 2024-05-10T04:43:06.638 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d56c3700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d01004a0 0x7f72d019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:06.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d4ec2700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72d0102e00 0x7f72d019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:06.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d5ec4700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d0105760 0x7f72d01a59e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f72c800ef90 tx=0x7f72c800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:06.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72c67fc700 1 -- 172.21.15.150:0/1027158921 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8018070 con 0x7f72d0105760 2024-05-10T04:43:06.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72c67fc700 1 -- 172.21.15.150:0/1027158921 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c800f040 con 0x7f72d0105760 2024-05-10T04:43:06.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72c67fc700 1 -- 172.21.15.150:0/1027158921 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8003ea0 con 0x7f72d0105760 2024-05-10T04:43:06.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d01a65a0 con 0x7f72d0105760 2024-05-10T04:43:06.639 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.634+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d01a6a60 con 0x7f72d0105760 2024-05-10T04:43:06.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.635+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d00fdc80 con 0x7f72d0105760 2024-05-10T04:43:06.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.637+0000 7f72c67fc700 1 -- 172.21.15.150:0/1027158921 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f72c800cb00 con 0x7f72d0105760 2024-05-10T04:43:06.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.637+0000 7f72c67fc700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72bc0a11c0 0x7f72bc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.637+0000 7f72c67fc700 1 -- 172.21.15.150:0/1027158921 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f72c80cd2a0 con 0x7f72d0105760 2024-05-10T04:43:06.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.637+0000 7f72d56c3700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72bc0a11c0 0x7f72bc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.640 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.638+0000 7f72d56c3700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72bc0a11c0 0x7f72bc0a36d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b2e0 tx=0x7f72cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:06.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.641+0000 7f72c67fc700 1 -- 172.21.15.150:0/1027158921 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f72c8096720 con 0x7f72d0105760 2024-05-10T04:43:06.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:06 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3407815691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:43:06.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:06 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2703342123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:43:06.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.838+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f72d01139c0 con 0x7f72d0105760 2024-05-10T04:43:06.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.838+0000 7f72c67fc700 1 -- 172.21.15.150:0/1027158921 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f72c8099d40 con 0x7f72d0105760 2024-05-10T04:43:06.840 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:43:06.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72bc0a11c0 msgr2=0x7f72bc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.842 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72bc0a11c0 0x7f72bc0a36d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b2e0 tx=0x7f72cc005fb0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d0105760 msgr2=0x7f72d01a59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d0105760 0x7f72d01a59e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f72c800ef90 tx=0x7f72c800c450 comp rx=0 tx=0).stop 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 shutdown_connections 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72d0105760 0x7f72d01a59e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f72bc0a11c0 0x7f72bc0a36d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d01004a0 0x7f72d019e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 --2- 172.21.15.150:0/1027158921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72d0102e00 0x7f72d019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 >> 172.21.15.150:0/1027158921 conn(0x7f72d00fc0d0 msgr2=0x7f72d01012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 shutdown_connections 2024-05-10T04:43:06.843 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.841+0000 7f72d7927700 1 -- 172.21.15.150:0/1027158921 wait complete. 2024-05-10T04:43:06.854 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:06 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3407815691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:43:06.854 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:06 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2703342123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:43:06.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-10T04:43:06.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:06 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3407815691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:43:06.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:06 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2703342123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:43:06.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 -- 172.21.15.150:0/3808985271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc101be0 msgr2=0x7f3ffc101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 --2- 172.21.15.150:0/3808985271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc101be0 0x7f3ffc101ff0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009990 tx=0x7f3fe4009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.993 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 -- 172.21.15.150:0/3808985271 shutdown_connections 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 --2- 172.21.15.150:0/3808985271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc102580 0x7f3ffc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 --2- 172.21.15.150:0/3808985271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ffc100ac0 0x7f3ffc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 --2- 172.21.15.150:0/3808985271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc101be0 0x7f3ffc101ff0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 -- 172.21.15.150:0/3808985271 >> 172.21.15.150:0/3808985271 conn(0x7f3ffc0fc0d0 msgr2=0x7f3ffc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 -- 172.21.15.150:0/3808985271 shutdown_connections 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.987+0000 7f4001a93700 1 -- 172.21.15.150:0/3808985271 wait complete. 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 Processor -- start 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 -- start start 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc100ac0 0x7f3ffc196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc101be0 0x7f3ffc1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ffc102580 0x7f3ffc19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc19ea20 con 0x7f3ffc100ac0 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc19eb60 con 0x7f3ffc101be0 2024-05-10T04:43:06.994 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f4001a93700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc19ecf0 con 0x7f3ffc102580 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f3ffb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc100ac0 0x7f3ffc196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f3ffb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc100ac0 0x7f3ffc196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45696/0 (socket says 172.21.15.150:45696) 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.988+0000 7f3ffb7fe700 1 -- 172.21.15.150:0/1949205764 learned_addr learned my addr 172.21.15.150:0/1949205764 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffbfff700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ffc102580 0x7f3ffc19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffaffd700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc101be0 0x7f3ffc1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffb7fe700 1 -- 172.21.15.150:0/1949205764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ffc102580 msgr2=0x7f3ffc19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffb7fe700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ffc102580 0x7f3ffc19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffb7fe700 1 -- 172.21.15.150:0/1949205764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc101be0 msgr2=0x7f3ffc1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffb7fe700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc101be0 0x7f3ffc1974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffb7fe700 1 -- 172.21.15.150:0/1949205764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe40095f0 con 0x7f3ffc100ac0 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffbfff700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ffc102580 0x7f3ffc19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-10T04:43:06.995 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ffb7fe700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc100ac0 0x7f3ffc196f80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098f0 tx=0x7f3fec009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ff8ff9700 1 -- 172.21.15.150:0/1949205764 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec012070 con 0x7f3ffc100ac0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc068420 con 0x7f3ffc100ac0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ff8ff9700 1 -- 172.21.15.150:0/1949205764 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fec00d850 con 0x7f3ffc100ac0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f3ff8ff9700 1 -- 172.21.15.150:0/1949205764 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec007c10 con 0x7f3ffc100ac0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.989+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc068980 con 0x7f3ffc100ac0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.990+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc069370 con 0x7f3ffc100ac0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.991+0000 7f3ff8ff9700 1 -- 172.21.15.150:0/1949205764 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f3fec016070 con 0x7f3ffc100ac0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.992+0000 7f3ff8ff9700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3fe80a1120 0x7f3fe80a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.992+0000 7f3ffaffd700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3fe80a1120 0x7f3fe80a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:06.996 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.992+0000 7f3ff8ff9700 1 -- 172.21.15.150:0/1949205764 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f3fec0ce6a0 con 0x7f3ffc100ac0 2024-05-10T04:43:06.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.992+0000 7f3ffaffd700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3fe80a1120 0x7f3fe80a3630 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005b40 tx=0x7f3fe4005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:06.997 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:06.995+0000 7f3ff8ff9700 1 -- 172.21.15.150:0/1949205764 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3fec097a90 con 0x7f3ffc100ac0 2024-05-10T04:43:07.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.250+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f3ffc10c310 con 0x7f3ffc100ac0 2024-05-10T04:43:07.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.258+0000 7f3ff8ff9700 1 -- 172.21.15.150:0/1949205764 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733418 (secure 0 0 0) 0x7f3fec09b0b0 con 0x7f3ffc100ac0 2024-05-10T04:43:07.270 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "epoch": 18, 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "active_gid": 14152, 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "active_name": "x", 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "active_addrs": { 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6800", 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3064659046 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v1", 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:6801", 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3064659046 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "active_addr": "172.21.15.150:6801/3064659046", 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "active_change": "2024-05-10T04:33:11.238519+0000", 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "active_mgr_features": 4540138322906710015, 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "available": true, 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "standbys": [ 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "gid": 24112, 2024-05-10T04:43:07.272 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "z", 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_features": 4540138322906710015, 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "available_modules": [ 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "alerts", 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.273 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.274 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.275 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.276 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.277 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.278 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.279 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_destination": { 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_destination", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email address to send alerts to", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_from_name": { 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_from_name", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "Ceph", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email From: name", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.281 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_host": { 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_host", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP server", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.282 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_password": { 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_password", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Password to authenticate with", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.283 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_port": { 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_port", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "465", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP port", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.284 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_sender": { 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_sender", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP envelope sender", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.285 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_ssl": { 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_ssl", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.286 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_user": { 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_user", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "User to authenticate as", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "balancer", 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "active": { 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "active", 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.288 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_time": { 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_time", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.289 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0000", 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.290 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_weekday": { 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_weekday", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_max_iterations": { 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_max_iterations", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "25", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "250", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.292 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_metrics": { 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_metrics", 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T04:43:07.293 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_step": { 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_step", 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.5", 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0.001", 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "0.999", 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.294 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "aggressiveness of optimization", 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "end_time": { 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_time", 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2359", 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.295 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "end_weekday": { 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_weekday", 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.296 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.298 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.299 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.300 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.301 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.302 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "min_score": { 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_score", 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.303 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "upmap", 2024-05-10T04:43:07.304 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-compat", 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "none", 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap" 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Balancer mode", 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.305 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_ids": { 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_ids", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.306 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.307 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_deviation": { 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_deviation", 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T04:43:07.308 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_optimizations": { 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_optimizations", 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.309 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cephadm", 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_down_multiplier": { 2024-05-10T04:43:07.310 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_down_multiplier", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.311 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_refresh_rate": { 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_refresh_rate", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "20", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_starting_port": { 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_starting_port", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4721", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_ptrace": { 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_ptrace", 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.314 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_interval": { 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_interval", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.315 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_memory_target_ratio": { 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.7", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.316 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "cgroups_split": { 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cgroups_split", 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.317 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "config_checks_enabled": { 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_checks_enabled", 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.318 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "config_dashboard": { 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_dashboard", 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.319 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_alertmanager": { 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_alertmanager", 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.320 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_base": { 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_base", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.321 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Container image name, without the tag", 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_elasticsearch": { 2024-05-10T04:43:07.322 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_elasticsearch", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "elasticsearch container image", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_grafana": { 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_grafana", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.324 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_haproxy": { 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_haproxy", 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "HAproxy container image", 2024-05-10T04:43:07.325 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_agent": { 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_agent", 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.326 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger agent container image", 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_collector": { 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_collector", 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.327 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger collector container image", 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_query": { 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_query", 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.328 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger query container image", 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_keepalived": { 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_keepalived", 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.329 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Keepalived container image", 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_loki": { 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_loki", 2024-05-10T04:43:07.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.331 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.331 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T04:43:07.331 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Loki container image", 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.334 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_node_exporter": { 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_node_exporter", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.335 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_nvmeof": { 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_nvmeof", 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Nvme-of container image", 2024-05-10T04:43:07.336 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_prometheus": { 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_prometheus", 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.337 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_promtail": { 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_promtail", 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.338 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Promtail container image", 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_snmp_gateway": { 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_snmp_gateway", 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T04:43:07.339 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SNMP Gateway container image", 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "container_init": { 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_init", 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.340 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "daemon_cache_timeout": { 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "daemon_cache_timeout", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "default_cephadm_command_timeout": { 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.342 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "900", 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "default_registry": { 2024-05-10T04:43:07.343 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_registry", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.344 INFO:teuthology.orchestra.run.smithi150.stdout: "device_cache_timeout": { 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_cache_timeout", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1800", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache device inventory", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "device_enhanced_scan": { 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_enhanced_scan", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.346 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "facts_cache_timeout": { 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "facts_cache_timeout", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache host facts data", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.347 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "host_check_interval": { 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "host_check_interval", 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.348 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to perform a host check", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "hw_monitoring": { 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hw_monitoring", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.349 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "inventory_list_all": { 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "inventory_list_all", 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.350 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.351 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.352 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "log_refresh_metadata": { 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_refresh_metadata", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.353 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.354 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.355 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.356 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.357 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T04:43:07.358 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "*", 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.359 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "max_count_per_host": { 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_count_per_host", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.360 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of daemons per service per host", 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd_draining_count": { 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_osd_draining_count", 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.361 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "migration_current": { 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "migration_current", 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.362 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "internal - do not modify", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.363 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "root", 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm-package", 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "root" 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "oob_default_addr": { 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "oob_default_addr", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "169.254.1.1", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.365 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "prometheus_alerts_path": { 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus_alerts_path", 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T04:43:07.366 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_insecure": { 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_insecure", 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.367 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_password": { 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_password", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.368 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_url": { 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_url", 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.369 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_username": { 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_username", 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.370 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "secure_monitoring_stack": { 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "secure_monitoring_stack", 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.371 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "service_discovery_port": { 2024-05-10T04:43:07.372 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "service_discovery_port", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8765", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "cephadm service discovery port", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.373 INFO:teuthology.orchestra.run.smithi150.stdout: "ssh_config_file": { 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssh_config_file", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.374 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "use_agent": { 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_agent", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "use_repo_digest": { 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_repo_digest", 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T04:43:07.376 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_failed_host_check": { 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_failed_host_check", 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.377 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_daemons": { 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_daemons", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.378 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_hosts": { 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_hosts", 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.379 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crash", 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.380 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.381 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.383 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.384 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.385 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "retain_interval": { 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retain_interval", 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "31536000", 2024-05-10T04:43:07.386 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_recent_interval": { 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_recent_interval", 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.387 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1209600", 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dashboard", 2024-05-10T04:43:07.388 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.389 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.390 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.391 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_ENABLED": { 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.392 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.393 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.394 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.395 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.396 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.397 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T04:43:07.398 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.400 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.401 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.402 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.403 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.404 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.405 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_URL": { 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_URL", 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.406 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.407 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.409 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T04:43:07.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.411 INFO:teuthology.orchestra.run.smithi150.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.412 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.413 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.414 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.415 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.418 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.420 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.421 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T04:43:07.422 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.424 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.425 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8", 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.426 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "45", 2024-05-10T04:43:07.427 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.428 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T04:43:07.430 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.431 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.432 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.434 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.436 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "cross_origin_url": { 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cross_origin_url", 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.437 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "crt_file": { 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crt_file", 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.438 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "debug": { 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "debug", 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.439 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable/disable debug options", 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "jwt_token_ttl": { 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "jwt_token_ttl", 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.440 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "28800", 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:43:07.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.442 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.443 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.444 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.445 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.446 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.447 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "motd": { 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "motd", 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.448 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "The message of the day", 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect_resolve_ip_addr": { 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.449 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.450 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.451 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8080", 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:43:07.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl_server_port": { 2024-05-10T04:43:07.453 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl_server_port", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8443", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.454 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "redirect", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect" 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.455 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.456 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "url_prefix": { 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url_prefix", 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.457 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.458 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_monitoring": { 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_monitoring", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "monitor device health metrics", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.459 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.460 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.461 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.462 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.462 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.462 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.462 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.462 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.463 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.465 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "mark_out_threshold": { 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mark_out_threshold", 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2419200", 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.466 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": { 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_name", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "device_health_metrics", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "retention_period": { 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retention_period", 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15552000", 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain device health metrics", 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_frequency": { 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_frequency", 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.469 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "self_heal": { 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "self_heal", 2024-05-10T04:43:07.470 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.471 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.472 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_threshold": { 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_threshold", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7257600", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "diskprediction_local", 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.474 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.475 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.478 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.479 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.480 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "predict_interval": { 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predict_interval", 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.482 INFO:teuthology.orchestra.run.smithi150.stdout: "predictor_model": { 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predictor_model", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "prophetstor", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.484 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "influx", 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": false, 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "influxdb python module not found", 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "batch_size": { 2024-05-10T04:43:07.485 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "batch_size", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5000", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "database": { 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "database", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "ceph", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.487 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": { 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hostname", 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.488 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server hostname", 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "30", 2024-05-10T04:43:07.489 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "5", 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.490 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.491 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.492 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.493 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.494 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.495 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.496 INFO:teuthology.orchestra.run.smithi150.stdout: "password": { 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "password", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "password of InfluxDB server user", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.497 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "port": { 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "port", 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8086", 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.498 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server port", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "false", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.499 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "threads": { 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threads", 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:07.500 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "32", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "username": { 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "username", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.501 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "username of InfluxDB server user", 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "verify_ssl": { 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "verify_ssl", 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.502 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "true", 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "insights", 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.503 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.504 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.505 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.506 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.507 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.508 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.509 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "iostat", 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.510 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.511 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.513 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.514 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.515 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.516 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "k8sevents", 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_event_retention_days": { 2024-05-10T04:43:07.517 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ceph_event_retention_days", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.518 INFO:teuthology.orchestra.run.smithi150.stdout: "config_check_secs": { 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_check_secs", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "10", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.520 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.521 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.523 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.524 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.525 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "localpool", 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: "failure_domain": { 2024-05-10T04:43:07.526 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "failure_domain", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "host", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "failure domain for any created local pool", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T04:43:07.527 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.528 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.530 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.531 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.532 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.533 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": { 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_size", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default min_size for any created local pool", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T04:43:07.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.535 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.535 INFO:teuthology.orchestra.run.smithi150.stdout: "num_rep": { 2024-05-10T04:43:07.535 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "num_rep", 2024-05-10T04:43:07.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.535 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3", 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default replica count for any created local pool", 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": { 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_num", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "128", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.540 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "prefix": { 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prefix", 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name prefix for any created local pool", 2024-05-10T04:43:07.541 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "subtree": { 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "subtree", 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "rack", 2024-05-10T04:43:07.542 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mds_autoscaler", 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.543 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.544 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.545 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.546 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.547 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.548 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.549 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirroring", 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.552 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.553 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.555 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.556 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "nfs", 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.557 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.558 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.559 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.561 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.562 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.563 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:43:07.564 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "fail_fs": { 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "fail_fs", 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.565 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.566 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.567 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.568 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.569 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator": { 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:43:07.572 INFO:teuthology.orchestra.run.smithi150.stdout: "rook", 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "test_orchestrator" 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Orchestrator backend", 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_perf_query", 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.573 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.574 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.576 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.577 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.578 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_support", 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.580 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.581 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.582 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.584 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.586 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_autoscaler", 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.587 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.588 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.589 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.590 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.591 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.592 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "threshold": { 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threshold", 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:43:07.595 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1.0", 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "scaling threshold", 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.596 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "progress", 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_pg_recovery_event": { 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_pg_recovery_event", 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.597 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.598 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.599 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.600 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.601 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.602 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.603 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.604 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "max_completed_events": { 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_completed_events", 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.605 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "50", 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "number of past completed events to remember", 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.606 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long the module is going to sleep", 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.607 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "cache": { 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cache", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.608 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "exclude_perf_counters": { 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "exclude_perf_counters", 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.609 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.611 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.612 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.613 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.614 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.615 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools": { 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools", 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.616 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.617 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "300", 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.618 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_interval": { 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_interval", 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15.0", 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.619 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.620 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "9283", 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T04:43:07.622 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "stale_cache_strategy": { 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stale_cache_strategy", 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "log", 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.623 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.624 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "default", 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "default", 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "error" 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.626 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.628 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.629 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.630 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.631 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.633 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_snap_create": { 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_snap_create", 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.634 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "mirror_snapshot_schedule": { 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.635 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "trash_purge_schedule": { 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "trash_purge_schedule", 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.636 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "restful", 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_auth": { 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_auth", 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.638 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.639 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.641 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.642 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.643 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.644 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.645 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.646 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.647 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rgw", 2024-05-10T04:43:07.648 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.649 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.650 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.651 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.652 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.653 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.654 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rook", 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.657 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.659 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.660 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.661 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "storage_class": { 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "storage_class", 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.662 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "local", 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.663 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "selftest", 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "roption1": { 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption1", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "roption2": { 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption2", 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "xyz", 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption1": { 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption1", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption2": { 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption2", 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption3": { 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption3", 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption4": { 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption4", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption5": { 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption5", 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption6": { 2024-05-10T04:43:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption6", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption7": { 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption7", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "42", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "testkey": { 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testkey", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "testlkey": { 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testlkey", 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "testnewline": { 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testnewline", 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snap_schedule", 2024-05-10T04:43:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_m_granularity": { 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_m_granularity", 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "dump_on_update": { 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dump_on_update", 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "dump database to debug log on update", 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stats", 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "status", 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telegraf", 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "address": { 2024-05-10T04:43:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "address", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15", 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telemetry", 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_basic": { 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_basic", 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_crash": { 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_crash", 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_device": { 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_device", 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_ident": { 2024-05-10T04:43:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_ident", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_perf": { 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_perf", 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "contact": { 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "contact", 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "description": { 2024-05-10T04:43:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "description", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "device_url": { 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_url", 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "24", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "8", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "last_opt_revision": { 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "last_opt_revision", 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1", 2024-05-10T04:43:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard": { 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard", 2024-05-10T04:43:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard_description": { 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard_description", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "organization": { 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "organization", 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "proxy": { 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "proxy", 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "url": { 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url", 2024-05-10T04:43:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "test_orchestrator", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_clones": { 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_clones", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_delay": { 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_delay", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_no_wait": { 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T04:43:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "discovery_interval": { 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "discovery_interval", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "100", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "identifier": { 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "identifier", 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_host": { 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_host", 2024-05-10T04:43:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_port": { 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_port", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10051", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_sender": { 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_sender", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.766 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "gid": 24113, 2024-05-10T04:43:07.767 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "y", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "mgr_features": 4540138322906710015, 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "available_modules": [ 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "alerts", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.768 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.770 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.771 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.773 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_destination": { 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_destination", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email address to send alerts to", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.774 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_from_name": { 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_from_name", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "Ceph", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email From: name", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_host": { 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_host", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP server", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_password": { 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_password", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Password to authenticate with", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_port": { 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_port", 2024-05-10T04:43:07.777 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "465", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP port", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_sender": { 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_sender", 2024-05-10T04:43:07.778 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP envelope sender", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_ssl": { 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_ssl", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.779 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_user": { 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_user", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.780 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "User to authenticate as", 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "balancer", 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "active": { 2024-05-10T04:43:07.781 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "active", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_time": { 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_time", 2024-05-10T04:43:07.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0000", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_weekday": { 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_weekday", 2024-05-10T04:43:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_max_iterations": { 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_max_iterations", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "25", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "250", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_metrics": { 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_metrics", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.785 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_step": { 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_step", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.786 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.5", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0.001", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "0.999", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "aggressiveness of optimization", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "end_time": { 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_time", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2359", 2024-05-10T04:43:07.787 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "end_weekday": { 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_weekday", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:43:07.788 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.789 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.790 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.791 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.792 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.793 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "min_score": { 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_score", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "upmap", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-compat", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "none", 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap" 2024-05-10T04:43:07.795 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Balancer mode", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_ids": { 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_ids", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.796 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T04:43:07.797 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_deviation": { 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_deviation", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T04:43:07.798 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_optimizations": { 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_optimizations", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.799 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cephadm", 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_down_multiplier": { 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_down_multiplier", 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.800 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_refresh_rate": { 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_refresh_rate", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "20", 2024-05-10T04:43:07.801 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_starting_port": { 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_starting_port", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4721", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.802 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_ptrace": { 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_ptrace", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.803 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_interval": { 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_interval", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.804 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_memory_target_ratio": { 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.7", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "cgroups_split": { 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cgroups_split", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.806 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "config_checks_enabled": { 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_checks_enabled", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "config_dashboard": { 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_dashboard", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.808 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_alertmanager": { 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_alertmanager", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_base": { 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_base", 2024-05-10T04:43:07.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Container image name, without the tag", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_elasticsearch": { 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_elasticsearch", 2024-05-10T04:43:07.810 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "elasticsearch container image", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_grafana": { 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_grafana", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.811 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_haproxy": { 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_haproxy", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "HAproxy container image", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_agent": { 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_agent", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.813 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger agent container image", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_collector": { 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_collector", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.814 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger collector container image", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_query": { 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_query", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.815 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger query container image", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_keepalived": { 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_keepalived", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Keepalived container image", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.816 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_loki": { 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_loki", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Loki container image", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_node_exporter": { 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_node_exporter", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_nvmeof": { 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_nvmeof", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Nvme-of container image", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_prometheus": { 2024-05-10T04:43:07.819 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_prometheus", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_promtail": { 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_promtail", 2024-05-10T04:43:07.820 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Promtail container image", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_snmp_gateway": { 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_snmp_gateway", 2024-05-10T04:43:07.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SNMP Gateway container image", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "container_init": { 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_init", 2024-05-10T04:43:07.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "daemon_cache_timeout": { 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "daemon_cache_timeout", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.823 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "default_cephadm_command_timeout": { 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "900", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "default_registry": { 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_registry", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "device_cache_timeout": { 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_cache_timeout", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.826 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1800", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache device inventory", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "device_enhanced_scan": { 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_enhanced_scan", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.827 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "facts_cache_timeout": { 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "facts_cache_timeout", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.828 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache host facts data", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "host_check_interval": { 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "host_check_interval", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.829 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to perform a host check", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "hw_monitoring": { 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hw_monitoring", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.830 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "inventory_list_all": { 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "inventory_list_all", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.831 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.832 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "log_refresh_metadata": { 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_refresh_metadata", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.833 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.835 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T04:43:07.837 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T04:43:07.838 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "*", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "max_count_per_host": { 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_count_per_host", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.839 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of daemons per service per host", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd_draining_count": { 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_osd_draining_count", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.840 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "migration_current": { 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "migration_current", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.841 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "internal - do not modify", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.842 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "root", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm-package", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "root" 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "oob_default_addr": { 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "oob_default_addr", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.843 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "169.254.1.1", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "prometheus_alerts_path": { 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus_alerts_path", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.844 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_insecure": { 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_insecure", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.845 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_password": { 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_password", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.846 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_url": { 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_url", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_username": { 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_username", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.848 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "secure_monitoring_stack": { 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "secure_monitoring_stack", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "service_discovery_port": { 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "service_discovery_port", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8765", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "cephadm service discovery port", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.850 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "ssh_config_file": { 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssh_config_file", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "use_agent": { 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_agent", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.852 INFO:teuthology.orchestra.run.smithi150.stdout: "use_repo_digest": { 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_repo_digest", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_failed_host_check": { 2024-05-10T04:43:07.853 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_failed_host_check", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_daemons": { 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_daemons", 2024-05-10T04:43:07.854 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_hosts": { 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_hosts", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.855 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crash", 2024-05-10T04:43:07.856 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.857 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.858 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.859 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.860 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "retain_interval": { 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retain_interval", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.861 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "31536000", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_recent_interval": { 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_recent_interval", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.862 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1209600", 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dashboard", 2024-05-10T04:43:07.863 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.865 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.866 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_ENABLED": { 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T04:43:07.867 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T04:43:07.868 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.869 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.870 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.871 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.872 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.873 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.874 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.875 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.876 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.877 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.878 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_URL": { 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_URL", 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.880 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.882 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.883 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.884 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.885 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.886 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T04:43:07.887 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T04:43:07.888 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.889 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.890 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.891 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.892 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.893 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.894 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.895 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.896 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.897 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "45", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.898 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.899 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T04:43:07.900 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.901 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.902 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.904 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.905 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "cross_origin_url": { 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cross_origin_url", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.906 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "crt_file": { 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crt_file", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.907 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "debug": { 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "debug", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable/disable debug options", 2024-05-10T04:43:07.908 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "jwt_token_ttl": { 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "jwt_token_ttl", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "28800", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.909 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.911 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.912 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.913 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.914 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "motd": { 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "motd", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.915 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "The message of the day", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect_resolve_ip_addr": { 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.916 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:43:07.917 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8080", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.918 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.919 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl_server_port": { 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl_server_port", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8443", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.920 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "redirect", 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect" 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.921 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.922 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "url_prefix": { 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url_prefix", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.923 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_monitoring": { 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_monitoring", 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.924 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "monitor device health metrics", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.925 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.926 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.927 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.928 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "mark_out_threshold": { 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mark_out_threshold", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2419200", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.930 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": { 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_name", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "device_health_metrics", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "retention_period": { 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retention_period", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15552000", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain device health metrics", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_frequency": { 2024-05-10T04:43:07.932 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_frequency", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "self_heal": { 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "self_heal", 2024-05-10T04:43:07.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.934 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_threshold": { 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_threshold", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7257600", 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "diskprediction_local", 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.936 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.937 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.938 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.939 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "predict_interval": { 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predict_interval", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.941 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "predictor_model": { 2024-05-10T04:43:07.942 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predictor_model", 2024-05-10T04:43:07.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.945 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.945 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "prophetstor", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "influx", 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": false, 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "influxdb python module not found", 2024-05-10T04:43:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "batch_size": { 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "batch_size", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5000", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "database": { 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "database", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "ceph", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": { 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hostname", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server hostname", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "30", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "5", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "password": { 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "password", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "password of InfluxDB server user", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "port": { 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "port", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8086", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server port", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "false", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "threads": { 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threads", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "32", 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "username": { 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "username", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "username of InfluxDB server user", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "verify_ssl": { 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "verify_ssl", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "true", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "insights", 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.962 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.963 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.965 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.966 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "iostat", 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.968 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.969 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.970 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.971 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.972 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "k8sevents", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_event_retention_days": { 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ceph_event_retention_days", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.973 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "config_check_secs": { 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_check_secs", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "10", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.974 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.975 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.976 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.978 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "localpool", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "failure_domain": { 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "failure_domain", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "host", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "failure domain for any created local pool", 2024-05-10T04:43:07.980 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.981 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.982 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.983 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.985 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": { 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_size", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default min_size for any created local pool", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.986 INFO:teuthology.orchestra.run.smithi150.stdout: "num_rep": { 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "num_rep", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default replica count for any created local pool", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": { 2024-05-10T04:43:07.987 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_num", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "128", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "prefix": { 2024-05-10T04:43:07.988 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prefix", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name prefix for any created local pool", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "subtree": { 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "subtree", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.989 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "rack", 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.990 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mds_autoscaler", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.991 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.992 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.993 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.994 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.995 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirroring", 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:07.998 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:07.999 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.000 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "nfs", 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.001 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.002 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.003 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.004 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.005 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.006 INFO:teuthology.orchestra.run.smithi150.stdout: "fail_fs": { 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "fail_fs", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.007 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.008 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.009 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.010 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator": { 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.012 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "rook", 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "test_orchestrator" 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Orchestrator backend", 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_perf_query", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.014 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.016 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.017 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_support", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.019 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.020 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.022 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_autoscaler", 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.024 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.026 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.027 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.028 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.029 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "threshold": { 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threshold", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1.0", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "scaling threshold", 2024-05-10T04:43:08.030 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "progress", 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_pg_recovery_event": { 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_pg_recovery_event", 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.031 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.032 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.033 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.034 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.035 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.036 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.037 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "max_completed_events": { 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_completed_events", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "50", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "number of past completed events to remember", 2024-05-10T04:43:08.038 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long the module is going to sleep", 2024-05-10T04:43:08.039 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus", 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "cache": { 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cache", 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.040 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "exclude_perf_counters": { 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "exclude_perf_counters", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.041 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.042 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.043 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.044 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.046 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools": { 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "300", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_interval": { 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_interval", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15.0", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.049 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:08.050 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "9283", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "stale_cache_strategy": { 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stale_cache_strategy", 2024-05-10T04:43:08.051 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "log", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.052 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "default", 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "default", 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "error" 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:43:08.053 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.055 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.056 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.057 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.058 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.059 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_snap_create": { 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_snap_create", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "mirror_snapshot_schedule": { 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: "trash_purge_schedule": { 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "trash_purge_schedule", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.062 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "restful", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_auth": { 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_auth", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.063 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.064 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.065 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.066 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.067 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.068 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:08.069 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:08.070 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.071 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rgw", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.072 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.073 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.074 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rook", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.077 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.078 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.079 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.080 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.081 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "storage_class": { 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "storage_class", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "local", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.082 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "selftest", 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.083 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.084 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.085 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.086 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.087 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "roption1": { 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption1", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.088 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "roption2": { 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption2", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "xyz", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.089 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption1": { 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption1", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption2": { 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption2", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption3": { 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption3", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption4": { 2024-05-10T04:43:08.092 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption4", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption5": { 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption5", 2024-05-10T04:43:08.093 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption6": { 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption6", 2024-05-10T04:43:08.094 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption7": { 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption7", 2024-05-10T04:43:08.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "42", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "testkey": { 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testkey", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.096 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "testlkey": { 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testlkey", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.097 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "testnewline": { 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testnewline", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.098 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snap_schedule", 2024-05-10T04:43:08.099 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_m_granularity": { 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_m_granularity", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.100 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "dump_on_update": { 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dump_on_update", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "dump database to debug log on update", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.101 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.102 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.103 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.104 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.105 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stats", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.107 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.109 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.110 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.111 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "status", 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.112 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.113 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.114 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.115 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.116 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telegraf", 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.117 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "address": { 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "address", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.118 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.120 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.121 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.122 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.123 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telemetry", 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_basic": { 2024-05-10T04:43:08.124 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_basic", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_crash": { 2024-05-10T04:43:08.125 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_crash", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_device": { 2024-05-10T04:43:08.126 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_device", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_ident": { 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_ident", 2024-05-10T04:43:08.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_perf": { 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_perf", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.128 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "contact": { 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "contact", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.129 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "description": { 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "description", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.130 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "device_url": { 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_url", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T04:43:08.131 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.132 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "24", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "8", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "last_opt_revision": { 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "last_opt_revision", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.134 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard": { 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.135 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard_description": { 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard_description", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.136 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.137 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.138 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.139 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.140 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "organization": { 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "organization", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.141 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "proxy": { 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "proxy", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.142 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "url": { 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T04:43:08.143 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "test_orchestrator", 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.144 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.145 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.146 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.147 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.148 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.149 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.150 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.151 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.152 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.153 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_clones": { 2024-05-10T04:43:08.154 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_clones", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_delay": { 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_delay", 2024-05-10T04:43:08.155 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_no_wait": { 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.156 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix", 2024-05-10T04:43:08.157 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "discovery_interval": { 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "discovery_interval", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "100", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.158 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "identifier": { 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "identifier", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.159 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.160 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.161 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.162 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.163 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.164 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_host": { 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_host", 2024-05-10T04:43:08.165 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_port": { 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_port", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.166 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10051", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_sender": { 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_sender", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.167 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.168 INFO:teuthology.orchestra.run.smithi150.stdout: "modules": [ 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "dashboard", 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "iostat", 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "nfs", 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "restful" 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "available_modules": [ 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "alerts", 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.169 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How frequently to reexamine health status", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.171 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.172 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.173 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.174 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_destination": { 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_destination", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.175 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email address to send alerts to", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_from_name": { 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_from_name", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "Ceph", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Email From: name", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.176 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_host": { 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_host", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP server", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.177 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_password": { 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_password", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Password to authenticate with", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_port": { 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_port", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "465", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP port", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_sender": { 2024-05-10T04:43:08.179 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_sender", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SMTP envelope sender", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_ssl": { 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_ssl", 2024-05-10T04:43:08.180 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "smtp_user": { 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "smtp_user", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.181 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "User to authenticate as", 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.182 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "balancer", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "active": { 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "active", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically balance PGs across cluster", 2024-05-10T04:43:08.183 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_time": { 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_time", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0000", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "beginning time of day to automatically balance", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:43:08.184 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "begin_weekday": { 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "begin_weekday", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_max_iterations": { 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_max_iterations", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "25", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "250", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_metrics": { 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_metrics", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "pgs,objects,bytes", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.187 INFO:teuthology.orchestra.run.smithi150.stdout: "crush_compat_step": { 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crush_compat_step", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.5", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0.001", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "0.999", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "aggressiveness of optimization", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "end_time": { 2024-05-10T04:43:08.188 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_time", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2359", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ending time of day to automatically balance", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "end_weekday": { 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "end_weekday", 2024-05-10T04:43:08.189 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "0", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "6", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.190 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.191 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.193 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.194 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "min_score": { 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_score", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:08.195 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "upmap", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.196 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "crush-compat", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "none", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap" 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Balancer mode", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_ids": { 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_ids", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.198 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_deviation": { 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_deviation", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.199 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "upmap_max_optimizations": { 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "upmap_max_optimizations", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.200 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cephadm", 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_down_multiplier": { 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_down_multiplier", 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.201 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_refresh_rate": { 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_refresh_rate", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "20", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "agent_starting_port": { 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "agent_starting_port", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.203 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4721", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_ptrace": { 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_ptrace", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.204 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_interval": { 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_interval", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.205 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "autotune_memory_target_ratio": { 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "autotune_memory_target_ratio", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0.7", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.206 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "cgroups_split": { 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cgroups_split", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-10T04:43:08.207 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "config_checks_enabled": { 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_checks_enabled", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-10T04:43:08.208 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "config_dashboard": { 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_dashboard", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_alertmanager": { 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_alertmanager", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.210 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_base": { 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_base", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Container image name, without the tag", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_elasticsearch": { 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_elasticsearch", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "elasticsearch container image", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.212 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_grafana": { 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_grafana", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.213 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_haproxy": { 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_haproxy", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "HAproxy container image", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.214 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_agent": { 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_agent", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger agent container image", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_collector": { 2024-05-10T04:43:08.215 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_collector", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger collector container image", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_jaeger_query": { 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_jaeger_query", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.216 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Jaeger query container image", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_keepalived": { 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_keepalived", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.217 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Keepalived container image", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_loki": { 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_loki", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.218 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Loki container image", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_node_exporter": { 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_node_exporter", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.219 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_nvmeof": { 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_nvmeof", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.220 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Nvme-of container image", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_prometheus": { 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_prometheus", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.221 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Prometheus container image", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_promtail": { 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_promtail", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.222 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Promtail container image", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "container_image_snmp_gateway": { 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_image_snmp_gateway", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-10T04:43:08.223 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "SNMP Gateway container image", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "container_init": { 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "container_init", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.224 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Run podman/docker with `--init`", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "daemon_cache_timeout": { 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "daemon_cache_timeout", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.225 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "default_cephadm_command_timeout": { 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_cephadm_command_timeout", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "900", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-10T04:43:08.226 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "default_registry": { 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "default_registry", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "docker.io", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-10T04:43:08.227 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "device_cache_timeout": { 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_cache_timeout", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1800", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache device inventory", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "device_enhanced_scan": { 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_enhanced_scan", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.229 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "facts_cache_timeout": { 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "facts_cache_timeout", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "seconds to cache host facts data", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.230 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "host_check_interval": { 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "host_check_interval", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to perform a host check", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "hw_monitoring": { 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hw_monitoring", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.232 INFO:teuthology.orchestra.run.smithi150.stdout: "inventory_list_all": { 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "inventory_list_all", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.233 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.234 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "log_refresh_metadata": { 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_refresh_metadata", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.235 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.236 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.237 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.238 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.239 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.240 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "*", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "max_count_per_host": { 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_count_per_host", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.241 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of daemons per service per host", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "max_osd_draining_count": { 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_osd_draining_count", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.242 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "migration_current": { 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "migration_current", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.243 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "internal - do not modify", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "mode": { 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mode", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "root", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.244 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm-package", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "root" 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "mode for remote execution of cephadm", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "oob_default_addr": { 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "oob_default_addr", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "169.254.1.1", 2024-05-10T04:43:08.245 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "prometheus_alerts_path": { 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus_alerts_path", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-10T04:43:08.246 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_insecure": { 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_insecure", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.247 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_password": { 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_password", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.248 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_url": { 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_url", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.249 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "registry_username": { 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "registry_username", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.250 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "secure_monitoring_stack": { 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "secure_monitoring_stack", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.251 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "service_discovery_port": { 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "service_discovery_port", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8765", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "cephadm service discovery port", 2024-05-10T04:43:08.252 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "ssh_config_file": { 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssh_config_file", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.253 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "use_agent": { 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_agent", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.254 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "use_repo_digest": { 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "use_repo_digest", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.255 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_failed_host_check": { 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_failed_host_check", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if the host check fails", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_daemons": { 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_daemons", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.257 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_on_stray_hosts": { 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_on_stray_hosts", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crash", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.259 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.261 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.262 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.263 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "retain_interval": { 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retain_interval", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "31536000", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.264 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_recent_interval": { 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_recent_interval", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1209600", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dashboard", 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.266 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_HOST": { 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.267 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.268 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_ENABLED": { 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_ENABLED", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.269 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.270 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "ENABLE_BROWSABLE_API": { 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.271 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.273 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.274 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-10T04:43:08.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.276 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.277 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.278 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_PASSWORD": { 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.279 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.280 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_URL": { 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_URL", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.281 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_API_USERNAME": { 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.282 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.283 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.284 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.285 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_HOST": { 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.287 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.288 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.290 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-10T04:43:08.291 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-10T04:43:08.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.293 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-10T04:43:08.294 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_ENABLED": { 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.295 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.298 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "45", 2024-05-10T04:43:08.299 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ACCESS_KEY": { 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.300 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "admin", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.301 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SECRET_KEY": { 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.302 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "RGW_API_SSL_VERIFY": { 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "UNSAFE_TLS_v1_2": { 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.304 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.305 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.306 INFO:teuthology.orchestra.run.smithi150.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "cross_origin_url": { 2024-05-10T04:43:08.307 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cross_origin_url", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "crt_file": { 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "crt_file", 2024-05-10T04:43:08.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "debug": { 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "debug", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.309 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Enable/disable debug options", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "jwt_token_ttl": { 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "jwt_token_ttl", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.310 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "28800", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.313 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.315 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.316 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "motd": { 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "motd", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.317 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "The message of the day", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect_resolve_ip_addr": { 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "redirect_resolve_ip_addr", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.318 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.319 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8080", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.321 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl_server_port": { 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl_server_port", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8443", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.322 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "redirect", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "redirect" 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.323 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "url_prefix": { 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url_prefix", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_monitoring": { 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_monitoring", 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "monitor device health metrics", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "mark_out_threshold": { 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mark_out_threshold", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "2419200", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "pool_name": { 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pool_name", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "device_health_metrics", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "retention_period": { 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "retention_period", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15552000", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long to retain device health metrics", 2024-05-10T04:43:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_frequency": { 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_frequency", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "self_heal": { 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "self_heal", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how frequently to wake up and check device health", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "warn_threshold": { 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "warn_threshold", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7257600", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "diskprediction_local", 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "predict_interval": { 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predict_interval", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "86400", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "predictor_model": { 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "predictor_model", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "prophetstor", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "600", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "influx", 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": false, 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "influxdb python module not found", 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "batch_size": { 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "batch_size", 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5000", 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "database": { 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "database", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "ceph", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "hostname": { 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "hostname", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server hostname", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "30", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "5", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "password": { 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "password", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "password of InfluxDB server user", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "port": { 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "port", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "8086", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "InfluxDB server port", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "ssl": { 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ssl", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "false", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "threads": { 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threads", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "32", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "username": { 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "username", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "username of InfluxDB server user", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "verify_ssl": { 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "verify_ssl", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "true", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "insights", 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "iostat", 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "k8sevents", 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_event_retention_days": { 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "ceph_event_retention_days", 2024-05-10T04:43:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "7", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "config_check_secs": { 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "config_check_secs", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "10", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "localpool", 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "failure_domain": { 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "failure_domain", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "host", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "failure domain for any created local pool", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "min_size": { 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "min_size", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default min_size for any created local pool", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "num_rep": { 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "num_rep", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3", 2024-05-10T04:43:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default replica count for any created local pool", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_num": { 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_num", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "128", 2024-05-10T04:43:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "default pg_num for any created local pool", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "prefix": { 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prefix", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "name prefix for any created local pool", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "subtree": { 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "subtree", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "rack", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mds_autoscaler", 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirroring", 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "nfs", 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:43:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "fail_fs": { 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "fail_fs", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator": { 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "orchestrator", 2024-05-10T04:43:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "cephadm", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "rook", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "test_orchestrator" 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Orchestrator backend", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_perf_query", 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "osd_support", 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "pg_autoscaler", 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.425 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.426 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.427 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.428 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.429 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.430 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "threshold": { 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "threshold", 2024-05-10T04:43:08.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "3.0", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1.0", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "scaling threshold", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "progress", 2024-05-10T04:43:08.432 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_pg_recovery_event": { 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_pg_recovery_event", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.433 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.434 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.435 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.436 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.437 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.438 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.439 INFO:teuthology.orchestra.run.smithi150.stdout: "max_completed_events": { 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_completed_events", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "50", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "number of past completed events to remember", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "sleep_interval": { 2024-05-10T04:43:08.440 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "sleep_interval", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "5", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "how long the module is going to sleep", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "prometheus", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "cache": { 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "cache", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.442 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "exclude_perf_counters": { 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "exclude_perf_counters", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.443 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.444 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.446 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.448 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools": { 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "300", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "scrape_interval": { 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "scrape_interval", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15.0", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "::", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.452 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "9283", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "stale_cache_strategy": { 2024-05-10T04:43:08.453 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stale_cache_strategy", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "log", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_behaviour": { 2024-05-10T04:43:08.454 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_behaviour", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "default", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "default", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "error" 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.455 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "standby_error_status_code": { 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "standby_error_status_code", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "500", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "400", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "599", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.457 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.459 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.460 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.461 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_snap_create": { 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_snap_create", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.462 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "mirror_snapshot_schedule": { 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "mirror_snapshot_schedule", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.463 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "trash_purge_schedule": { 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "trash_purge_schedule", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.464 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "restful", 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "enable_auth": { 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enable_auth", 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.465 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "key_file": { 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "key_file", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.466 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.467 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.468 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.469 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.470 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.471 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "server_addr": { 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_addr", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.472 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "server_port": { 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "server_port", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.473 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rgw", 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.474 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.475 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.476 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.477 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.478 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.479 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rook", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.480 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.482 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "storage_class": { 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "storage_class", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "local", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "selftest", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.486 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.487 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.488 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.489 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.490 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "roption1": { 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption1", 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.491 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "roption2": { 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "roption2", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "xyz", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.492 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption1": { 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption1", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.493 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption2": { 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption2", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.494 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption3": { 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption3", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "float", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption4": { 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption4", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption5": { 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption5", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.497 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption6": { 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption6", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.498 INFO:teuthology.orchestra.run.smithi150.stdout: "rwoption7": { 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rwoption7", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "1", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "42", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.499 INFO:teuthology.orchestra.run.smithi150.stdout: "testkey": { 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testkey", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "testlkey": { 2024-05-10T04:43:08.500 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testlkey", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "testnewline": { 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "testnewline", 2024-05-10T04:43:08.501 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.502 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snap_schedule", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "allow_m_granularity": { 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "allow_m_granularity", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "allow minute scheduled snapshots", 2024-05-10T04:43:08.503 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "dump_on_update": { 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "dump_on_update", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "dump database to debug log on update", 2024-05-10T04:43:08.504 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.505 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.507 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.509 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "stats", 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.510 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.511 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.513 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.514 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "status", 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.515 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.516 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.517 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telegraf", 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "address": { 2024-05-10T04:43:08.520 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "address", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.521 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "15", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.522 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.523 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.524 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.525 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.526 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "telemetry", 2024-05-10T04:43:08.527 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_basic": { 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_basic", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.528 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_crash": { 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_crash", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.529 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_device": { 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_device", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_ident": { 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_ident", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.531 INFO:teuthology.orchestra.run.smithi150.stdout: "channel_perf": { 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "channel_perf", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "contact": { 2024-05-10T04:43:08.532 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "contact", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "description": { 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "description", 2024-05-10T04:43:08.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "device_url": { 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "device_url", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.534 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "enabled": { 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "enabled", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.536 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "24", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "8", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "last_opt_revision": { 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "last_opt_revision", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.537 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "1", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard": { 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.538 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "leaderboard_description": { 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "leaderboard_description", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.539 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.540 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.541 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.542 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.543 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.544 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "organization": { 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "organization", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.545 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "proxy": { 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "proxy", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.546 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "url": { 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "url", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "test_orchestrator", 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.548 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.550 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.551 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.552 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.554 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.555 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.557 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.557 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.558 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "max_concurrent_clones": { 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "max_concurrent_clones", 2024-05-10T04:43:08.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "4", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_delay": { 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_delay", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.561 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "0", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "snapshot_clone_no_wait": { 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "snapshot_clone_no_wait", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.562 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "True", 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix", 2024-05-10T04:43:08.563 INFO:teuthology.orchestra.run.smithi150.stdout: "can_run": true, 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "error_string": "", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "module_options": { 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "discovery_interval": { 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "discovery_interval", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "uint", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "100", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.564 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "identifier": { 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "identifier", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "interval": { 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "interval", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "secs", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "60", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "log_level": { 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_level", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.567 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster": { 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.568 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_cluster_level": { 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_cluster_level", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "info", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.569 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [ 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "", 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "critical", 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "debug", 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "error", 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "info", 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "warning" 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "log_to_file": { 2024-05-10T04:43:08.570 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "log_to_file", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "bool", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 1, 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "False", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_host": { 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_host", 2024-05-10T04:43:08.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_port": { 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_port", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "int", 2024-05-10T04:43:08.572 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "10051", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "zabbix_sender": { 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "zabbix_sender", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "str", 2024-05-10T04:43:08.573 INFO:teuthology.orchestra.run.smithi150.stdout: "level": "advanced", 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flags": 0, 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "min": "", 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "max": "", 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "enum_allowed": [], 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "desc": "", 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "long_desc": "", 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "tags": [], 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "see_also": [] 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.574 INFO:teuthology.orchestra.run.smithi150.stdout: "services": { 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "dashboard": "https://172.21.15.150:8443/" 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "always_on_modules": { 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "octopus": [ 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:43:08.575 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "pacific": [ 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.576 INFO:teuthology.orchestra.run.smithi150.stdout: "quincy": [ 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "reef": [ 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "balancer", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "crash", 2024-05-10T04:43:08.577 INFO:teuthology.orchestra.run.smithi150.stdout: "devicehealth", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "orchestrator", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "pg_autoscaler", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "progress", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "rbd_support", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "status", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "telemetry", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "volumes" 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "last_failure_osd_epoch": 3, 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "active_clients": [ 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "devicehealth", 2024-05-10T04:43:08.578 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 2510422392 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "libcephsqlite", 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:43:08.579 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 2863686883 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "rbd_support", 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 2036243910 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.580 INFO:teuthology.orchestra.run.smithi150.stdout: "name": "volumes", 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: "addrvec": [ 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: "type": "v2", 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: "addr": "172.21.15.150:0", 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: "nonce": 3461201232 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-10T04:43:08.581 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:07 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/982863059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:07 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1027158921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:07 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1949205764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:07 smithi181 ceph-mon[47314]: pgmap v378: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:07 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/982863059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:07 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1027158921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:07 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1949205764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T04:43:08.583 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:07 smithi161 ceph-mon[46912]: pgmap v378: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.263+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3fe80a1120 msgr2=0x7f3fe80a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3fe80a1120 0x7f3fe80a3630 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005b40 tx=0x7f3fe4005a90 comp rx=0 tx=0).stop 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc100ac0 msgr2=0x7f3ffc196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc100ac0 0x7f3ffc196f80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098f0 tx=0x7f3fec009cb0 comp rx=0 tx=0).stop 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 shutdown_connections 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f3fe80a1120 0x7f3fe80a3630 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ffc102580 0x7f3ffc19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ffc100ac0 0x7f3ffc196f80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 --2- 172.21.15.150:0/1949205764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ffc101be0 0x7f3ffc1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 >> 172.21.15.150:0/1949205764 conn(0x7f3ffc0fc0d0 msgr2=0x7f3ffc104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:08.584 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 shutdown_connections 2024-05-10T04:43:08.585 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:07.264+0000 7f4001a93700 1 -- 172.21.15.150:0/1949205764 wait complete. 2024-05-10T04:43:08.590 DEBUG:tasks.check_counter:Getting stats from a 2024-05-10T04:43:08.591 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-10T04:43:08.591 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:07 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/982863059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:43:08.591 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:07 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1027158921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:43:08.591 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:07 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1949205764' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-10T04:43:08.591 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:07 smithi150 ceph-mon[42404]: pgmap v378: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:08.829 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:43:09.620 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:09 smithi150 ceph-mon[42404]: pgmap v379: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T04:43:09.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:09 smithi161 ceph-mon[46912]: pgmap v379: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T04:43:09.855 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:43:09.855 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:09.855 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 104, 2024-05-10T04:43:09.855 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 98, 2024-05-10T04:43:09.855 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 462322, 2024-05-10T04:43:09.855 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 50573, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.017570133, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005904254, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.011194841, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001698233, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 98, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.009136917, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000093233 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.856 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 461488, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 50256 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 215, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 279, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 52826, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 249359, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.024879671, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.009796278, 2024-05-10T04:43:09.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.006052739, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.006783934, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 277, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.009504150, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000034311 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 182800 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 552, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 554, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 165789, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 171730, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.050659653, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.019494641, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.012391714, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.014318726, 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:09.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 552, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.021796981, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000039487 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:43:09.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1312, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.033868056, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000025814 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-05-10T04:43:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-05-10T04:43:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:09.865 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:09.866 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-05-10T04:43:09.867 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 2, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 1, 2024-05-10T04:43:09.868 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4195158, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4194844, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 2 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 3, 2024-05-10T04:43:09.869 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 114388, 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 207252 2024-05-10T04:43:09.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.871 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.873 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.874 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.875 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.876 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.877 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.878 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.879 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.880 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 1641, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:43:09.882 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:09.883 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:09.884 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 189256, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 49, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:09.885 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 176, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 8320, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 13, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:09.886 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 767, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 767, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 767, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.377992780, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000492819 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-05-10T04:43:09.887 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 767, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 768 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 767, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 767, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 486, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 279, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-05-10T04:43:09.888 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:09.889 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-05-10T04:43:09.890 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 90, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 4, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 5, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 5, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-05-10T04:43:09.891 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:09.892 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 867, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 594790, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 867, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:09.893 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 867, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 594790, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:09.894 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 767, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 859, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 767, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 859, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 767, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 767, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 767, 2024-05-10T04:43:09.896 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 767, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:09.897 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x5654e46de4e0": { 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:09.899 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:09.900 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:09.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:09.900 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:43:09.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:09 smithi181 ceph-mon[47314]: pgmap v379: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-10T04:43:10.248 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-10T04:43:10.248 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-10T04:43:10.249 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-10T04:43:10.249 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-10T04:43:10.249 DEBUG:tasks.check_counter:Getting stats from d 2024-05-10T04:43:10.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-10T04:43:10.475 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:43:11.454 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001444255, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000391819, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:43:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:11.456 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 5, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 70, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3200, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 88592, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.006386336, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004015477, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001688702, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 69, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.003824920, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000055433 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.457 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 88592 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 92, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 89, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 435648, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 50048, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:43:11.458 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.016287373, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005709328, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.011395524, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001727700, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 88, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.004887197, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000055536 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:11.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 435648, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 50048 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:11.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:11.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:43:11.462 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148840, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 28, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-10T04:43:11.463 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:43:11.464 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 93, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 422666, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 93, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 93, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 422666, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:11.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:11.466 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:43:11.475 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:11 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:43:11.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:11 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:43:11.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:11 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-10T04:43:11.850 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-10T04:43:11.851 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-10T04:43:11.851 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-10T04:43:11.851 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-10T04:43:11.851 DEBUG:tasks.check_counter:Getting stats from g 2024-05-10T04:43:11.851 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-10T04:43:12.083 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:43:12.518 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:12 smithi150 ceph-mon[42404]: pgmap v380: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:12.519 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.519 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.519 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:43:12.519 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:43:12.519 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:12 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:12 smithi161 ceph-mon[46912]: pgmap v380: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:12.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:43:12.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:43:12.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:12 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.668 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:12 smithi181 ceph-mon[47314]: pgmap v380: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:12.668 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.669 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:12.669 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-10T04:43:12.669 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-10T04:43:12.669 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:12 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' 2024-05-10T04:43:13.078 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:43:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:13.078 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 377, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 373, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 551792, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 136531, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.047882382, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.016189181, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.017891020, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.011928758, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 372, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.018796506, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000050528 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:13.079 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 458720, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 50384 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 8, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 71, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 4590, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 184425, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.006677613, 2024-05-10T04:43:13.080 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.003633949, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.002360301, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000067444, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 70, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.006176026, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000088228 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 183424 2024-05-10T04:43:13.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 489, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 490, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 126351, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 151769, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.036413191, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.009792439, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.009930126, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.013820753, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 488, 2024-05-10T04:43:13.082 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.018048121, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000036983 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-05-10T04:43:13.083 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1322, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.038299858, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000028971 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.084 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-05-10T04:43:13.085 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:13.086 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:13.087 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:13.088 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:13.089 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 4, 2024-05-10T04:43:13.090 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 1, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4195692, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4195551, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 4 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 3, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-05-10T04:43:13.092 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 114296, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 207252 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-05-10T04:43:13.093 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.094 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.095 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.096 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.097 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.098 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.099 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.100 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-05-10T04:43:13.102 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.103 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 2, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 1, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 2, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 3016, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:13.104 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:13.105 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:13.106 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 197336, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 53, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:13.107 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 176, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 8320, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 13, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-05-10T04:43:13.108 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 768, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 768, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 768, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.859983552, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001119770 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 768, 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 769 2024-05-10T04:43:13.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 768, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 768, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 483, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 283, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-05-10T04:43:13.110 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-05-10T04:43:13.111 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-05-10T04:43:13.112 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 90, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 4, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 6, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 6, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:13.113 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.114 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 870, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 596264, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 870, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 870, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 596264, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.115 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 768, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 1393, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 768, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1393, 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:13.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 768, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 768, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 768, 2024-05-10T04:43:13.117 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 768, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:13.118 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x5555cd4488f0": { 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-05-10T04:43:13.119 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:13.120 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:43:13.474 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-10T04:43:13.474 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-10T04:43:13.474 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-10T04:43:13.475 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-10T04:43:13.475 DEBUG:tasks.check_counter:Getting stats from j 2024-05-10T04:43:13.475 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-10T04:43:13.661 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:13 smithi150 ceph-mon[42404]: pgmap v381: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:13.704 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.a/config 2024-05-10T04:43:13.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:13 smithi161 ceph-mon[46912]: pgmap v381: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:13.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:13 smithi181 ceph-mon[47314]: pgmap v381: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:14.686 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001210072, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000288382, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:14.687 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 5, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 71, 2024-05-10T04:43:14.688 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3200, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 88768, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.006836910, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.003848576, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.002242177, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 70, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.004039673, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000057709 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:14.689 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 88768 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 93, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 90, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 435920, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 50640, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.016261589, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.005369683, 2024-05-10T04:43:14.690 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.012065718, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001740483, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 89, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.003850139, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000043259 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 435920, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 50640 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.691 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:14.692 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:14.693 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148840, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 28, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:14.694 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-10T04:43:14.695 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 94, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 422796, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 94, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 94, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 422796, 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-10T04:43:14.696 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-10T04:43:14.697 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-10T04:43:14.697 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-10T04:43:14.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:14.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-10T04:43:14.697 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-10T04:43:15.089 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-10T04:43:15.089 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-10T04:43:15.089 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-10T04:43:15.089 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-10T04:43:15.089 DEBUG:tasks.check_counter:Getting stats from b 2024-05-10T04:43:15.089 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-10T04:43:15.338 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:43:15.764 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:15 smithi161 ceph-mon[46912]: pgmap v382: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:15.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:15 smithi181 ceph-mon[47314]: pgmap v382: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:15.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:15 smithi150 ceph-mon[42404]: pgmap v382: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:16.350 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:43:16.354 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:16.354 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 923, 2024-05-10T04:43:16.354 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 949, 2024-05-10T04:43:16.354 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 269660, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 980166, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.092481664, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.026122857, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.030380368, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.025058362, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 947, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.177947184, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000187906 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 860, 2024-05-10T04:43:16.355 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000265199, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000308 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1080, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 776, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 790812, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 1006295, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 84, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.135467183, 2024-05-10T04:43:16.356 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.029276107, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.079112032, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.030667407, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 771, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.425564238, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000551963 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 950, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000210698, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000221 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 583120, 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 99056 2024-05-10T04:43:16.357 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 810, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 896, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 262377, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 539716, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 14, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.080351233, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.027088590, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.032471139, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.018303204, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 899, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.538149450, 2024-05-10T04:43:16.358 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001710956 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 959, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000237459, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000247 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 194992 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-10T04:43:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 498, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.138899603, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000278914 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 135, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.005254915, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000038925 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.360 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 165, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 165, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 165, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.124564050, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000754933 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 60, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-10T04:43:16.361 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 25, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 5, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 20, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 17, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 1, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 1, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 2, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 301, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 289, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:16.362 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 77, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 77, 2024-05-10T04:43:16.363 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 49, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 6, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 43, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 28, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 20, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 36, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 104, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 79, 2024-05-10T04:43:16.364 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 1, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 29 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 14, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 29, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 1, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 6, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 5, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:16.365 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 6, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-10T04:43:16.366 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 172, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 143, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 143, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 29, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 18, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 13, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 12, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 12, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 1, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4732638, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4823949, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-10T04:43:16.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 145, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.306079068, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002110890 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 24, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 35, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 11, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 19, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 20, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 1, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 25, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 35, 2024-05-10T04:43:16.368 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 10, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 1, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 19, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 18, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 119760, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207252 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 196, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 165, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 15, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-05-10T04:43:16.369 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.006669123, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000833640 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 125, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.035146712, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000281173 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-10T04:43:16.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000627988, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000313994 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-10T04:43:16.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:16.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 8, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.020128185, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002516023 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-10T04:43:16.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000278441, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000278441 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.374 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.005681864, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002840932 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002694598, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001347299 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.375 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.017661463, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.008830731 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.376 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001516031, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000758015 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.377 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.025404080, 2024-05-10T04:43:16.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002822675 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.008755565, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002188891 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.379 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 26, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 26, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 215, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.380 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:16.381 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:16.382 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 291311, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 126, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 616, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 7, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:43:16.383 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 176, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 100314, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 824, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:16.384 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 1060, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 706037, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 37, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 1060, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1060, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 19.764580678, 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.018645830 2024-05-10T04:43:16.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1060, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1230 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 1060, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 90, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 970, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 40, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 36, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 7, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 146, 2024-05-10T04:43:16.386 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 49, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 649, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 83, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 64, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:16.387 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-10T04:43:16.388 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 90, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 9, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-10T04:43:16.389 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 146, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 10 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-10T04:43:16.390 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 137, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 5, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 5, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2809, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1050469, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2809, 2024-05-10T04:43:16.391 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2809, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1050469, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:16.392 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1060, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 706574, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1060, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 706574, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-10T04:43:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 1060, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 1060, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1060, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1060, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 5, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 537, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 5, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 537, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55e60b26c340": { 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 172, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 629645, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 172, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 145, 2024-05-10T04:43:16.396 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 629645, 2024-05-10T04:43:16.397 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:16.397 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:16.397 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:16.397 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:16.397 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.397 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:16.397 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:43:16.710 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-10T04:43:16.710 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-10T04:43:16.710 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=14 2024-05-10T04:43:16.710 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-10T04:43:16.710 DEBUG:tasks.check_counter:Getting stats from e 2024-05-10T04:43:16.710 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-10T04:43:16.936 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:43:17.732 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:17 smithi161 ceph-mon[46912]: pgmap v383: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:17.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:17 smithi181 ceph-mon[47314]: pgmap v383: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:17.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:17 smithi150 ceph-mon[42404]: pgmap v383: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 283, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 285, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 95033, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 88014, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.037096123, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.010515933, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.009788200, 2024-05-10T04:43:17.950 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.011455160, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 283, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.019591177, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000069226 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:17.951 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 391, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 385, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 537556, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 141005, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.044090264, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.014171451, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.019771095, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.009935290, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 385, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.016594623, 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000043102 2024-05-10T04:43:17.952 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 462576, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 52624 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 72, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1508, 2024-05-10T04:43:17.953 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 187584, 2024-05-10T04:43:17.954 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-05-10T04:43:17.954 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:17.954 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.005425741, 2024-05-10T04:43:17.954 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.002717754, 2024-05-10T04:43:17.954 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001788068, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000070670, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 70, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.005825386, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000083219 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.955 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 186640 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1132, 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.033787819, 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000029847 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.956 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.957 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 0, 2024-05-10T04:43:17.958 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-10T04:43:17.959 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:17.960 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.961 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:17.962 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 1, 2024-05-10T04:43:17.963 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 1, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4194844, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4194304, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.964 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 1 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 3, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 3, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 0, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 1, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 1, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 0, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 0, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 0, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 0, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-05-10T04:43:17.965 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 116172, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207252 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.966 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-10T04:43:17.967 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-10T04:43:17.968 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.969 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-10T04:43:17.970 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-10T04:43:17.971 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-10T04:43:17.972 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-10T04:43:17.973 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.974 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.975 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 0, 2024-05-10T04:43:17.976 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:17.977 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:17.978 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:17.979 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 198350, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 53, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 176, 2024-05-10T04:43:17.980 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 8320, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 13, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 570, 2024-05-10T04:43:17.981 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 570, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 570, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.281994044, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000494726 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 570, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 571 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 570, 2024-05-10T04:43:17.982 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 570, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 282, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 286, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-10T04:43:17.983 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-10T04:43:17.984 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 90, 2024-05-10T04:43:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 6, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 6, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-10T04:43:17.986 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 674, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 565112, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 674, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 674, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 565112, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 570, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 545, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 570, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 545, 2024-05-10T04:43:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:17.990 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 570, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 570, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 570, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 570, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:17.991 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:17.992 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55fc022d44e0": { 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:17.993 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:17.994 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.994 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:17.994 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:43:18.319 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-10T04:43:18.319 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-10T04:43:18.319 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-10T04:43:18.320 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-10T04:43:18.320 DEBUG:tasks.check_counter:Getting stats from h 2024-05-10T04:43:18.320 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-10T04:43:18.543 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:43:19.557 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:43:19.557 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 661, 2024-05-10T04:43:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 663, 2024-05-10T04:43:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 823294, 2024-05-10T04:43:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 205581, 2024-05-10T04:43:19.558 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-05-10T04:43:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-05-10T04:43:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.062940433, 2024-05-10T04:43:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.022269566, 2024-05-10T04:43:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.015878437, 2024-05-10T04:43:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.017813502, 2024-05-10T04:43:19.559 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 661, 2024-05-10T04:43:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.024800793, 2024-05-10T04:43:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000037520 2024-05-10T04:43:19.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.561 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 118, 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 105, 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 490472, 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 53486, 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:19.562 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.019856595, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.005898226, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.013649671, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002478968, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 105, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.009973289, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000094983 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:19.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 488816, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 52176 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 289, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 356, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 71079, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 276033, 2024-05-10T04:43:19.564 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.030644514, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.009451598, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.007580303, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.010336361, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 354, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012928488, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000036521 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 186464 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1604, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.042806730, 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000026687 2024-05-10T04:43:19.566 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-10T04:43:19.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-10T04:43:19.568 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 21, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 4, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 17, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 14, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 2, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:19.569 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-10T04:43:19.570 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:19.571 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:19.572 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-05-10T04:43:19.573 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:19.574 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-10T04:43:19.575 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 172, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4732638, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4823949, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4822811, 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-10T04:43:19.576 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 172 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 15, 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 20, 2024-05-10T04:43:19.577 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 5, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 10, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 11, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 1, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 21, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 25, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 4, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 116152, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207252 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-10T04:43:19.578 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.580 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-10T04:43:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.581 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:19.582 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-10T04:43:19.583 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-10T04:43:19.584 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:19.585 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:19.586 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.587 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.589 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3020, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.590 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:19.591 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:19.592 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:19.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 189290, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 50, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 176, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:19.594 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 60610, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 370, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 953, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 26, 2024-05-10T04:43:19.595 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 4, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 953, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 953, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.371992119, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000390338 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 953, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 954 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 953, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 953, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-10T04:43:19.596 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 574, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 376, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 2, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-05-10T04:43:19.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-10T04:43:19.598 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-10T04:43:19.599 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 90, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 5, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.600 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:19.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 1064, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1280449, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 1064, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:19.602 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 1064, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1280449, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:19.603 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 953, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 629655, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 953, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 629655, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:19.604 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 953, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 953, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 953, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 953, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.605 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:19.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x561655bbe4e0": { 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:19.607 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:19.608 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:43:19.763 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:19 smithi161 ceph-mon[46912]: pgmap v384: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:43:19.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:19 smithi181 ceph-mon[47314]: pgmap v384: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:43:19.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:19 smithi150 ceph-mon[42404]: pgmap v384: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:43:19.957 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-10T04:43:19.958 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-10T04:43:19.958 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-10T04:43:19.958 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-10T04:43:19.958 DEBUG:tasks.check_counter:Getting stats from k 2024-05-10T04:43:19.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-10T04:43:20.180 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.b/config 2024-05-10T04:43:21.190 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-05-10T04:43:21.190 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:21.190 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 261, 2024-05-10T04:43:21.190 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 247, 2024-05-10T04:43:21.190 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 89293, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 118344, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.032206890, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.012770516, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.015898949, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002060511, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 246, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.030538915, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000124141 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:21.191 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 447, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000110213, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000246 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 162, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 154, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 487054, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 105576, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:21.192 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.026767888, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.007953160, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.019134197, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002667248, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 153, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.016730746, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000109351 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 11, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000002814, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000255 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.193 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 481920, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 58736 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 24, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 94, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4590, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 313592, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 18, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.011866278, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.003389435, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.007139164, 2024-05-10T04:43:21.194 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000184285, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 92, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012758207, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000138676 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 21, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000003729, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000177 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 189456 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-05-10T04:43:21.195 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 47, 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001629139, 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000034662 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-05-10T04:43:21.196 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000018214, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000009107 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 1, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 1, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000144342, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000144342 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-05-10T04:43:21.197 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 16, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 16, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 13, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 4, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 1, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 1, 2024-05-10T04:43:21.198 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 7, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 4, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 3, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:21.199 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:21.200 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.201 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 11, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 7, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 8, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:21.202 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 4, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-05-10T04:43:21.203 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 4, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 1, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 1, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4195692, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-05-10T04:43:21.204 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.007607155, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001901788 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 20, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 22, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 2, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 19, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 22, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 3, 2024-05-10T04:43:21.205 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 16, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 20, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 4, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 1, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 1, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 115220, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 199060 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 1, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 1, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 8, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-05-10T04:43:21.206 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.207 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.208 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000144342, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000144342 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:21.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.210 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-05-10T04:43:21.211 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.213 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.214 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:21.214 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.214 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.214 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.214 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.215 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.216 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:21.217 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.218 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.219 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.220 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.221 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-05-10T04:43:21.222 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 1, 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 213, 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:21.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:21.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-05-10T04:43:21.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 245863, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 73, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:21.226 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 176, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 80013, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 324, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-05-10T04:43:21.227 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 41, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 11901, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 33, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 41, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 41, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.229995167, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005609638 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 41, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 64 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.228 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 41, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 0, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 41, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 0, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 4, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 7, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-05-10T04:43:21.229 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 6, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-05-10T04:43:21.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-05-10T04:43:21.231 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 90, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 8, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 8, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 16, 2024-05-10T04:43:21.232 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 1 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:21.233 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 443, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 530816, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 443, 2024-05-10T04:43:21.234 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 443, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 530816, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:21.235 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 41, 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 11901, 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 41, 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 11901, 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.236 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:21.237 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 41, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 41, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 41, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 41, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.238 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.239 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x563ce74581a0": { 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1388, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-05-10T04:43:21.240 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4, 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1388, 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-05-10T04:43:21.241 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-05-10T04:43:21.568 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-10T04:43:21.568 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-10T04:43:21.568 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-10T04:43:21.568 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=11 2024-05-10T04:43:21.568 DEBUG:tasks.check_counter:Getting stats from c 2024-05-10T04:43:21.569 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-10T04:43:21.816 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:43:21.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:21 smithi161 ceph-mon[46912]: pgmap v385: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:21.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:21 smithi181 ceph-mon[47314]: pgmap v385: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:21.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:21 smithi150 ceph-mon[42404]: pgmap v385: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:23.005 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 401, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 390, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 583600, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 142958, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 1, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.058670818, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.017187058, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.022487992, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.015840942, 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:23.006 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 389, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.025845199, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000066440 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 486160, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 52768 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:23.007 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 71, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 716, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 188892, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.005021037, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.002728906, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.001364860, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000077025, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 69, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.004912448, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000071194 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.008 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 188208 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 298, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 298, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 81189, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 91889, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-05-10T04:43:23.009 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.029366825, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.010885452, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.007890991, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.007447323, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 297, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.013732017, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000046235 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.010 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1165, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.038837373, 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000033336 2024-05-10T04:43:23.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.012 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 0, 2024-05-10T04:43:23.013 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 1, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:43:23.014 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:23.015 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:43:23.016 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:43:23.017 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 1, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:43:23.018 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4194844, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 4194304, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.019 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 1 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 3, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 3, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 0, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 1, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 1, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 0, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 0, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 0, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 0, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-05-10T04:43:23.020 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 117588, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 207252 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.021 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.022 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.023 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.024 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.027 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.028 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.029 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.030 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.031 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 0, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:43:23.032 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:23.033 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:23.034 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 181239, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 46, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:23.035 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 176, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 8320, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 13, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.036 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 586, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 21, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 3, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 586, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 586, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.300994315, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000513642 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 586, 2024-05-10T04:43:23.037 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 587 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 586, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 586, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 290, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 294, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:43:23.038 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 1, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:43:23.039 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:43:23.040 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 90, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 4, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 4, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 4, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-05-10T04:43:23.041 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:43:23.042 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 698, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 593991, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 698, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 698, 2024-05-10T04:43:23.043 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 593991, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:23.044 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 586, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 545, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 586, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 545, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:23.045 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 586, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 586, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 586, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 586, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:23.046 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.047 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55c9849cc8f0": { 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:23.048 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:23.049 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:23.049 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:23.049 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.049 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:23.049 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:43:23.436 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-10T04:43:23.437 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-10T04:43:23.437 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-10T04:43:23.437 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-10T04:43:23.437 DEBUG:tasks.check_counter:Getting stats from f 2024-05-10T04:43:23.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-10T04:43:23.654 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:23 smithi181 ceph-mon[47314]: pgmap v386: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:23.662 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:43:23.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:23 smithi161 ceph-mon[46912]: pgmap v386: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:23.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:23 smithi150 ceph-mon[42404]: pgmap v386: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-05-10T04:43:24.784 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:43:24.784 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:24.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 31, 2024-05-10T04:43:24.784 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 31, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 9529, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 49635, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 12, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.009481929, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.001453371, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.005165214, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000403901, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 30, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.012045016, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000401500 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.785 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 13, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000003449, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000265 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 448 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 167, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 157, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 493924, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 185798, 2024-05-10T04:43:24.786 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 17, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.026953898, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.007063027, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.020050575, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002985245, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 156, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.021450252, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000137501 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 12, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000001514, 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000126 2024-05-10T04:43:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 488400, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 58160 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 243, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 300, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 78453, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 284291, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.032151013, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.013264811, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.016257941, 2024-05-10T04:43:24.788 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.002314320, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 298, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.022289367, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000074796 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 453, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000140612, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000310 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 189952 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.789 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 48, 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001649614, 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000034366 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:24.790 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000015240, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000007620 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 2, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 1, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000182634, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000182634 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.791 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 1, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 16, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 16, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 13, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 4, 2024-05-10T04:43:24.792 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 1, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 1, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 3, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 8, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 4, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 1, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 3, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:43:24.793 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:24.794 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:43:24.795 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 11, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 7, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 8, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:24.796 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 2, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 2, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:43:24.797 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 1, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4195158, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.003912968, 2024-05-10T04:43:24.798 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001956484 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 20, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 22, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 19, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 22, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 3, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 16, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 20, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 4, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 1, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 1, 2024-05-10T04:43:24.799 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 116844, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 199060 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 2, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 2, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 3, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.800 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:43:24.802 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000182634, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000182634 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:24.803 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.804 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.805 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.806 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:24.810 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 1, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 1, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 1, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 1, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 1, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 224, 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:24.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:24.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:24.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 238475, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 78, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:43:24.814 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 176, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 80037, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 327, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:43:24.815 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 37, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 11066, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 37, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 37, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.225995622, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006107989 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.816 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 37, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 59 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 37, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 0, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 37, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 0, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 2, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 6, 2024-05-10T04:43:24.817 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 6, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:24.818 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:43:24.819 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 90, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 4, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 8, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 8, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:43:24.820 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 14, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 1 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.821 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 437, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 532497, 2024-05-10T04:43:24.822 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 437, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 437, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 532497, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:24.823 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 37, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 11066, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 37, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 11066, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.824 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 37, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 37, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 37, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 37, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.825 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:24.826 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x5598d78a04e0": { 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 2, 2024-05-10T04:43:24.827 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 854, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 2, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 854, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:24.828 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:43:25.162 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-10T04:43:25.162 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-10T04:43:25.162 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-10T04:43:25.162 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=11 2024-05-10T04:43:25.163 DEBUG:tasks.check_counter:Getting stats from i 2024-05-10T04:43:25.163 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-10T04:43:25.383 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:43:25.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:25 smithi161 ceph-mon[46912]: pgmap v387: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:25.891 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:25 smithi181 ceph-mon[47314]: pgmap v387: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:25.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:25 smithi150 ceph-mon[42404]: pgmap v387: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:26.486 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:43:26.486 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:26.486 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 390, 2024-05-10T04:43:26.486 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 364, 2024-05-10T04:43:26.486 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 577639, 2024-05-10T04:43:26.486 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 116113, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.049678469, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.017121444, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.029096771, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.004656228, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 364, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.027688400, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000076067 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 454, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000131748, 2024-05-10T04:43:26.487 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000290 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 494832, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 57056 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 27, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 95, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 5262, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 349558, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 23, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.013150638, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.003336772, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.008200616, 2024-05-10T04:43:26.488 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000231673, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 93, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.021137756, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000227287 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 16, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000001950, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000121 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 189776 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 13, 2024-05-10T04:43:26.489 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 15, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 3625, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 11556, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.002793785, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000208623, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.001982329, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000479736, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 13, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.018527358, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001425181 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.490 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:26.491 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 35, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001866712, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000053334 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000019006, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000009503 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-05-10T04:43:26.492 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 16, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 16, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:26.493 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 13, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 4, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 3, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 7, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 4, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 3, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:43:26.494 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:26.495 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.496 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 11, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 7, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 8, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:26.497 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 1, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 1, 2024-05-10T04:43:26.498 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 1, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4194844, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001350396, 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001350396 2024-05-10T04:43:26.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 20, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 22, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 19, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 22, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 3, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 16, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 20, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 4, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-05-10T04:43:26.500 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 117600, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 199060 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:43:26.501 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:43:26.502 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.503 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.504 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:43:26.505 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:43:26.506 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.507 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.508 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.510 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:43:26.511 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:26.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:26.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:26.514 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 238183, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 77, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 176, 2024-05-10T04:43:26.515 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 79637, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 321, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 35, 2024-05-10T04:43:26.516 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 10662, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 27, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 35, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 35, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.233995479, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006685585 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 35, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 57 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 35, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 0, 2024-05-10T04:43:26.517 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 35, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 1, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 5, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 6, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:26.518 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:43:26.519 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 90, 2024-05-10T04:43:26.520 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 4, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 9, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 9, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 14, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 1 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.521 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:26.522 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 426, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 538339, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 426, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 426, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 538339, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.523 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 35, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 10662, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 35, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 10662, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.524 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 35, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 35, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 35, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 35, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.525 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:26.526 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x5644431fe270": { 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 540, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1, 2024-05-10T04:43:26.527 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1, 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 540, 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:26.528 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:43:26.861 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-10T04:43:26.861 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-10T04:43:26.861 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-10T04:43:26.861 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-05-10T04:43:26.861 DEBUG:tasks.check_counter:Getting stats from l 2024-05-10T04:43:26.862 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:50174015a7e79f951b988949abe236791f52d4da shell --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-10T04:43:27.084 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/mon.c/config 2024-05-10T04:43:27.835 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:27 smithi161 ceph-mon[46912]: pgmap v388: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:27.904 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:27 smithi181 ceph-mon[47314]: pgmap v388: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:27.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:27 smithi150 ceph-mon[42404]: pgmap v388: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-10T04:43:28.143 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-05-10T04:43:28.143 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-05-10T04:43:28.143 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 377, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 363, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 552681, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 111299, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.050500264, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.016928334, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.030803217, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.004674987, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 363, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.028953383, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000079761 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.144 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 432, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000164617, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000381 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 477664, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 60384 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 26, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 97, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 4975, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 353270, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 23, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2024-05-10T04:43:28.145 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.012518669, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.003278268, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.007710347, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000254536, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 95, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.017124652, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000180259 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 16, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000001833, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000114 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.146 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 188736 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 17, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 19, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 4573, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 7972, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.003598514, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000273507, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.002761489, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000433713, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 17, 2024-05-10T04:43:28.147 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.024991194, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001470070 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-05-10T04:43:28.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 31, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001415831, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000045671 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000018438, 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000009219 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 11, 2024-05-10T04:43:28.150 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 15, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 15, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 13, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 4, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 3, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 4, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 2, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 2, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-05-10T04:43:28.151 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-05-10T04:43:28.152 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-05-10T04:43:28.153 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 6, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 4, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 5, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-05-10T04:43:28.154 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 1, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 1, 2024-05-10T04:43:28.155 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 1, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 4194304, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 4194844, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001701045, 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001701045 2024-05-10T04:43:28.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.157 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-05-10T04:43:28.157 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.157 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-05-10T04:43:28.157 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 19, 2024-05-10T04:43:28.157 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 21, 2024-05-10T04:43:28.157 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 2, 2024-05-10T04:43:28.157 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 18, 2024-05-10T04:43:28.158 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 20, 2024-05-10T04:43:28.158 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 2, 2024-05-10T04:43:28.158 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 15, 2024-05-10T04:43:28.158 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 19, 2024-05-10T04:43:28.158 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 4, 2024-05-10T04:43:28.158 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 116844, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 199060 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-05-10T04:43:28.159 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-05-10T04:43:28.160 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-05-10T04:43:28.160 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-05-10T04:43:28.160 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.160 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.160 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.162 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.164 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.166 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.168 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-05-10T04:43:28.168 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.168 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.168 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.168 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.169 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.170 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-05-10T04:43:28.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 0, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 0, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 0, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 0, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-05-10T04:43:28.172 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-05-10T04:43:28.173 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-10T04:43:28.174 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-05-10T04:43:28.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 238183, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 77, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9176, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 176, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-05-10T04:43:28.176 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 75501, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 307, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 35, 2024-05-10T04:43:28.177 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 10662, 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 33, 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 35, 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 35, 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.238995393, 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006828439 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-05-10T04:43:28.178 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 35, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 57 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 35, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 0, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 35, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 0, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 10, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 0, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 1, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 5, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-05-10T04:43:28.179 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 6, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 20, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-05-10T04:43:28.180 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-05-10T04:43:28.181 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 90, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 4, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 9, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 9, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 14, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 1 2024-05-10T04:43:28.182 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.183 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 416, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 514736, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:28.184 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 416, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 416, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 514736, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:28.185 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 35, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 10662, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 35, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 10662, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.186 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 35, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 35, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 35, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 35, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.187 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-05-10T04:43:28.188 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55fbd17124e0": { 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 1, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 540, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 1, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-05-10T04:43:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1, 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 540, 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-05-10T04:43:28.190 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-05-10T04:43:28.529 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-10T04:43:28.529 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-10T04:43:28.529 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-10T04:43:28.529 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=6 2024-05-10T04:43:28.529 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-10T04:43:28.540 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-10T04:43:28.540 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:43:28.540 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:43:28.565 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-10T04:43:28.565 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-10T04:43:28.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:43:28.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 -- 172.21.15.150:0/2585928147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898069090 msgr2=0x7fd898106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:28.816 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 --2- 172.21.15.150:0/2585928147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898069090 0x7fd898106b20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009990 tx=0x7fd88c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 -- 172.21.15.150:0/2585928147 shutdown_connections 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 --2- 172.21.15.150:0/2585928147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd898109aa0 0x7fd89810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 --2- 172.21.15.150:0/2585928147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898069090 0x7fd898106b20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 --2- 172.21.15.150:0/2585928147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd898107060 0x7fd898109480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 -- 172.21.15.150:0/2585928147 >> 172.21.15.150:0/2585928147 conn(0x7fd8980fc0d0 msgr2=0x7fd8980fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 -- 172.21.15.150:0/2585928147 shutdown_connections 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.813+0000 7fd89ec92700 1 -- 172.21.15.150:0/2585928147 wait complete. 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 Processor -- start 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 -- start start 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd898069090 0x7fd89819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd898107060 0x7fd89819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898109aa0 0x7fd8981a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8981a1c80 con 0x7fd898109aa0 2024-05-10T04:43:28.817 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8981a1df0 con 0x7fd898107060 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ec92700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8981a1fb0 con 0x7fd898069090 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89d22f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898109aa0 0x7fd8981a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ca2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd898069090 0x7fd89819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ca2e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd898069090 0x7fd89819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.150:50204/0 (socket says 172.21.15.150:50204) 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89ca2e700 1 -- 172.21.15.150:0/281507245 learned_addr learned my addr 172.21.15.150:0/281507245 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd897fff700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd898107060 0x7fd89819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89d22f700 1 -- 172.21.15.150:0/281507245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd898069090 msgr2=0x7fd89819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89d22f700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd898069090 0x7fd89819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89d22f700 1 -- 172.21.15.150:0/281507245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd898107060 msgr2=0x7fd89819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89d22f700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd898107060 0x7fd89819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89d22f700 1 -- 172.21.15.150:0/281507245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c0095f0 con 0x7fd898109aa0 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.814+0000 7fd89d22f700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898109aa0 0x7fd8981a1670 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd890007f10 tx=0x7fd89000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:28.818 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.815+0000 7fd895ffb700 1 -- 172.21.15.150:0/281507245 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890009ea0 con 0x7fd898109aa0 2024-05-10T04:43:28.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.815+0000 7fd895ffb700 1 -- 172.21.15.150:0/281507245 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89000f040 con 0x7fd898109aa0 2024-05-10T04:43:28.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.815+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8981a2290 con 0x7fd898109aa0 2024-05-10T04:43:28.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.815+0000 7fd895ffb700 1 -- 172.21.15.150:0/281507245 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890004400 con 0x7fd898109aa0 2024-05-10T04:43:28.819 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.815+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8981a27a0 con 0x7fd898109aa0 2024-05-10T04:43:28.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.816+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8980fdc80 con 0x7fd898109aa0 2024-05-10T04:43:28.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.817+0000 7fd895ffb700 1 -- 172.21.15.150:0/281507245 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fd890004ad0 con 0x7fd898109aa0 2024-05-10T04:43:28.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.817+0000 7fd895ffb700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd8880a11a0 0x7fd8880a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:28.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.818+0000 7fd89ca2e700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd8880a11a0 0x7fd8880a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:28.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.818+0000 7fd895ffb700 1 -- 172.21.15.150:0/281507245 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fd8900cc470 con 0x7fd898109aa0 2024-05-10T04:43:28.820 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.818+0000 7fd89ca2e700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd8880a11a0 0x7fd8880a36b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00b2e0 tx=0x7fd88c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:28.823 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:28.822+0000 7fd895ffb700 1 -- 172.21.15.150:0/281507245 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd890095860 con 0x7fd898109aa0 2024-05-10T04:43:29.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.022+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8980694d0 con 0x7fd898109aa0 2024-05-10T04:43:29.025 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.024+0000 7fd895ffb700 1 -- 172.21.15.150:0/281507245 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24811 (secure 0 0 0) 0x7fd890098e80 con 0x7fd898109aa0 2024-05-10T04:43:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:43:29.026 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":90,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:40:13.400103+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd8880a11a0 msgr2=0x7fd8880a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd8880a11a0 0x7fd8880a36b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00b2e0 tx=0x7fd88c01a040 comp rx=0 tx=0).stop 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898109aa0 msgr2=0x7fd8981a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898109aa0 0x7fd8981a1670 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd890007f10 tx=0x7fd89000d3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 shutdown_connections 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd898109aa0 0x7fd8981a1670 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fd8880a11a0 0x7fd8880a36b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd898069090 0x7fd89819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 --2- 172.21.15.150:0/281507245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd898107060 0x7fd89819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 >> 172.21.15.150:0/281507245 conn(0x7fd8980fc0d0 msgr2=0x7fd898107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 shutdown_connections 2024-05-10T04:43:29.028 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.026+0000 7fd89ec92700 1 -- 172.21.15.150:0/281507245 wait complete. 2024-05-10T04:43:29.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-10T04:34:59.553928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-10T04:38:06.864687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-10T04:38:07.923038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-10T04:38:11.003459+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-10T04:43:29.042 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-10T04:43:29.182 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 -- 172.21.15.150:0/1955950995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 msgr2=0x7fb628100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1955950995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628100ee0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009990 tx=0x7fb61c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 -- 172.21.15.150:0/1955950995 shutdown_connections 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1955950995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628102590 0x7fb628108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1955950995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628100ee0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1955950995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb628101bf0 0x7fb628102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 -- 172.21.15.150:0/1955950995 >> 172.21.15.150:0/1955950995 conn(0x7fb6280fc0e0 msgr2=0x7fb6280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 -- 172.21.15.150:0/1955950995 shutdown_connections 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 -- 172.21.15.150:0/1955950995 wait complete. 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 Processor -- start 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.179+0000 7fb62ebda700 1 -- start start 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62ebda700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.183 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62ebda700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628101bf0 0x7fb628197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62ebda700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb628102590 0x7fb62819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62ebda700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62819e8e0 con 0x7fb628100ad0 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62ebda700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62819ea20 con 0x7fb628101bf0 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62ebda700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62819ebb0 con 0x7fb628102590 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60668/0 (socket says 172.21.15.150:60668) 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 -- 172.21.15.150:0/1042460667 learned_addr learned my addr 172.21.15.150:0/1042460667 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb627fff700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628101bf0 0x7fb628197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62d177700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb628102590 0x7fb62819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 -- 172.21.15.150:0/1042460667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb628102590 msgr2=0x7fb62819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb628102590 0x7fb62819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 -- 172.21.15.150:0/1042460667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628101bf0 msgr2=0x7fb628197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.184 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628101bf0 0x7fb628197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 -- 172.21.15.150:0/1042460667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0095f0 con 0x7fb628100ad0 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb627fff700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628101bf0 0x7fb628197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62c976700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628196ed0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00bc60 tx=0x7fb61c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb625ffb700 1 -- 172.21.15.150:0/1042460667 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c01d070 con 0x7fb628100ad0 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb625ffb700 1 -- 172.21.15.150:0/1042460667 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb61c003e30 con 0x7fb628100ad0 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62819ee30 con 0x7fb628100ad0 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.180+0000 7fb625ffb700 1 -- 172.21.15.150:0/1042460667 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0173f0 con 0x7fb628100ad0 2024-05-10T04:43:29.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.181+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62819f1a0 con 0x7fb628100ad0 2024-05-10T04:43:29.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.182+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb628107ce0 con 0x7fb628100ad0 2024-05-10T04:43:29.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.183+0000 7fb625ffb700 1 -- 172.21.15.150:0/1042460667 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb61c00f560 con 0x7fb628100ad0 2024-05-10T04:43:29.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.183+0000 7fb625ffb700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6180a11c0 0x7fb6180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.184+0000 7fb625ffb700 1 -- 172.21.15.150:0/1042460667 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb61c0ce110 con 0x7fb628100ad0 2024-05-10T04:43:29.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.184+0000 7fb627fff700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6180a11c0 0x7fb6180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.186 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.184+0000 7fb627fff700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6180a11c0 0x7fb6180a36d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb628197f70 tx=0x7fb61400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:29.188 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.187+0000 7fb625ffb700 1 -- 172.21.15.150:0/1042460667 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb61c097530 con 0x7fb628100ad0 2024-05-10T04:43:29.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:29 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/281507245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:29.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.382+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb62810c560 con 0x7fb628100ad0 2024-05-10T04:43:29.384 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.383+0000 7fb625ffb700 1 -- 172.21.15.150:0/1042460667 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb61c09ab50 con 0x7fb628100ad0 2024-05-10T04:43:29.384 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 1 2024-05-10T04:43:29.386 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6180a11c0 msgr2=0x7fb6180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6180a11c0 0x7fb6180a36d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb628197f70 tx=0x7fb61400a680 comp rx=0 tx=0).stop 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 msgr2=0x7fb628196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628196ed0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00bc60 tx=0x7fb61c003b90 comp rx=0 tx=0).stop 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 shutdown_connections 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb6180a11c0 0x7fb6180a36d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb628102590 0x7fb62819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb628100ad0 0x7fb628196ed0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 --2- 172.21.15.150:0/1042460667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628101bf0 0x7fb628197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 >> 172.21.15.150:0/1042460667 conn(0x7fb6280fc0e0 msgr2=0x7fb628104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 shutdown_connections 2024-05-10T04:43:29.387 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.385+0000 7fb62ebda700 1 -- 172.21.15.150:0/1042460667 wait complete. 2024-05-10T04:43:29.398 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:29 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/281507245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:29.399 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-10T04:43:29.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:29 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/281507245' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:29.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 -- 172.21.15.150:0/3593222826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84100ad0 msgr2=0x7f0c84100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 --2- 172.21.15.150:0/3593222826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84100ad0 0x7f0c84100ee0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009990 tx=0x7f0c78009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 -- 172.21.15.150:0/3593222826 shutdown_connections 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 --2- 172.21.15.150:0/3593222826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c84102590 0x7f0c84108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 --2- 172.21.15.150:0/3593222826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84100ad0 0x7f0c84100ee0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 --2- 172.21.15.150:0/3593222826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c84101bf0 0x7f0c84102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 -- 172.21.15.150:0/3593222826 >> 172.21.15.150:0/3593222826 conn(0x7f0c840fc0e0 msgr2=0x7f0c840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 -- 172.21.15.150:0/3593222826 shutdown_connections 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 -- 172.21.15.150:0/3593222826 wait complete. 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.535+0000 7f0c8950c700 1 Processor -- start 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c8950c700 1 -- start start 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c8950c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c84100ad0 0x7f0c84196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c8950c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84101bf0 0x7f0c841973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c8950c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c84102590 0x7f0c8419e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.539 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c8950c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8419e930 con 0x7f0c84101bf0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c8950c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8419ea70 con 0x7f0c84102590 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c8950c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8419ec00 con 0x7f0c84100ad0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84101bf0 0x7f0c841973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84101bf0 0x7f0c841973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60684/0 (socket says 172.21.15.150:60684) 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 -- 172.21.15.150:0/1752488180 learned_addr learned my addr 172.21.15.150:0/1752488180 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c837fe700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c84102590 0x7f0c8419e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c82ffd700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c84100ad0 0x7f0c84196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 -- 172.21.15.150:0/1752488180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c84100ad0 msgr2=0x7f0c84196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c84100ad0 0x7f0c84196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 -- 172.21.15.150:0/1752488180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c84102590 msgr2=0x7f0c8419e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c84102590 0x7f0c8419e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.536+0000 7f0c827fc700 1 -- 172.21.15.150:0/1752488180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c780095f0 con 0x7f0c84101bf0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.537+0000 7f0c827fc700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84101bf0 0x7f0c841973d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c9e0 tx=0x7f0c7400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.537+0000 7f0c6bfff700 1 -- 172.21.15.150:0/1752488180 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74007ac0 con 0x7f0c84101bf0 2024-05-10T04:43:29.540 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.537+0000 7f0c6bfff700 1 -- 172.21.15.150:0/1752488180 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c7400ce80 con 0x7f0c84101bf0 2024-05-10T04:43:29.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.537+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8419eee0 con 0x7f0c84101bf0 2024-05-10T04:43:29.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.537+0000 7f0c6bfff700 1 -- 172.21.15.150:0/1752488180 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74018660 con 0x7f0c84101bf0 2024-05-10T04:43:29.541 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.537+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8419f300 con 0x7f0c84101bf0 2024-05-10T04:43:29.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.538+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c84107ce0 con 0x7f0c84101bf0 2024-05-10T04:43:29.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.539+0000 7f0c6bfff700 1 -- 172.21.15.150:0/1752488180 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f0c7400f450 con 0x7f0c84101bf0 2024-05-10T04:43:29.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.539+0000 7f0c6bfff700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c6c0a11c0 0x7f0c6c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.540+0000 7f0c82ffd700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c6c0a11c0 0x7f0c6c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.540+0000 7f0c6bfff700 1 -- 172.21.15.150:0/1752488180 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f0c740cce90 con 0x7f0c84101bf0 2024-05-10T04:43:29.542 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.540+0000 7f0c82ffd700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c6c0a11c0 0x7f0c6c0a36d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0c780098e0 tx=0x7f0c7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:29.544 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.543+0000 7f0c6bfff700 1 -- 172.21.15.150:0/1752488180 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0c74096310 con 0x7f0c84101bf0 2024-05-10T04:43:29.742 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.741+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c8410c2b0 con 0x7f0c84101bf0 2024-05-10T04:43:29.743 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.742+0000 7f0c6bfff700 1 -- 172.21.15.150:0/1752488180 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c7400fe60 con 0x7f0c84101bf0 2024-05-10T04:43:29.743 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:43:29.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c6c0a11c0 msgr2=0x7f0c6c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c6c0a11c0 0x7f0c6c0a36d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0c780098e0 tx=0x7f0c7801a040 comp rx=0 tx=0).stop 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84101bf0 msgr2=0x7f0c841973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84101bf0 0x7f0c841973d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400c9e0 tx=0x7f0c7400ccf0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 shutdown_connections 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c84102590 0x7f0c8419e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f0c6c0a11c0 0x7f0c6c0a36d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c84100ad0 0x7f0c84196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 --2- 172.21.15.150:0/1752488180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c84101bf0 0x7f0c841973d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 >> 172.21.15.150:0/1752488180 conn(0x7f0c840fc0e0 msgr2=0x7f0c84104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 shutdown_connections 2024-05-10T04:43:29.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.744+0000 7f0c8950c700 1 -- 172.21.15.150:0/1752488180 wait complete. 2024-05-10T04:43:29.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 -- 172.21.15.150:0/2311850187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 msgr2=0x7f4b281024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 --2- 172.21.15.150:0/2311850187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b281024b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009990 tx=0x7f4b1c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 -- 172.21.15.150:0/2311850187 shutdown_connections 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 --2- 172.21.15.150:0/2311850187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b28101700 0x7f4b28109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 --2- 172.21.15.150:0/2311850187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b281024b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 --2- 172.21.15.150:0/2311850187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b28100db0 0x7f4b281011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 -- 172.21.15.150:0/2311850187 >> 172.21.15.150:0/2311850187 conn(0x7f4b280fb8f0 msgr2=0x7f4b280fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 -- 172.21.15.150:0/2311850187 shutdown_connections 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.893+0000 7f4b2f114700 1 -- 172.21.15.150:0/2311850187 wait complete. 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 Processor -- start 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 -- start start 2024-05-10T04:43:29.897 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b28100db0 0x7f4b28197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b28101700 0x7f4b281983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b2819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2819f990 con 0x7f4b281020a0 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2819fb00 con 0x7f4b28101700 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2f114700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2819fcc0 con 0x7f4b28100db0 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b2819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b2819f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60712/0 (socket says 172.21.15.150:60712) 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 -- 172.21.15.150:0/269607041 learned_addr learned my addr 172.21.15.150:0/269607041 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b27fff700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b28101700 0x7f4b281983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2ceb0700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b28100db0 0x7f4b28197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 -- 172.21.15.150:0/269607041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b28100db0 msgr2=0x7f4b28197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b28100db0 0x7f4b28197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 -- 172.21.15.150:0/269607041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b28101700 msgr2=0x7f4b281983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b28101700 0x7f4b281983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.894+0000 7f4b2d6b1700 1 -- 172.21.15.150:0/269607041 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0095f0 con 0x7f4b281020a0 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.895+0000 7f4b2d6b1700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b2819f380 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000ef30 tx=0x7f4b2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.895+0000 7f4b25ffb700 1 -- 172.21.15.150:0/269607041 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b20018070 con 0x7f4b281020a0 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.895+0000 7f4b25ffb700 1 -- 172.21.15.150:0/269607041 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b2000f040 con 0x7f4b281020a0 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.895+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2819ff40 con 0x7f4b281020a0 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.895+0000 7f4b25ffb700 1 -- 172.21.15.150:0/269607041 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000cd00 con 0x7f4b281020a0 2024-05-10T04:43:29.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.895+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b281a0370 con 0x7f4b281020a0 2024-05-10T04:43:29.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.896+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b280678b0 con 0x7f4b281020a0 2024-05-10T04:43:29.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.897+0000 7f4b25ffb700 1 -- 172.21.15.150:0/269607041 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f4b20003a00 con 0x7f4b281020a0 2024-05-10T04:43:29.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.898+0000 7f4b25ffb700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4b180a11c0 0x7f4b180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:29.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.898+0000 7f4b2ceb0700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4b180a11c0 0x7f4b180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:29.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.898+0000 7f4b25ffb700 1 -- 172.21.15.150:0/269607041 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f4b200cc820 con 0x7f4b281020a0 2024-05-10T04:43:29.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.898+0000 7f4b2ceb0700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4b180a11c0 0x7f4b180a36d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b2e0 tx=0x7f4b1c011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:29.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:29.903+0000 7f4b25ffb700 1 -- 172.21.15.150:0/269607041 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4b20095c40 con 0x7f4b281020a0 2024-05-10T04:43:30.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.103+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b281a0680 con 0x7f4b281020a0 2024-05-10T04:43:30.104 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.103+0000 7f4b25ffb700 1 -- 172.21.15.150:0/269607041 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b200b7e60 con 0x7f4b281020a0 2024-05-10T04:43:30.104 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 64 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4b180a11c0 msgr2=0x7f4b180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4b180a11c0 0x7f4b180a36d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b2e0 tx=0x7f4b1c011040 comp rx=0 tx=0).stop 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 msgr2=0x7f4b2819f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b2819f380 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000ef30 tx=0x7f4b2000c3a0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 shutdown_connections 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b281020a0 0x7f4b2819f380 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f4b180a11c0 0x7f4b180a36d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b28100db0 0x7f4b28197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 --2- 172.21.15.150:0/269607041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b28101700 0x7f4b281983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 >> 172.21.15.150:0/269607041 conn(0x7f4b280fb8f0 msgr2=0x7f4b280fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 shutdown_connections 2024-05-10T04:43:30.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.105+0000 7f4b2f114700 1 -- 172.21.15.150:0/269607041 wait complete. 2024-05-10T04:43:30.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-10T04:43:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 -- 172.21.15.150:0/683467604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 msgr2=0x7fb3281085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/683467604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb3281085f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c210 tx=0x7fb32000c520 comp rx=0 tx=0).stop 2024-05-10T04:43:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 -- 172.21.15.150:0/683467604 shutdown_connections 2024-05-10T04:43:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/683467604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb3281085f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/683467604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb328100280 0x7fb328100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.257 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/683467604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3281013a0 0x7fb3281017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 -- 172.21.15.150:0/683467604 >> 172.21.15.150:0/683467604 conn(0x7fb3280fb8b0 msgr2=0x7fb3280fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 -- 172.21.15.150:0/683467604 shutdown_connections 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.253+0000 7fb32f3bc700 1 -- 172.21.15.150:0/683467604 wait complete. 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 Processor -- start 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 -- start start 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb328100280 0x7fb328195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3281013a0 0x7fb328196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb32819d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819d8d0 con 0x7fb328101d40 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819da40 con 0x7fb328100280 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32f3bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32819dc00 con 0x7fb3281013a0 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32d959700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb32819d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32d959700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb32819d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60730/0 (socket says 172.21.15.150:60730) 2024-05-10T04:43:30.258 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.254+0000 7fb32d959700 1 -- 172.21.15.150:0/3428791465 learned_addr learned my addr 172.21.15.150:0/3428791465 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32d158700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb328100280 0x7fb328195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32c957700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3281013a0 0x7fb328196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32d959700 1 -- 172.21.15.150:0/3428791465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3281013a0 msgr2=0x7fb328196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32d959700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3281013a0 0x7fb328196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32d959700 1 -- 172.21.15.150:0/3428791465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb328100280 msgr2=0x7fb328195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32d959700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb328100280 0x7fb328195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32d959700 1 -- 172.21.15.150:0/3428791465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32000ae10 con 0x7fb328101d40 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32d959700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb32819d2c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb320004fd0 tx=0x7fb32000df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb31e7fc700 1 -- 172.21.15.150:0/3428791465 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32000f040 con 0x7fb328101d40 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb31e7fc700 1 -- 172.21.15.150:0/3428791465 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb32001e940 con 0x7fb328101d40 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32819de80 con 0x7fb328101d40 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb31e7fc700 1 -- 172.21.15.150:0/3428791465 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32001c9f0 con 0x7fb328101d40 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.255+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32819e250 con 0x7fb328101d40 2024-05-10T04:43:30.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.256+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb328107490 con 0x7fb328101d40 2024-05-10T04:43:30.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.257+0000 7fb31e7fc700 1 -- 172.21.15.150:0/3428791465 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb32001cb50 con 0x7fb328101d40 2024-05-10T04:43:30.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.258+0000 7fb31e7fc700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3140a11c0 0x7fb3140a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.260 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.258+0000 7fb31e7fc700 1 -- 172.21.15.150:0/3428791465 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb3200cfc70 con 0x7fb328101d40 2024-05-10T04:43:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.258+0000 7fb32d158700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3140a11c0 0x7fb3140a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.261 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.259+0000 7fb32d158700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3140a11c0 0x7fb3140a36d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb318005950 tx=0x7fb31800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:30.263 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.262+0000 7fb31e7fc700 1 -- 172.21.15.150:0/3428791465 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3200990f0 con 0x7fb328101d40 2024-05-10T04:43:30.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:30 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1042460667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:43:30.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:30 smithi161 ceph-mon[46912]: pgmap v389: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:43:30.335 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:30 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1752488180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:43:30.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:30 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1042460667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:43:30.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:30 smithi181 ceph-mon[47314]: pgmap v389: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:43:30.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:30 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1752488180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:43:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:30 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1042460667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-10T04:43:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:30 smithi150 ceph-mon[42404]: pgmap v389: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-10T04:43:30.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:30 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1752488180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-10T04:43:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.460+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb32810b280 con 0x7fb328101d40 2024-05-10T04:43:30.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.461+0000 7fb31e7fc700 1 -- 172.21.15.150:0/3428791465 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb32009c710 con 0x7fb328101d40 2024-05-10T04:43:30.462 INFO:teuthology.orchestra.run.smithi150.stdout:pg_num: 32 2024-05-10T04:43:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3140a11c0 msgr2=0x7fb3140a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3140a11c0 0x7fb3140a36d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb318005950 tx=0x7fb31800a300 comp rx=0 tx=0).stop 2024-05-10T04:43:30.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 msgr2=0x7fb32819d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb32819d2c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb320004fd0 tx=0x7fb32000df20 comp rx=0 tx=0).stop 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 shutdown_connections 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb328101d40 0x7fb32819d2c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb328100280 0x7fb328195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3140a11c0 0x7fb3140a36d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 --2- 172.21.15.150:0/3428791465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3281013a0 0x7fb328196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 >> 172.21.15.150:0/3428791465 conn(0x7fb3280fb8b0 msgr2=0x7fb3281045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 shutdown_connections 2024-05-10T04:43:30.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.463+0000 7fb32f3bc700 1 -- 172.21.15.150:0/3428791465 wait complete. 2024-05-10T04:43:30.478 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 -- 172.21.15.150:0/2338805250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 msgr2=0x7f9528107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 --2- 172.21.15.150:0/2338805250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528107460 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f951c009990 tx=0x7f951c009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 -- 172.21.15.150:0/2338805250 shutdown_connections 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 --2- 172.21.15.150:0/2338805250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95281079a0 0x7f9528109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 --2- 172.21.15.150:0/2338805250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95281026e0 0x7f9528104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 --2- 172.21.15.150:0/2338805250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528107460 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 -- 172.21.15.150:0/2338805250 >> 172.21.15.150:0/2338805250 conn(0x7f95280fc0e0 msgr2=0x7f95280fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:30.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 -- 172.21.15.150:0/2338805250 shutdown_connections 2024-05-10T04:43:30.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.610+0000 7f952d49d700 1 -- 172.21.15.150:0/2338805250 wait complete. 2024-05-10T04:43:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.611+0000 7f952d49d700 1 Processor -- start 2024-05-10T04:43:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.611+0000 7f952d49d700 1 -- start start 2024-05-10T04:43:30.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f952d49d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95281026e0 0x7f9528197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f952d49d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f952d49d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95281079a0 0x7f952819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f952d49d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819f9f0 con 0x7f9528105040 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f952d49d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819fb60 con 0x7f95281026e0 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f952d49d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819fd20 con 0x7f95281079a0 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60754/0 (socket says 172.21.15.150:60754) 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 -- 172.21.15.150:0/2600321969 learned_addr learned my addr 172.21.15.150:0/2600321969 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95277fe700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95281079a0 0x7f952819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f9526ffd700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95281026e0 0x7f9528197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 -- 172.21.15.150:0/2600321969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95281079a0 msgr2=0x7f952819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95281079a0 0x7f952819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 -- 172.21.15.150:0/2600321969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95281026e0 msgr2=0x7f9528197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95281026e0 0x7f9528197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 -- 172.21.15.150:0/2600321969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0095f0 con 0x7f9528105040 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95277fe700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95281079a0 0x7f952819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.612+0000 7f95267fc700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528198450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f951c0095c0 tx=0x7f951c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.613+0000 7f950ffff700 1 -- 172.21.15.150:0/2600321969 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c01d070 con 0x7f9528105040 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.613+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952819ffa0 con 0x7f9528105040 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.613+0000 7f950ffff700 1 -- 172.21.15.150:0/2600321969 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951c022470 con 0x7f9528105040 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.613+0000 7f950ffff700 1 -- 172.21.15.150:0/2600321969 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c017650 con 0x7f9528105040 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.613+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95281a0310 con 0x7f9528105040 2024-05-10T04:43:30.617 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.614+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95280fdc90 con 0x7f9528105040 2024-05-10T04:43:30.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.615+0000 7f950ffff700 1 -- 172.21.15.150:0/2600321969 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f951c0225e0 con 0x7f9528105040 2024-05-10T04:43:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.615+0000 7f950ffff700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95100a11c0 0x7f95100a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.616+0000 7f950ffff700 1 -- 172.21.15.150:0/2600321969 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f951c0ce420 con 0x7f9528105040 2024-05-10T04:43:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.616+0000 7f9526ffd700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95100a11c0 0x7f95100a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:30.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.616+0000 7f9526ffd700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95100a11c0 0x7f95100a36d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9518005950 tx=0x7f95180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:30.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.619+0000 7f950ffff700 1 -- 172.21.15.150:0/2600321969 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f951c0978d0 con 0x7f9528105040 2024-05-10T04:43:30.835 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.834+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9528068730 con 0x7f9528105040 2024-05-10T04:43:30.836 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.834+0000 7f950ffff700 1 -- 172.21.15.150:0/2600321969 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+599 (secure 0 0 0) 0x7f951c022900 con 0x7f9528105040 2024-05-10T04:43:30.836 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/1928870080 2024-05-11T04:33:11.238386+0000 2024-05-10T04:43:30.836 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3285068402 2024-05-11T04:33:11.238386+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/881426482 2024-05-11T04:33:11.238386+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/618215791 2024-05-11T04:32:48.043455+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/2517418767 2024-05-11T04:32:48.043455+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3236490116 2024-05-11T04:32:48.043455+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/881426482 2024-05-11T04:33:11.238386+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6801/2970915543 2024-05-11T04:32:48.043455+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:6800/2970915543 2024-05-11T04:32:48.043455+0000 2024-05-10T04:43:30.837 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150:0/3890124032 2024-05-11T04:33:11.238386+0000 2024-05-10T04:43:30.838 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.836+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95100a11c0 msgr2=0x7f95100a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.836+0000 7f952d49d700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95100a11c0 0x7f95100a36d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9518005950 tx=0x7f95180058e0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 msgr2=0x7f9528198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528198450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f951c0095c0 tx=0x7f951c00f690 comp rx=0 tx=0).stop 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 shutdown_connections 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f95100a11c0 0x7f95100a36d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95281079a0 0x7f952819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95281026e0 0x7f9528197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 --2- 172.21.15.150:0/2600321969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9528105040 0x7f9528198450 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 >> 172.21.15.150:0/2600321969 conn(0x7f95280fc0e0 msgr2=0x7f95281031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 shutdown_connections 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:30.837+0000 7f952d49d700 1 -- 172.21.15.150:0/2600321969 wait complete. 2024-05-10T04:43:30.839 INFO:teuthology.orchestra.run.smithi150.stderr:listed 10 entries 2024-05-10T04:43:30.851 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-10T04:43:30.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:43:30.851 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-10T04:43:30.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:43:30.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:43:30.910 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-10T04:43:30.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:43:30.943 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2024-05-10T04:43:31.001 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-10T04:43:31.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:43:31.001 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-10T04:43:31.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-10T04:43:31.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2024-05-10T04:43:31.102 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/269607041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:43:31.102 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3428791465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:43:31.102 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:31 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2600321969' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:43:31.157 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-10T04:43:31.168 INFO:tasks.cephadm:Teardown begin 2024-05-10T04:43:31.168 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-10T04:43:31.168 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:43:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 -- 172.21.15.150:0/3090806713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c103690 msgr2=0x7f844c109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 --2- 172.21.15.150:0/3090806713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c103690 0x7f844c109f40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f844400b0a0 tx=0x7f844400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.304 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 -- 172.21.15.150:0/3090806713 shutdown_connections 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 --2- 172.21.15.150:0/3090806713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c103690 0x7f844c109f40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 --2- 172.21.15.150:0/3090806713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844c101bd0 0x7f844c101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 --2- 172.21.15.150:0/3090806713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c102cf0 0x7f844c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 -- 172.21.15.150:0/3090806713 >> 172.21.15.150:0/3090806713 conn(0x7f844c0fd1e0 msgr2=0x7f844c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 -- 172.21.15.150:0/3090806713 shutdown_connections 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 -- 172.21.15.150:0/3090806713 wait complete. 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 Processor -- start 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.301+0000 7f8452f2f700 1 -- start start 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8452f2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844c101bd0 0x7f844c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8452f2f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c102cf0 0x7f844c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8452f2f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c103690 0x7f844c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8452f2f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c19fb60 con 0x7f844c102cf0 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8452f2f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c19fcd0 con 0x7f844c103690 2024-05-10T04:43:31.305 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8452f2f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c19fe90 con 0x7f844c101bd0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c102cf0 0x7f844c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c102cf0 0x7f844c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60776/0 (socket says 172.21.15.150:60776) 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 -- 172.21.15.150:0/2044663479 learned_addr learned my addr 172.21.15.150:0/2044663479 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8450ccb700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844c101bd0 0x7f844c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f84514cc700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c103690 0x7f844c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 -- 172.21.15.150:0/2044663479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844c101bd0 msgr2=0x7f844c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844c101bd0 0x7f844c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 -- 172.21.15.150:0/2044663479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c103690 msgr2=0x7f844c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c103690 0x7f844c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 -- 172.21.15.150:0/2044663479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8444009d00 con 0x7f844c102cf0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f844bfff700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c102cf0 0x7f844c1985c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f843800d950 tx=0x7f843800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8449ffb700 1 -- 172.21.15.150:0/2044663479 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843800f890 con 0x7f844c102cf0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c1a0170 con 0x7f844c102cf0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8449ffb700 1 -- 172.21.15.150:0/2044663479 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8438010460 con 0x7f844c102cf0 2024-05-10T04:43:31.306 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.302+0000 7f8449ffb700 1 -- 172.21.15.150:0/2044663479 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843800e610 con 0x7f844c102cf0 2024-05-10T04:43:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.303+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844c1a06d0 con 0x7f844c102cf0 2024-05-10T04:43:31.307 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.304+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844c108de0 con 0x7f844c102cf0 2024-05-10T04:43:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.305+0000 7f8449ffb700 1 -- 172.21.15.150:0/2044663479 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f84380108d0 con 0x7f844c102cf0 2024-05-10T04:43:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.305+0000 7f8449ffb700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f843c0a1120 0x7f843c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.306+0000 7f8449ffb700 1 -- 172.21.15.150:0/2044663479 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f84380cca10 con 0x7f844c102cf0 2024-05-10T04:43:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.306+0000 7f8450ccb700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f843c0a1120 0x7f843c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.308 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.306+0000 7f8450ccb700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f843c0a1120 0x7f843c0a3630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f844c102bf0 tx=0x7f8440007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:31.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.310+0000 7f8449ffb700 1 -- 172.21.15.150:0/2044663479 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8438095e00 con 0x7f844c102cf0 2024-05-10T04:43:31.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/269607041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:43:31.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3428791465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:43:31.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:31 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2600321969' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:43:31.556 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.555+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f844c10d610 con 0x7f844c102cf0 2024-05-10T04:43:31.557 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.556+0000 7f8449ffb700 1 -- 172.21.15.150:0/2044663479 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f8438016090 con 0x7f844c102cf0 2024-05-10T04:43:31.557 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:43:31.557 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:43:31.559 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f843c0a1120 msgr2=0x7f843c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f843c0a1120 0x7f843c0a3630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f844c102bf0 tx=0x7f8440007400 comp rx=0 tx=0).stop 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c102cf0 msgr2=0x7f844c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c102cf0 0x7f844c1985c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f843800d950 tx=0x7f843800dc60 comp rx=0 tx=0).stop 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 shutdown_connections 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f844c103690 0x7f844c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f843c0a1120 0x7f843c0a3630 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844c101bd0 0x7f844c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 --2- 172.21.15.150:0/2044663479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c102cf0 0x7f844c1985c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 >> 172.21.15.150:0/2044663479 conn(0x7f844c0fd1e0 msgr2=0x7f844c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 shutdown_connections 2024-05-10T04:43:31.560 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.558+0000 7f8452f2f700 1 -- 172.21.15.150:0/2044663479 wait complete. 2024-05-10T04:43:31.561 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:43:31.573 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-10T04:43:31.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/269607041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-10T04:43:31.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3428791465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-10T04:43:31.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:31 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2600321969' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-10T04:43:31.709 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 -- 172.21.15.150:0/4063407767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 msgr2=0x7f10a0101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 --2- 172.21.15.150:0/4063407767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a0101ff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1090009990 tx=0x7f1090009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 -- 172.21.15.150:0/4063407767 shutdown_connections 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 --2- 172.21.15.150:0/4063407767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0102580 0x7f10a0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 --2- 172.21.15.150:0/4063407767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a0100ac0 0x7f10a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 --2- 172.21.15.150:0/4063407767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a0101ff0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 -- 172.21.15.150:0/4063407767 >> 172.21.15.150:0/4063407767 conn(0x7f10a00fc0d0 msgr2=0x7f10a00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 -- 172.21.15.150:0/4063407767 shutdown_connections 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.705+0000 7f10a7d54700 1 -- 172.21.15.150:0/4063407767 wait complete. 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 Processor -- start 2024-05-10T04:43:31.710 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 -- start start 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a0100ac0 0x7f10a0196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a01973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0102580 0x7f10a019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a019e900 con 0x7f10a0101be0 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a019ea40 con 0x7f10a0102580 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a7d54700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a019ebd0 con 0x7f10a0100ac0 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a01973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a01973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60790/0 (socket says 172.21.15.150:60790) 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 -- 172.21.15.150:0/1100011636 learned_addr learned my addr 172.21.15.150:0/1100011636 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a62f1700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0102580 0x7f10a019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a5af0700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a0100ac0 0x7f10a0196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 -- 172.21.15.150:0/1100011636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a0100ac0 msgr2=0x7f10a0196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a0100ac0 0x7f10a0196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 -- 172.21.15.150:0/1100011636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0102580 msgr2=0x7f10a019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.711 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0102580 0x7f10a019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 -- 172.21.15.150:0/1100011636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900095f0 con 0x7f10a0101be0 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.706+0000 7f10a52ef700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a01973a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f109000c010 tx=0x7f1090003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.707+0000 7f1096ffd700 1 -- 172.21.15.150:0/1100011636 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109001d070 con 0x7f10a0101be0 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.707+0000 7f1096ffd700 1 -- 172.21.15.150:0/1100011636 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1090003e30 con 0x7f10a0101be0 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.707+0000 7f1096ffd700 1 -- 172.21.15.150:0/1100011636 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10900173f0 con 0x7f10a0101be0 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.707+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a019ee50 con 0x7f10a0101be0 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.707+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a019f1c0 con 0x7f10a0101be0 2024-05-10T04:43:31.712 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.707+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0107cd0 con 0x7f10a0101be0 2024-05-10T04:43:31.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.709+0000 7f1096ffd700 1 -- 172.21.15.150:0/1100011636 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f109000f5f0 con 0x7f10a0101be0 2024-05-10T04:43:31.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.709+0000 7f1096ffd700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f108c0a1120 0x7f108c0a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:31.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.710+0000 7f1096ffd700 1 -- 172.21.15.150:0/1100011636 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f10900ce070 con 0x7f10a0101be0 2024-05-10T04:43:31.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.710+0000 7f10a5af0700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f108c0a1120 0x7f108c0a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:31.713 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.710+0000 7f10a5af0700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f108c0a1120 0x7f108c0a3630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f109c005fd0 tx=0x7f109c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:31.714 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.713+0000 7f1096ffd700 1 -- 172.21.15.150:0/1100011636 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f10900974f0 con 0x7f10a0101be0 2024-05-10T04:43:31.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.958+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10a00694d0 con 0x7f10a0101be0 2024-05-10T04:43:31.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.959+0000 7f1096ffd700 1 -- 172.21.15.150:0/1100011636 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10933 (secure 0 0 0) 0x7f109009ab10 con 0x7f10a0101be0 2024-05-10T04:43:31.960 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:43:31.961 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14598,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2690665605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2690665605},{"type":"v1","addr":"172.21.15.150:6835","nonce":2690665605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/2935316346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":2935316346},{"type":"v1","addr":"172.21.15.150:6837","nonce":2935316346}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-10T04:38:08.750329+0000","modified":"2024-05-10T04:39:38.447762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24484,"mds_3":24455,"mds_4":24445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14622":{"gid":14622,"name":"e","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/2306790268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":2306790268},{"type":"v1","addr":"172.21.15.161:6835","nonce":2306790268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14664":{"gid":14664,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/308347873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":308347873},{"type":"v1","addr":"172.21.15.150:6839","nonce":308347873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14676":{"gid":14676,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6841/3789106275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6840","nonce":3789106275},{"type":"v1","addr":"172.21.15.150:6841","nonce":3789106275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":14,"addr":"172.21.15.181:6833/1504167586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1504167586},{"type":"v1","addr":"172.21.15.181:6833","nonce":1504167586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":14,"addr":"172.21.15.161:6833/825127258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":825127258},{"type":"v1","addr":"172.21.15.161:6833","nonce":825127258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.181:6835/3090072990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3090072990},{"type":"v1","addr":"172.21.15.181:6835","nonce":3090072990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/1032605592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1032605592},{"type":"v1","addr":"172.21.15.181:6837","nonce":1032605592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/55043330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":55043330},{"type":"v1","addr":"172.21.15.161:6837","nonce":55043330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.181:6839/4287682926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":4287682926},{"type":"v1","addr":"172.21.15.181:6839","nonce":4287682926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6839/3001288873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":3001288873},{"type":"v1","addr":"172.21.15.161:6839","nonce":3001288873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-10T04:43:31.962 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f108c0a1120 msgr2=0x7f108c0a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f108c0a1120 0x7f108c0a3630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f109c005fd0 tx=0x7f109c005e50 comp rx=0 tx=0).stop 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 msgr2=0x7f10a01973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a01973a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f109000c010 tx=0x7f1090003b90 comp rx=0 tx=0).stop 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 shutdown_connections 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10a0102580 0x7f10a019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f108c0a1120 0x7f108c0a3630 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a0100ac0 0x7f10a0196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 --2- 172.21.15.150:0/1100011636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10a0101be0 0x7f10a01973a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 >> 172.21.15.150:0/1100011636 conn(0x7f10a00fc0d0 msgr2=0x7f10a0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 shutdown_connections 2024-05-10T04:43:31.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:31.961+0000 7f10a7d54700 1 -- 172.21.15.150:0/1100011636 wait complete. 2024-05-10T04:43:31.964 INFO:teuthology.orchestra.run.smithi150.stderr:dumped fsmap epoch 29 2024-05-10T04:43:31.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2334192816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8101c00 msgr2=0x7fb3f8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2334192816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8101c00 0x7fb3f8102010 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009990 tx=0x7fb3e8009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2334192816 shutdown_connections 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2334192816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f81025a0 0x7fb3f8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2334192816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f8100ae0 0x7fb3f8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2334192816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8101c00 0x7fb3f8102010 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2334192816 >> 172.21.15.150:0/2334192816 conn(0x7fb3f80fc0f0 msgr2=0x7fb3f80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2334192816 shutdown_connections 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.109+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2334192816 wait complete. 2024-05-10T04:43:32.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 Processor -- start 2024-05-10T04:43:32.112 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 -- start start 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8100ae0 0x7fb3f8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f8101c00 0x7fb3f8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f81025a0 0x7fb3f819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819fa90 con 0x7fb3f8100ae0 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819fbd0 con 0x7fb3f8101c00 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fe9e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f819fd60 con 0x7fb3f81025a0 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8100ae0 0x7fb3f8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8100ae0 0x7fb3f8197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60810/0 (socket says 172.21.15.150:60810) 2024-05-10T04:43:32.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 -- 172.21.15.150:0/2807216868 learned_addr learned my addr 172.21.15.150:0/2807216868 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:32.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f77fe700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f8101c00 0x7fb3f8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3fcf7d700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f81025a0 0x7fb3f819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 -- 172.21.15.150:0/2807216868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f81025a0 msgr2=0x7fb3f819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f81025a0 0x7fb3f819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 -- 172.21.15.150:0/2807216868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f8101c00 msgr2=0x7fb3f8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f8101c00 0x7fb3f8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.110+0000 7fb3f7fff700 1 -- 172.21.15.150:0/2807216868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e80095f0 con 0x7fb3f8100ae0 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.111+0000 7fb3f7fff700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8100ae0 0x7fb3f8197ff0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000d980 tx=0x7fb3e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.111+0000 7fb3f57fa700 1 -- 172.21.15.150:0/2807216868 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0004220 con 0x7fb3f8100ae0 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.111+0000 7fb3f57fa700 1 -- 172.21.15.150:0/2807216868 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e0004380 con 0x7fb3f8100ae0 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.111+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f819ffe0 con 0x7fb3f8100ae0 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.111+0000 7fb3f57fa700 1 -- 172.21.15.150:0/2807216868 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0003bf0 con 0x7fb3f8100ae0 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.111+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81a0490 con 0x7fb3f8100ae0 2024-05-10T04:43:32.117 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.112+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f8068730 con 0x7fb3f8100ae0 2024-05-10T04:43:32.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.113+0000 7fb3f57fa700 1 -- 172.21.15.150:0/2807216868 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fb3e0003d50 con 0x7fb3f8100ae0 2024-05-10T04:43:32.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.114+0000 7fb3f57fa700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3e40a1120 0x7fb3e40a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.114+0000 7fb3f57fa700 1 -- 172.21.15.150:0/2807216868 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7fb3e00cc7f0 con 0x7fb3f8100ae0 2024-05-10T04:43:32.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.114+0000 7fb3f77fe700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3e40a1120 0x7fb3e40a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.114+0000 7fb3f77fe700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3e40a1120 0x7fb3e40a3630 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800b5c0 tx=0x7fb3e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:32.119 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.118+0000 7fb3f57fa700 1 -- 172.21.15.150:0/2807216868 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb3e0095be0 con 0x7fb3f8100ae0 2024-05-10T04:43:32.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.315+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3f810c2e0 con 0x7fb3f8100ae0 2024-05-10T04:43:32.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.316+0000 7fb3f57fa700 1 -- 172.21.15.150:0/2807216868 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24811 (secure 0 0 0) 0x7fb3e0099200 con 0x7fb3f8100ae0 2024-05-10T04:43:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-10T04:43:32.318 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":90,"fsid":"28928ad0-0e86-11ef-bc98-c7b262605968","created":"2024-05-10T04:32:24.457948+0000","modified":"2024-05-10T04:40:13.400103+0000","last_up_change":"2024-05-10T04:37:35.581410+0000","last_in_change":"2024-05-10T04:37:18.941716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-10T04:34:59.553928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-10T04:38:06.864687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-10T04:38:07.923038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-10T04:38:11.003459+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cc48a860-ef1e-4d94-be5a-6c8758a2b7e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6803","nonce":4183993127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6805","nonce":4183993127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6809","nonce":4183993127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4183993127},{"type":"v1","addr":"172.21.15.150:6807","nonce":4183993127}]},"public_addr":"172.21.15.150:6803/4183993127","cluster_addr":"172.21.15.150:6805/4183993127","heartbeat_back_addr":"172.21.15.150:6809/4183993127","heartbeat_front_addr":"172.21.15.150:6807/4183993127","state":["exists","up"]},{"osd":1,"uuid":"13a6ad73-ce28-44f3-bdf4-178f42703543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6801","nonce":2361791264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6803","nonce":2361791264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6807","nonce":2361791264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2361791264},{"type":"v1","addr":"172.21.15.161:6805","nonce":2361791264}]},"public_addr":"172.21.15.161:6801/2361791264","cluster_addr":"172.21.15.161:6803/2361791264","heartbeat_back_addr":"172.21.15.161:6807/2361791264","heartbeat_front_addr":"172.21.15.161:6805/2361791264","state":["exists","up"]},{"osd":2,"uuid":"d1c4f641-65de-4cd3-8a02-b1694aa6e086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6801","nonce":2822398363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6803","nonce":2822398363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6807","nonce":2822398363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2822398363},{"type":"v1","addr":"172.21.15.181:6805","nonce":2822398363}]},"public_addr":"172.21.15.181:6801/2822398363","cluster_addr":"172.21.15.181:6803/2822398363","heartbeat_back_addr":"172.21.15.181:6807/2822398363","heartbeat_front_addr":"172.21.15.181:6805/2822398363","state":["exists","up"]},{"osd":3,"uuid":"589765c5-703f-4ac8-b3d4-07fdbc0de50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6811","nonce":217079495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6813","nonce":217079495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6817","nonce":217079495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":217079495},{"type":"v1","addr":"172.21.15.150:6815","nonce":217079495}]},"public_addr":"172.21.15.150:6811/217079495","cluster_addr":"172.21.15.150:6813/217079495","heartbeat_back_addr":"172.21.15.150:6817/217079495","heartbeat_front_addr":"172.21.15.150:6815/217079495","state":["exists","up"]},{"osd":4,"uuid":"8507ef53-8650-4e6f-b8a2-482d3b632548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6809","nonce":402152898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6811","nonce":402152898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6815","nonce":402152898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":402152898},{"type":"v1","addr":"172.21.15.161:6813","nonce":402152898}]},"public_addr":"172.21.15.161:6809/402152898","cluster_addr":"172.21.15.161:6811/402152898","heartbeat_back_addr":"172.21.15.161:6815/402152898","heartbeat_front_addr":"172.21.15.161:6813/402152898","state":["exists","up"]},{"osd":5,"uuid":"4632c68c-e68e-44c9-9f14-8f5a2c836c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6809","nonce":189565731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6811","nonce":189565731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6815","nonce":189565731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":189565731},{"type":"v1","addr":"172.21.15.181:6813","nonce":189565731}]},"public_addr":"172.21.15.181:6809/189565731","cluster_addr":"172.21.15.181:6811/189565731","heartbeat_back_addr":"172.21.15.181:6815/189565731","heartbeat_front_addr":"172.21.15.181:6813/189565731","state":["exists","up"]},{"osd":6,"uuid":"89f0a895-52f5-4cb3-97df-71df296938c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6819","nonce":3315977759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6821","nonce":3315977759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6825","nonce":3315977759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3315977759},{"type":"v1","addr":"172.21.15.150:6823","nonce":3315977759}]},"public_addr":"172.21.15.150:6819/3315977759","cluster_addr":"172.21.15.150:6821/3315977759","heartbeat_back_addr":"172.21.15.150:6825/3315977759","heartbeat_front_addr":"172.21.15.150:6823/3315977759","state":["exists","up"]},{"osd":7,"uuid":"09b5c539-71b8-42c8-97c4-3a3de4864a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6817","nonce":724331973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6819","nonce":724331973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6823","nonce":724331973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":724331973},{"type":"v1","addr":"172.21.15.161:6821","nonce":724331973}]},"public_addr":"172.21.15.161:6817/724331973","cluster_addr":"172.21.15.161:6819/724331973","heartbeat_back_addr":"172.21.15.161:6823/724331973","heartbeat_front_addr":"172.21.15.161:6821/724331973","state":["exists","up"]},{"osd":8,"uuid":"4aceac05-7393-4887-8b99-8ed52159a6a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6817","nonce":646590427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6819","nonce":646590427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6823","nonce":646590427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":646590427},{"type":"v1","addr":"172.21.15.181:6821","nonce":646590427}]},"public_addr":"172.21.15.181:6817/646590427","cluster_addr":"172.21.15.181:6819/646590427","heartbeat_back_addr":"172.21.15.181:6823/646590427","heartbeat_front_addr":"172.21.15.181:6821/646590427","state":["exists","up"]},{"osd":9,"uuid":"f84f2137-43f9-4072-8828-316f6a5f3582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6827","nonce":2987376476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6829","nonce":2987376476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6833","nonce":2987376476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2987376476},{"type":"v1","addr":"172.21.15.150:6831","nonce":2987376476}]},"public_addr":"172.21.15.150:6827/2987376476","cluster_addr":"172.21.15.150:6829/2987376476","heartbeat_back_addr":"172.21.15.150:6833/2987376476","heartbeat_front_addr":"172.21.15.150:6831/2987376476","state":["exists","up"]},{"osd":10,"uuid":"0a0228ed-9ae2-4aa7-bd86-db1412ad4969","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6825","nonce":2621232559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6827","nonce":2621232559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6831","nonce":2621232559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2621232559},{"type":"v1","addr":"172.21.15.161:6829","nonce":2621232559}]},"public_addr":"172.21.15.161:6825/2621232559","cluster_addr":"172.21.15.161:6827/2621232559","heartbeat_back_addr":"172.21.15.161:6831/2621232559","heartbeat_front_addr":"172.21.15.161:6829/2621232559","state":["exists","up"]},{"osd":11,"uuid":"815f00ef-e11c-42c7-a1b4-52bbfefc253e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6825","nonce":2057565898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6827","nonce":2057565898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6831","nonce":2057565898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":2057565898},{"type":"v1","addr":"172.21.15.181:6829","nonce":2057565898}]},"public_addr":"172.21.15.181:6825/2057565898","cluster_addr":"172.21.15.181:6827/2057565898","heartbeat_back_addr":"172.21.15.181:6831/2057565898","heartbeat_front_addr":"172.21.15.181:6829/2057565898","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:20.964229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:38.335159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:34:56.239347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:13.700353+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:30.586245+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:35:48.066688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:05.476057+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:22.866606+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:40.763982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:36:58.282671+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:15.735570+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-10T04:37:34.024971+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1928870080":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/3285068402":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:0/618215791":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/2517418767":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3236490116":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/881426482":"2024-05-11T04:33:11.238386+0000","172.21.15.150:6801/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:6800/2970915543":"2024-05-11T04:32:48.043455+0000","172.21.15.150:0/3890124032":"2024-05-11T04:33:11.238386+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-10T04:43:32.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3e40a1120 msgr2=0x7fb3e40a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.322 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3e40a1120 0x7fb3e40a3630 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800b5c0 tx=0x7fb3e801a040 comp rx=0 tx=0).stop 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8100ae0 msgr2=0x7fb3f8197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8100ae0 0x7fb3f8197ff0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000d980 tx=0x7fb3e000dc90 comp rx=0 tx=0).stop 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 shutdown_connections 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fb3e40a1120 0x7fb3e40a3630 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3f81025a0 0x7fb3f819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f8100ae0 0x7fb3f8197ff0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 --2- 172.21.15.150:0/2807216868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3f8101c00 0x7fb3f8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 >> 172.21.15.150:0/2807216868 conn(0x7fb3f80fc0f0 msgr2=0x7fb3f80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 shutdown_connections 2024-05-10T04:43:32.323 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.319+0000 7fb3fe9e0700 1 -- 172.21.15.150:0/2807216868 wait complete. 2024-05-10T04:43:32.332 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-mon[42404]: pgmap v390: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:32.332 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2044663479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:32.332 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/1100011636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:32.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-10T04:43:32.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:32 smithi181 ceph-mon[47314]: pgmap v390: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:32.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:32 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2044663479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:32.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:32 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/1100011636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 -- 172.21.15.150:0/109305208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c101c00 msgr2=0x7f135c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 --2- 172.21.15.150:0/109305208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c101c00 0x7f135c102010 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1350009990 tx=0x7f1350009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 -- 172.21.15.150:0/109305208 shutdown_connections 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 --2- 172.21.15.150:0/109305208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f135c1036c0 0x7f135c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 --2- 172.21.15.150:0/109305208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c101c00 0x7f135c102010 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 --2- 172.21.15.150:0/109305208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c102d20 0x7f135c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 -- 172.21.15.150:0/109305208 >> 172.21.15.150:0/109305208 conn(0x7f135c0fd210 msgr2=0x7f135c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 -- 172.21.15.150:0/109305208 shutdown_connections 2024-05-10T04:43:32.470 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.468+0000 7f1360b5a700 1 -- 172.21.15.150:0/109305208 wait complete. 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 Processor -- start 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 -- start start 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f135c101c00 0x7f135c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c102d20 0x7f135c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c1036c0 0x7f135c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c19fa00 con 0x7f135c1036c0 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c19fb70 con 0x7f135c101c00 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f1360b5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c19fd30 con 0x7f135c102d20 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f135ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c1036c0 0x7f135c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f135ad9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c1036c0 0x7f135c19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60836/0 (socket says 172.21.15.150:60836) 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f135ad9d700 1 -- 172.21.15.150:0/2856068677 learned_addr learned my addr 172.21.15.150:0/2856068677 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:32.473 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.469+0000 7f135a59c700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f135c101c00 0x7f135c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f135ad9d700 1 -- 172.21.15.150:0/2856068677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c102d20 msgr2=0x7f135c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f135ad9d700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c102d20 0x7f135c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f135ad9d700 1 -- 172.21.15.150:0/2856068677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f135c101c00 msgr2=0x7f135c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f135ad9d700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f135c101c00 0x7f135c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f135ad9d700 1 -- 172.21.15.150:0/2856068677 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13500095f0 con 0x7f135c1036c0 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f135a59c700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f135c101c00 0x7f135c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f135ad9d700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c1036c0 0x7f135c19f3f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1354007ee0 tx=0x7f135400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:32.474 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f134b7fe700 1 -- 172.21.15.150:0/2856068677 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135400ddd0 con 0x7f135c1036c0 2024-05-10T04:43:32.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f134b7fe700 1 -- 172.21.15.150:0/2856068677 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f135400f040 con 0x7f135c1036c0 2024-05-10T04:43:32.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c19ffb0 con 0x7f135c1036c0 2024-05-10T04:43:32.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f134b7fe700 1 -- 172.21.15.150:0/2856068677 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354015400 con 0x7f135c1036c0 2024-05-10T04:43:32.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.470+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c1a0440 con 0x7f135c1036c0 2024-05-10T04:43:32.475 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.471+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135c108e10 con 0x7f135c1036c0 2024-05-10T04:43:32.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.472+0000 7f134b7fe700 1 -- 172.21.15.150:0/2856068677 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f1354004000 con 0x7f135c1036c0 2024-05-10T04:43:32.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.473+0000 7f134b7fe700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13440a11c0 0x7f13440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.473+0000 7f135a59c700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13440a11c0 0x7f13440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.473+0000 7f134b7fe700 1 -- 172.21.15.150:0/2856068677 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9165+0+0 (secure 0 0 0) 0x7f13540cccd0 con 0x7f135c1036c0 2024-05-10T04:43:32.476 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.473+0000 7f135a59c700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13440a11c0 0x7f13440a36d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f13500038c0 tx=0x7f135001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:32.477 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.476+0000 7f134b7fe700 1 -- 172.21.15.150:0/2856068677 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f13540960c0 con 0x7f135c1036c0 2024-05-10T04:43:32.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:32 smithi161 ceph-mon[46912]: pgmap v390: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-10T04:43:32.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:32 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2044663479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:32.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:32 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/1100011636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-10T04:43:32.721 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.720+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f135c10d3e0 con 0x7f135c1036c0 2024-05-10T04:43:32.732 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.731+0000 7f134b7fe700 1 -- 172.21.15.150:0/2856068677 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1354023090 con 0x7f135c1036c0 2024-05-10T04:43:32.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13440a11c0 msgr2=0x7f13440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13440a11c0 0x7f13440a36d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f13500038c0 tx=0x7f135001a040 comp rx=0 tx=0).stop 2024-05-10T04:43:32.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c1036c0 msgr2=0x7f135c19f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c1036c0 0x7f135c19f3f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1354007ee0 tx=0x7f135400d3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 shutdown_connections 2024-05-10T04:43:32.736 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f135c1036c0 0x7f135c19f3f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f135c101c00 0x7f135c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13440a11c0 0x7f13440a36d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 --2- 172.21.15.150:0/2856068677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c102d20 0x7f135c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 >> 172.21.15.150:0/2856068677 conn(0x7f135c0fd210 msgr2=0x7f135c105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:32.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 shutdown_connections 2024-05-10T04:43:32.737 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.734+0000 7f1360b5a700 1 -- 172.21.15.150:0/2856068677 wait complete. 2024-05-10T04:43:32.737 INFO:teuthology.orchestra.run.smithi150.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-10T04:43:32.749 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-10T04:43:32.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 -- 172.21.15.150:0/2037896686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974101680 msgr2=0x7f6974108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 --2- 172.21.15.150:0/2037896686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974101680 0x7f6974108f90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f696400b0a0 tx=0x7f696400b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.898 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 -- 172.21.15.150:0/2037896686 shutdown_connections 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 --2- 172.21.15.150:0/2037896686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974101680 0x7f6974108f90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 --2- 172.21.15.150:0/2037896686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6974102030 0x7f6974102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 --2- 172.21.15.150:0/2037896686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6974100d30 0x7f6974101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 -- 172.21.15.150:0/2037896686 >> 172.21.15.150:0/2037896686 conn(0x7f69740fb8f0 msgr2=0x7f69740fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 -- 172.21.15.150:0/2037896686 shutdown_connections 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.894+0000 7f69794eb700 1 -- 172.21.15.150:0/2037896686 wait complete. 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 Processor -- start 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 -- start start 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6974100d30 0x7f6974196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6974101680 0x7f69741974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974102030 0x7f697419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697419e9f0 con 0x7f6974102030 2024-05-10T04:43:32.899 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697419eb60 con 0x7f6974101680 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69794eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697419ed20 con 0x7f6974100d30 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974102030 0x7f697419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69737fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974102030 0x7f697419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60914/0 (socket says 172.21.15.150:60914) 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69737fe700 1 -- 172.21.15.150:0/3106863273 learned_addr learned my addr 172.21.15.150:0/3106863273 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f6972ffd700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6974100d30 0x7f6974196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.895+0000 7f69727fc700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6974101680 0x7f69741974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69737fe700 1 -- 172.21.15.150:0/3106863273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6974100d30 msgr2=0x7f6974196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69737fe700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6974100d30 0x7f6974196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69737fe700 1 -- 172.21.15.150:0/3106863273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6974101680 msgr2=0x7f69741974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69737fe700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6974101680 0x7f69741974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69737fe700 1 -- 172.21.15.150:0/3106863273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c009520 con 0x7f6974102030 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69727fc700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6974101680 0x7f69741974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69737fe700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974102030 0x7f697419e3e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f696400b070 tx=0x7f69640095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:32.900 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f695bfff700 1 -- 172.21.15.150:0/3106863273 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696400e040 con 0x7f6974102030 2024-05-10T04:43:32.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f695bfff700 1 -- 172.21.15.150:0/3106863273 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6964003c00 con 0x7f6974102030 2024-05-10T04:43:32.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f695bfff700 1 -- 172.21.15.150:0/3106863273 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696401b6c0 con 0x7f6974102030 2024-05-10T04:43:32.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6964009d00 con 0x7f6974102030 2024-05-10T04:43:32.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.896+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697419f180 con 0x7f6974102030 2024-05-10T04:43:32.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.897+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6974107e30 con 0x7f6974102030 2024-05-10T04:43:32.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.898+0000 7f695bfff700 1 -- 172.21.15.150:0/3106863273 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f696401d470 con 0x7f6974102030 2024-05-10T04:43:32.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.899+0000 7f695bfff700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f695c0a11c0 0x7f695c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:32.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.899+0000 7f695bfff700 1 -- 172.21.15.150:0/3106863273 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9595+0+0 (secure 0 0 0) 0x7f69640ce660 con 0x7f6974102030 2024-05-10T04:43:32.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.899+0000 7f6972ffd700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f695c0a11c0 0x7f695c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:32.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.899+0000 7f6972ffd700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f695c0a11c0 0x7f695c0a36d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f69740681e0 tx=0x7f696c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:32.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:32.902+0000 7f695bfff700 1 -- 172.21.15.150:0/3106863273 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6964097930 con 0x7f6974102030 2024-05-10T04:43:33.085 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.086 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.086 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: starting mds.b at 2024-05-10T04:43:33.086 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-h[83368]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.086 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-h[83368]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.087 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:43:32 smithi161 ceph-mds[83391]: -- 172.21.15.161:0/3176842353 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56483c2b01a0 con 0x56483c310880 2024-05-10T04:43:33.087 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-h[83368]: starting mds.h at 2024-05-10T04:43:33.087 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-k[82687]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.087 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-k[82687]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.088 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:43:32 smithi161 ceph-mds[82710]: -- 172.21.15.161:0/1626840304 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b7a0521a0 con 0x559b7a0b2d00 2024-05-10T04:43:33.088 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-k[82687]: starting mds.k at 2024-05-10T04:43:33.088 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-e[83027]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.088 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-e[83027]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.089 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:43:32 smithi161 ceph-mds[83051]: -- 172.21.15.161:0/958883311 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e90442c1a0 con 0x55e90448c400 2024-05-10T04:43:33.089 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:43:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-e[83027]: starting mds.e at 2024-05-10T04:43:33.092 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-a[42381]: 2024-05-10T04:43:32.721+0000 7f3bba11c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:43:33.093 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-a[93152]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.093 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-a[93152]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.093 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-mds[93176]: -- 172.21.15.150:0/1355893761 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5608971181a0 con 0x560897178400 2024-05-10T04:43:33.093 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:43:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-a[93152]: starting mds.a at 2024-05-10T04:43:33.093 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:43:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-g[92765]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.093 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:43:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-g[92765]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.093 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:43:32 smithi150 ceph-mds[92788]: -- 172.21.15.150:0/2048115427 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556a572281a0 con 0x556a57288400 2024-05-10T04:43:33.094 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:43:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-g[92765]: starting mds.g at 2024-05-10T04:43:33.094 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-i[83363]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.094 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-i[83363]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.094 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-i[83363]: starting mds.i at 2024-05-10T04:43:33.095 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-l[82333]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.095 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-l[82333]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.095 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-l[82333]: starting mds.l at 2024-05-10T04:43:33.096 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-c[83022]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.096 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-c[83022]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.096 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:43:32 smithi181 ceph-mds[83045]: -- 172.21.15.181:0/4141977162 <== mon.0 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559acd3ae1a0 con 0x559acd40e880 2024-05-10T04:43:33.096 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-c[83022]: starting mds.c at 2024-05-10T04:43:33.097 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-f[82683]: ignoring --setuser ceph since I am not root 2024-05-10T04:43:33.097 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-f[82683]: ignoring --setgroup ceph since I am not root 2024-05-10T04:43:33.097 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:43:32 smithi181 ceph-mds[82706]: -- 172.21.15.181:0/456925769 <== mon.2 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d860eb21a0 con 0x55d860f12400 2024-05-10T04:43:33.097 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:43:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-f[82683]: starting mds.f at 2024-05-10T04:43:33.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.149+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f697410c470 con 0x7f6974102030 2024-05-10T04:43:33.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2807216868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:33.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2856068677' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T04:43:33.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:33 smithi181 ceph-mon[47314]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T04:43:33.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:33 smithi181 ceph-mon[47314]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:43:33.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:33 smithi181 ceph-mon[47314]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T04:43:33.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:33 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/2856068677' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T04:43:33.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:33 smithi181 ceph-mon[47314]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T04:43:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2807216868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2856068677' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T04:43:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:33 smithi150 ceph-mon[42404]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T04:43:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:33 smithi150 ceph-mon[42404]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:43:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:33 smithi150 ceph-mon[42404]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T04:43:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:33 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/2856068677' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T04:43:33.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:33 smithi150 ceph-mon[42404]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T04:43:33.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2807216868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-10T04:43:33.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2856068677' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-10T04:43:33.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:33 smithi161 ceph-mon[46912]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-10T04:43:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:33 smithi161 ceph-mon[46912]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-10T04:43:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:33 smithi161 ceph-mon[46912]: osdmap e91: 12 total, 12 up, 12 in 2024-05-10T04:43:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:33 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/2856068677' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-10T04:43:33.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:33 smithi161 ceph-mon[46912]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-10T04:43:33.735 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.734+0000 7f695bfff700 1 -- 172.21.15.150:0/3106863273 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f696409af50 con 0x7f6974102030 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f695c0a11c0 msgr2=0x7f695c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f695c0a11c0 0x7f695c0a36d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f69740681e0 tx=0x7f696c009260 comp rx=0 tx=0).stop 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974102030 msgr2=0x7f697419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974102030 0x7f697419e3e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f696400b070 tx=0x7f69640095a0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 shutdown_connections 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6974102030 0x7f697419e3e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f695c0a11c0 0x7f695c0a36d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6974100d30 0x7f6974196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 --2- 172.21.15.150:0/3106863273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6974101680 0x7f69741974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 >> 172.21.15.150:0/3106863273 conn(0x7f69740fb8f0 msgr2=0x7f6974104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 shutdown_connections 2024-05-10T04:43:33.739 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.737+0000 7f69794eb700 1 -- 172.21.15.150:0/3106863273 wait complete. 2024-05-10T04:43:33.757 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-10T04:43:33.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 -- 172.21.15.150:0/838642877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa044101700 msgr2=0x7fa044109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:33.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 --2- 172.21.15.150:0/838642877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa044101700 0x7fa044109010 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b0a0 tx=0x7fa03c00b3b0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.901 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 -- 172.21.15.150:0/838642877 shutdown_connections 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 --2- 172.21.15.150:0/838642877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa044101700 0x7fa044109010 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 --2- 172.21.15.150:0/838642877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0441020a0 0x7fa0441024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 --2- 172.21.15.150:0/838642877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa044100db0 0x7fa0441011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 -- 172.21.15.150:0/838642877 >> 172.21.15.150:0/838642877 conn(0x7fa0440fb8f0 msgr2=0x7fa0440fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 -- 172.21.15.150:0/838642877 shutdown_connections 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 -- 172.21.15.150:0/838642877 wait complete. 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 Processor -- start 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.898+0000 7fa04bf27700 1 -- start start 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04bf27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa044100db0 0x7fa044198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04bf27700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa044101700 0x7fa044198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04bf27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0441020a0 0x7fa04419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04bf27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04419faf0 con 0x7fa0441020a0 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04bf27700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04419fc60 con 0x7fa044101700 2024-05-10T04:43:33.902 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04bf27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04419fe20 con 0x7fa044100db0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0441020a0 0x7fa04419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0441020a0 0x7fa04419f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60928/0 (socket says 172.21.15.150:60928) 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 -- 172.21.15.150:0/3230090315 learned_addr learned my addr 172.21.15.150:0/3230090315 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa049cc3700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa044100db0 0x7fa044198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa0494c2700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa044101700 0x7fa044198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 -- 172.21.15.150:0/3230090315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa044100db0 msgr2=0x7fa044198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa044100db0 0x7fa044198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 -- 172.21.15.150:0/3230090315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa044101700 msgr2=0x7fa044198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa044101700 0x7fa044198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 -- 172.21.15.150:0/3230090315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c009d00 con 0x7fa0441020a0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04a4c4700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0441020a0 0x7fa04419f4e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa03c007930 tx=0x7fa03c007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa03affd700 1 -- 172.21.15.150:0/3230090315 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00e040 con 0x7fa0441020a0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa03affd700 1 -- 172.21.15.150:0/3230090315 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa03c004c50 con 0x7fa0441020a0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0441a00a0 con 0x7fa0441020a0 2024-05-10T04:43:33.903 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.899+0000 7fa03affd700 1 -- 172.21.15.150:0/3230090315 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0052a0 con 0x7fa0441020a0 2024-05-10T04:43:33.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.900+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0441a0520 con 0x7fa0441020a0 2024-05-10T04:43:33.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.901+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0440678b0 con 0x7fa0441020a0 2024-05-10T04:43:33.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.902+0000 7fa03affd700 1 -- 172.21.15.150:0/3230090315 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fa03c01d470 con 0x7fa0441020a0 2024-05-10T04:43:33.904 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.902+0000 7fa03affd700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0300a11c0 0x7fa0300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:33.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.902+0000 7fa049cc3700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0300a11c0 0x7fa0300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:33.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.902+0000 7fa03affd700 1 -- 172.21.15.150:0/3230090315 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9595+0+0 (secure 0 0 0) 0x7fa03c0ce340 con 0x7fa0441020a0 2024-05-10T04:43:33.905 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.903+0000 7fa049cc3700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0300a11c0 0x7fa0300a36d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa034005950 tx=0x7fa03400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:33.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:33.907+0000 7fa03affd700 1 -- 172.21.15.150:0/3230090315 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa03c0975b0 con 0x7fa0441020a0 2024-05-10T04:43:34.108 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:34.107+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa0441a0770 con 0x7fa0441020a0 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3106863273' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: pgmap v392: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: Cluster is now healthy 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6833/3189681726,v1:172.21.15.161:6835/3189681726] up:boot 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6832/2862007432,v1:172.21.15.161:6834/2862007432] up:boot 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6836/3065781232,v1:172.21.15.181:6837/3065781232] up:boot 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.150:6839/1716471898,v1:172.21.15.150:6841/1716471898] up:boot 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6833/2215190230,v1:172.21.15.181:6834/2215190230] up:boot 2024-05-10T04:43:34.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6838/3031109912,v1:172.21.15.161:6839/3031109912] up:boot 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6832/2322694994,v1:172.21.15.181:6835/2322694994] up:boot 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.181:6838/2087872163,v1:172.21.15.181:6839/2087872163] up:boot 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.161:6836/836947260,v1:172.21.15.161:6837/836947260] up:boot 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: mds.? [v2:172.21.15.150:6838/206953859,v1:172.21.15.150:6840/206953859] up:boot 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3106863273' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: fsmap 12 up:standby 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:43:34.406 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:43:34.407 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:34 smithi181 ceph-mon[47314]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:43:34.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3106863273' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:43:34.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: pgmap v392: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2024-05-10T04:43:34.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T04:43:34.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T04:43:34.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: Cluster is now healthy 2024-05-10T04:43:34.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6833/3189681726,v1:172.21.15.161:6835/3189681726] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6832/2862007432,v1:172.21.15.161:6834/2862007432] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6836/3065781232,v1:172.21.15.181:6837/3065781232] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.150:6839/1716471898,v1:172.21.15.150:6841/1716471898] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6833/2215190230,v1:172.21.15.181:6834/2215190230] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6838/3031109912,v1:172.21.15.161:6839/3031109912] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6832/2322694994,v1:172.21.15.181:6835/2322694994] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.181:6838/2087872163,v1:172.21.15.181:6839/2087872163] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.161:6836/836947260,v1:172.21.15.161:6837/836947260] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: mds.? [v2:172.21.15.150:6838/206953859,v1:172.21.15.150:6840/206953859] up:boot 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3106863273' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: fsmap 12 up:standby 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:43:34.412 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:43:34.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:43:34.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:43:34.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:43:34.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:43:34.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:43:34.413 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:34 smithi150 ceph-mon[42404]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:43:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3106863273' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-10T04:43:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: pgmap v392: 129 pgs: 129 active+clean; 1.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2024-05-10T04:43:34.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: Cluster is now healthy 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6833/3189681726,v1:172.21.15.161:6835/3189681726] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6832/2862007432,v1:172.21.15.161:6834/2862007432] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6836/3065781232,v1:172.21.15.181:6837/3065781232] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.150:6839/1716471898,v1:172.21.15.150:6841/1716471898] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6833/2215190230,v1:172.21.15.181:6834/2215190230] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6838/3031109912,v1:172.21.15.161:6839/3031109912] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6832/2322694994,v1:172.21.15.181:6835/2322694994] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.181:6838/2087872163,v1:172.21.15.181:6839/2087872163] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.161:6836/836947260,v1:172.21.15.161:6837/836947260] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: mds.? [v2:172.21.15.150:6838/206953859,v1:172.21.15.150:6840/206953859] up:boot 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3106863273' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: fsmap 12 up:standby 2024-05-10T04:43:34.586 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-10T04:43:34.587 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:34 smithi161 ceph-mon[46912]: from='mgr.14152 172.21.15.150:0/993715438' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-10T04:43:35.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.109+0000 7fa03affd700 1 -- 172.21.15.150:0/3230090315 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa03c09abd0 con 0x7fa0441020a0 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0300a11c0 msgr2=0x7fa0300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0300a11c0 0x7fa0300a36d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa034005950 tx=0x7fa03400b410 comp rx=0 tx=0).stop 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0441020a0 msgr2=0x7fa04419f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0441020a0 0x7fa04419f4e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa03c007930 tx=0x7fa03c007960 comp rx=0 tx=0).stop 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 shutdown_connections 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0441020a0 0x7fa04419f4e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fa0300a11c0 0x7fa0300a36d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa044100db0 0x7fa044198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 --2- 172.21.15.150:0/3230090315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa044101700 0x7fa044198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.114 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 >> 172.21.15.150:0/3230090315 conn(0x7fa0440fb8f0 msgr2=0x7fa0440fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 shutdown_connections 2024-05-10T04:43:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.112+0000 7fa04bf27700 1 -- 172.21.15.150:0/3230090315 wait complete. 2024-05-10T04:43:35.115 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_metadata' removed 2024-05-10T04:43:35.128 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-10T04:43:35.268 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 -- 172.21.15.150:0/3863225819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388102050 msgr2=0x7f1388102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 --2- 172.21.15.150:0/3863225819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388102050 0x7f1388102460 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1378009990 tx=0x7f1378009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 -- 172.21.15.150:0/3863225819 shutdown_connections 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 --2- 172.21.15.150:0/3863225819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13881016b0 0x7f1388108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 --2- 172.21.15.150:0/3863225819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1388102050 0x7f1388102460 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 --2- 172.21.15.150:0/3863225819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1388100d60 0x7f1388101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 -- 172.21.15.150:0/3863225819 >> 172.21.15.150:0/3863225819 conn(0x7f13880fb880 msgr2=0x7f13880fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 -- 172.21.15.150:0/3863225819 shutdown_connections 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 -- 172.21.15.150:0/3863225819 wait complete. 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.265+0000 7f138dc6d700 1 Processor -- start 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f138dc6d700 1 -- start start 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f138dc6d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1388100d60 0x7f13881006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f138dc6d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13881016b0 0x7f138819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f138dc6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1388102050 0x7f138819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:35.269 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f138dc6d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138819f9f0 con 0x7f13881016b0 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f138dc6d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138819fb60 con 0x7f1388100d60 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f138dc6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138819fd20 con 0x7f1388102050 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13881016b0 0x7f138819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13881016b0 0x7f138819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60934/0 (socket says 172.21.15.150:60934) 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 -- 172.21.15.150:0/207202802 learned_addr learned my addr 172.21.15.150:0/207202802 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f13877fe700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1388100d60 0x7f13881006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1387fff700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1388102050 0x7f138819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 -- 172.21.15.150:0/207202802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1388102050 msgr2=0x7f138819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1388102050 0x7f138819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 -- 172.21.15.150:0/207202802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1388100d60 msgr2=0x7f13881006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1388100d60 0x7f13881006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 -- 172.21.15.150:0/207202802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13780095f0 con 0x7f13881016b0 2024-05-10T04:43:35.270 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.266+0000 7f1386ffd700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13881016b0 0x7f138819cc60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d950 tx=0x7f137c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:35.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.267+0000 7f1384ff9700 1 -- 172.21.15.150:0/207202802 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00f890 con 0x7f13881016b0 2024-05-10T04:43:35.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.267+0000 7f1384ff9700 1 -- 172.21.15.150:0/207202802 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f137c010460 con 0x7f13881016b0 2024-05-10T04:43:35.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.267+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138819ffa0 con 0x7f13881016b0 2024-05-10T04:43:35.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.267+0000 7f1384ff9700 1 -- 172.21.15.150:0/207202802 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00e610 con 0x7f13881016b0 2024-05-10T04:43:35.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.267+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13881a03c0 con 0x7f13881016b0 2024-05-10T04:43:35.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.268+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1388107e60 con 0x7f13881016b0 2024-05-10T04:43:35.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.269+0000 7f1384ff9700 1 -- 172.21.15.150:0/207202802 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7f137c0108d0 con 0x7f13881016b0 2024-05-10T04:43:35.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.269+0000 7f1384ff9700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13700a11c0 0x7f13700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:35.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.270+0000 7f13877fe700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13700a11c0 0x7f13700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:35.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.270+0000 7f1384ff9700 1 -- 172.21.15.150:0/207202802 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9041+0+0 (secure 0 0 0) 0x7f137c0cc9c0 con 0x7f13881016b0 2024-05-10T04:43:35.272 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.270+0000 7f13877fe700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13700a11c0 0x7f13700a36d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1378003730 tx=0x7f1378011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:35.274 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.273+0000 7f1384ff9700 1 -- 172.21.15.150:0/207202802 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f137c095e20 con 0x7f13881016b0 2024-05-10T04:43:35.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:35 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3230090315' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:35.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:35 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3230090315' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:35.472 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:35.471+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1388068730 con 0x7f13881016b0 2024-05-10T04:43:35.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:35 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3230090315' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:36.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.117+0000 7f1384ff9700 1 -- 172.21.15.150:0/207202802 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7f137c0160d0 con 0x7f13881016b0 2024-05-10T04:43:36.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.120+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13700a11c0 msgr2=0x7f13700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:36.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.120+0000 7f138dc6d700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13700a11c0 0x7f13700a36d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1378003730 tx=0x7f1378011040 comp rx=0 tx=0).stop 2024-05-10T04:43:36.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.120+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13881016b0 msgr2=0x7f138819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:36.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.120+0000 7f138dc6d700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13881016b0 0x7f138819cc60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d950 tx=0x7f137c00dc60 comp rx=0 tx=0).stop 2024-05-10T04:43:36.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.120+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 shutdown_connections 2024-05-10T04:43:36.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.121+0000 7f138dc6d700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7f13700a11c0 0x7f13700a36d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.121+0000 7f138dc6d700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1388102050 0x7f138819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.121+0000 7f138dc6d700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1388100d60 0x7f13881006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.121+0000 7f138dc6d700 1 --2- 172.21.15.150:0/207202802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f13881016b0 0x7f138819cc60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.121+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 >> 172.21.15.150:0/207202802 conn(0x7f13880fb880 msgr2=0x7f1388104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:36.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.121+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 shutdown_connections 2024-05-10T04:43:36.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.121+0000 7f138dc6d700 1 -- 172.21.15.150:0/207202802 wait complete. 2024-05-10T04:43:36.123 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data' removed 2024-05-10T04:43:36.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 -- 172.21.15.150:0/3492717040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba54101ab0 msgr2=0x7fba54101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 --2- 172.21.15.150:0/3492717040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba54101ab0 0x7fba54101f40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fba44009990 tx=0x7fba44009ca0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 -- 172.21.15.150:0/3492717040 shutdown_connections 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 --2- 172.21.15.150:0/3492717040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba541025d0 0x7fba54109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 --2- 172.21.15.150:0/3492717040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba541010d0 0x7fba541014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 --2- 172.21.15.150:0/3492717040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba54101ab0 0x7fba54101f40 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 -- 172.21.15.150:0/3492717040 >> 172.21.15.150:0/3492717040 conn(0x7fba540fb8f0 msgr2=0x7fba540fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 -- 172.21.15.150:0/3492717040 shutdown_connections 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.275+0000 7fba5a364700 1 -- 172.21.15.150:0/3492717040 wait complete. 2024-05-10T04:43:36.279 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 Processor -- start 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 -- start start 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba541010d0 0x7fba54196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba54101ab0 0x7fba54197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba541025d0 0x7fba5419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5419e8d0 con 0x7fba541010d0 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5419ea40 con 0x7fba54101ab0 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba5a364700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5419ec00 con 0x7fba541025d0 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba53fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba541010d0 0x7fba54196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba53fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba541010d0 0x7fba54196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:60958/0 (socket says 172.21.15.150:60958) 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba53fff700 1 -- 172.21.15.150:0/645190381 learned_addr learned my addr 172.21.15.150:0/645190381 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba537fe700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba54101ab0 0x7fba54197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.276+0000 7fba58901700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba541025d0 0x7fba5419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba53fff700 1 -- 172.21.15.150:0/645190381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba541025d0 msgr2=0x7fba5419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:36.280 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba53fff700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba541025d0 0x7fba5419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba53fff700 1 -- 172.21.15.150:0/645190381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba54101ab0 msgr2=0x7fba54197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba53fff700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba54101ab0 0x7fba54197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba53fff700 1 -- 172.21.15.150:0/645190381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba440095f0 con 0x7fba541010d0 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba58901700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba541025d0 0x7fba5419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba53fff700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba541010d0 0x7fba54196df0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fba4800d950 tx=0x7fba4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba517fa700 1 -- 172.21.15.150:0/645190381 <== mon.0 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48004220 con 0x7fba541010d0 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba517fa700 1 -- 172.21.15.150:0/645190381 <== mon.0 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba48004380 con 0x7fba541010d0 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5419eee0 con 0x7fba541010d0 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba517fa700 1 -- 172.21.15.150:0/645190381 <== mon.0 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48003bf0 con 0x7fba541010d0 2024-05-10T04:43:36.281 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.277+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5419f310 con 0x7fba541010d0 2024-05-10T04:43:36.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.278+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540678b0 con 0x7fba541010d0 2024-05-10T04:43:36.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.279+0000 7fba517fa700 1 -- 172.21.15.150:0/645190381 <== mon.0 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) v1 ==== 135874+0+0 (secure 0 0 0) 0x7fba48003d50 con 0x7fba541010d0 2024-05-10T04:43:36.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.280+0000 7fba517fa700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fba3c0a11c0 0x7fba3c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-10T04:43:36.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.280+0000 7fba537fe700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fba3c0a11c0 0x7fba3c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-10T04:43:36.282 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.280+0000 7fba517fa700 1 -- 172.21.15.150:0/645190381 <== mon.0 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8424+0+0 (secure 0 0 0) 0x7fba480cc540 con 0x7fba541010d0 2024-05-10T04:43:36.283 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.280+0000 7fba537fe700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fba3c0a11c0 0x7fba3c0a36d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba44003730 tx=0x7fba4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-10T04:43:36.285 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.284+0000 7fba517fa700 1 -- 172.21.15.150:0/645190381 <== mon.0 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fba48095bb0 con 0x7fba541010d0 2024-05-10T04:43:36.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:36 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/3230090315' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:43:36.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:36 smithi181 ceph-mon[47314]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T04:43:36.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:36 smithi181 ceph-mon[47314]: pgmap v394: 97 pgs: 97 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:43:36.405 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:36 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/207202802' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:36.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:36 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/3230090315' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:43:36.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:36 smithi150 ceph-mon[42404]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T04:43:36.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:36 smithi150 ceph-mon[42404]: pgmap v394: 97 pgs: 97 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:43:36.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:36 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/207202802' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:36.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:36.490+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba54107a10 con 0x7fba541010d0 2024-05-10T04:43:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:36 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/3230090315' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:43:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:36 smithi161 ceph-mon[46912]: osdmap e92: 12 total, 12 up, 12 in 2024-05-10T04:43:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:36 smithi161 ceph-mon[46912]: pgmap v394: 97 pgs: 97 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-10T04:43:36.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:36 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/207202802' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:37.118 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.117+0000 7fba517fa700 1 -- 172.21.15.150:0/645190381 <== mon.0 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v94) v1 ==== 178+0+0 (secure 0 0 0) 0x7fba480b7e60 con 0x7fba541010d0 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fba3c0a11c0 msgr2=0x7fba3c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fba3c0a11c0 0x7fba3c0a36d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba44003730 tx=0x7fba4401a040 comp rx=0 tx=0).stop 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba541010d0 msgr2=0x7fba54196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba541010d0 0x7fba54196df0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fba4800d950 tx=0x7fba4800dc60 comp rx=0 tx=0).stop 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 shutdown_connections 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:6800/3064659046,v1:172.21.15.150:6801/3064659046] conn(0x7fba3c0a11c0 0x7fba3c0a36d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba541025d0 0x7fba5419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba541010d0 0x7fba54196df0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:37.122 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 --2- 172.21.15.150:0/645190381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba54101ab0 0x7fba54197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-10T04:43:37.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.120+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 >> 172.21.15.150:0/645190381 conn(0x7fba540fb8f0 msgr2=0x7fba54105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-10T04:43:37.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.121+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 shutdown_connections 2024-05-10T04:43:37.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-10T04:43:37.121+0000 7fba5a364700 1 -- 172.21.15.150:0/645190381 wait complete. 2024-05-10T04:43:37.123 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'cephfs_data_ec' removed 2024-05-10T04:43:37.136 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:43:37.165 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:43:37.203 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:43:37.241 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-10T04:43:37.241 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T04:43:37.259 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T04:43:37.279 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-10T04:43:37.300 INFO:tasks.cephadm:Stopping all daemons... 2024-05-10T04:43:37.300 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-10T04:43:37.300 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a 2024-05-10T04:43:37.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:37 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/207202802' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:43:37.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:37 smithi181 ceph-mon[47314]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T04:43:37.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:37 smithi181 ceph-mon[47314]: from='client.? 172.21.15.150:0/645190381' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:37.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:37 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/207202802' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:43:37.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:37 smithi150 ceph-mon[42404]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T04:43:37.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:37 smithi150 ceph-mon[42404]: from='client.? 172.21.15.150:0/645190381' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:37.411 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:37 smithi150 systemd[1]: Stopping Ceph mon.a for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:37.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:37 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/207202802' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-10T04:43:37.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:37 smithi161 ceph-mon[46912]: osdmap e93: 12 total, 12 up, 12 in 2024-05-10T04:43:37.585 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:37 smithi161 ceph-mon[46912]: from='client.? 172.21.15.150:0/645190381' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-10T04:43:37.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:37 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-a[42381]: 2024-05-10T04:43:37.533+0000 7f3bbf927700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:43:37.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:37 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-a[42381]: 2024-05-10T04:43:37.533+0000 7f3bbf927700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-10T04:43:37.911 INFO:journalctl@ceph.mon.a.smithi150.stdout:May 10 04:43:37 smithi150 bash[116864]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-a 2024-05-10T04:43:38.012 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.a.service' 2024-05-10T04:43:38.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:38.070 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-10T04:43:38.071 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-10T04:43:38.071 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.b 2024-05-10T04:43:38.419 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:38 smithi161 systemd[1]: Stopping Ceph mon.b for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:38.419 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-b[46889]: 2024-05-10T04:43:38.296+0000 7f7bae255700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:43:38.419 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-b[46889]: 2024-05-10T04:43:38.296+0000 7f7bae255700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-10T04:43:38.738 INFO:journalctl@ceph.mon.b.smithi161.stdout:May 10 04:43:38 smithi161 bash[87163]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-b 2024-05-10T04:43:38.744 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.b.service' 2024-05-10T04:43:38.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:38.797 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-10T04:43:38.797 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-10T04:43:38.798 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.c 2024-05-10T04:43:39.140 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:38 smithi181 systemd[1]: Stopping Ceph mon.c for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:39.140 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-c[47291]: 2024-05-10T04:43:39.043+0000 7f18fcaea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:43:39.140 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-c[47291]: 2024-05-10T04:43:39.043+0000 7f18fcaea700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-10T04:43:39.404 INFO:journalctl@ceph.mon.c.smithi181.stdout:May 10 04:43:39 smithi181 bash[86725]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mon-c 2024-05-10T04:43:39.457 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mon.c.service' 2024-05-10T04:43:39.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:39.548 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-10T04:43:39.548 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-10T04:43:39.548 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.x 2024-05-10T04:43:39.881 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:43:39 smithi150 systemd[1]: Stopping Ceph mgr.x for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:39.881 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:43:39 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x[42654]: 2024-05-10T04:43:39.783+0000 7f78b6d06700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:43:40.161 INFO:journalctl@ceph.mgr.x.smithi150.stdout:May 10 04:43:39 smithi150 bash[117003]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-x 2024-05-10T04:43:40.205 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-10T04:43:40.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:40.245 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-10T04:43:40.245 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-10T04:43:40.245 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.y 2024-05-10T04:43:40.585 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:43:40 smithi161 systemd[1]: Stopping Ceph mgr.y for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:41.025 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:43:40 smithi161 bash[87312]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-y 2024-05-10T04:43:41.025 INFO:journalctl@ceph.mgr.y.smithi161.stdout:May 10 04:43:40 smithi161 systemd[1]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-10T04:43:41.031 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-10T04:43:41.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:41.069 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-10T04:43:41.069 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-10T04:43:41.069 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.z 2024-05-10T04:43:41.404 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:43:41 smithi181 systemd[1]: Stopping Ceph mgr.z for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:42.123 INFO:journalctl@ceph.mgr.z.smithi181.stdout:May 10 04:43:41 smithi181 bash[86874]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mgr-z 2024-05-10T04:43:42.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-10T04:43:42.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:42.271 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-10T04:43:42.271 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-10T04:43:42.271 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.0 2024-05-10T04:43:42.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:43:42 smithi150 systemd[1]: Stopping Ceph osd.0 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:42.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:43:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-0[60251]: 2024-05-10T04:43:42.499+0000 7f23ad340700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:43:42.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:43:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-0[60251]: 2024-05-10T04:43:42.499+0000 7f23ad340700 -1 osd.0 94 *** Got signal Terminated *** 2024-05-10T04:43:42.661 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:43:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-0[60251]: 2024-05-10T04:43:42.499+0000 7f23ad340700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:43:48.411 INFO:journalctl@ceph.osd.0.smithi150.stdout:May 10 04:43:48 smithi150 bash[117145]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-0 2024-05-10T04:43:49.355 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.0.service' 2024-05-10T04:43:49.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:49.398 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-10T04:43:49.398 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-10T04:43:49.398 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.1 2024-05-10T04:43:49.835 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:43:49 smithi161 systemd[1]: Stopping Ceph osd.1 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:49.835 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:43:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-1[51874]: 2024-05-10T04:43:49.669+0000 7f5141900700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:43:49.835 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:43:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-1[51874]: 2024-05-10T04:43:49.669+0000 7f5141900700 -1 osd.1 94 *** Got signal Terminated *** 2024-05-10T04:43:49.836 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:43:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-1[51874]: 2024-05-10T04:43:49.669+0000 7f5141900700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:43:55.586 INFO:journalctl@ceph.osd.1.smithi161.stdout:May 10 04:43:55 smithi161 bash[87450]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-1 2024-05-10T04:43:56.490 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.1.service' 2024-05-10T04:43:56.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:43:56.532 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-10T04:43:56.533 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-10T04:43:56.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.2 2024-05-10T04:43:56.904 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:43:56 smithi181 systemd[1]: Stopping Ceph osd.2 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:43:56.905 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:43:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-2[53066]: 2024-05-10T04:43:56.763+0000 7f3f4a616700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:43:56.905 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:43:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-2[53066]: 2024-05-10T04:43:56.763+0000 7f3f4a616700 -1 osd.2 94 *** Got signal Terminated *** 2024-05-10T04:43:56.905 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:43:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-2[53066]: 2024-05-10T04:43:56.763+0000 7f3f4a616700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:02.904 INFO:journalctl@ceph.osd.2.smithi181.stdout:May 10 04:44:02 smithi181 bash[87028]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-2 2024-05-10T04:44:03.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.2.service' 2024-05-10T04:44:03.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:03.710 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-10T04:44:03.710 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-10T04:44:03.710 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.3 2024-05-10T04:44:04.161 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:44:03 smithi150 systemd[1]: Stopping Ceph osd.3 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:04.161 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:44:03 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-3[68766]: 2024-05-10T04:44:03.968+0000 7f6099fe9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:04.161 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:44:03 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-3[68766]: 2024-05-10T04:44:03.968+0000 7f6099fe9700 -1 osd.3 94 *** Got signal Terminated *** 2024-05-10T04:44:04.162 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:44:03 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-3[68766]: 2024-05-10T04:44:03.968+0000 7f6099fe9700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:08.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:08 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:08.627+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:08.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:08 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:08.442+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:09.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:09 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:09.602+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:09.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:09 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:09.427+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:09.911 INFO:journalctl@ceph.osd.3.smithi150.stdout:May 10 04:44:09 smithi150 bash[117411]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-3 2024-05-10T04:44:10.619 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.3.service' 2024-05-10T04:44:10.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:10 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:10.596+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:10.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:10 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:10.386+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:10.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:10.660 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-10T04:44:10.660 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-10T04:44:10.660 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.4 2024-05-10T04:44:10.909 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:44:10 smithi161 systemd[1]: Stopping Ceph osd.4 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:10.910 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:10 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:10.634+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:11.335 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:44:10 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-4[59826]: 2024-05-10T04:44:10.907+0000 7f792f1f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:11.335 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:44:10 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-4[59826]: 2024-05-10T04:44:10.907+0000 7f792f1f3700 -1 osd.4 94 *** Got signal Terminated *** 2024-05-10T04:44:11.335 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:44:10 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-4[59826]: 2024-05-10T04:44:10.907+0000 7f792f1f3700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:11.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:11 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:11.574+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:11.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:11 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:11.394+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:12.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:11 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:11.631+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:12.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:12 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:12.553+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:12.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:12 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:12.405+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:13.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:12 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:12.599+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:13.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:13 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:13.537+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:13.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:13 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:13.514+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:13.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:13 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:13.400+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:13.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:13 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:13.583+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:14.161 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:13 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:13.908+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:14.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:14 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:14.548+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:14.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:14 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:14.469+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:14.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:14 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:14.372+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:15.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:14 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:14.601+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:15.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:14 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:14.955+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:15.589 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:15 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:15.484+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:15.590 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:15 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:15.332+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:15.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:15 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:15.589+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:16.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:15 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:15.611+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:16.161 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:15 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:15.909+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:16.162 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:15 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:15.895+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:16.654 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:16 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:16.502+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:16.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:16 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:16.378+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:16.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:16 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:16.595+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:17.085 INFO:journalctl@ceph.osd.4.smithi161.stdout:May 10 04:44:16 smithi161 bash[87721]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-4 2024-05-10T04:44:17.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:16 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:16.620+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:17.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:16 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:16.905+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:17.161 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:16 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:16.952+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:17.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:16 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:16.952+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:17.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:17 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:17.578+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:17.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:17 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:17.487+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:17.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:17 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:17.329+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:17.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:17 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:17.610+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:17.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:17 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:17.610+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:17.921 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.4.service' 2024-05-10T04:44:17.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:17.962 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-10T04:44:17.962 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-10T04:44:17.962 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.5 2024-05-10T04:44:18.326 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:18 smithi181 systemd[1]: Stopping Ceph osd.5 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:18.326 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:18 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:18.220+0000 7f86c92f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:18.326 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:18 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:18.220+0000 7f86c92f0700 -1 osd.5 94 *** Got signal Terminated *** 2024-05-10T04:44:18.326 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:18 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:18.220+0000 7f86c92f0700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:18.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:17 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:17.921+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:18.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:17 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:17.932+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:18.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:17 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:17.932+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:18.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:18 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:18.567+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:18.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:18 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:18.454+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:18.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:18 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:18.325+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:19.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:18 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:18.595+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:19.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:18 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:18.595+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:19.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:18 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:18.943+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:19.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:18 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:18.944+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:19.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:18 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:18.944+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:19.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:19 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:19.577+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:19.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:19 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:19.577+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:19.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:19 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:19.436+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:19.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:19 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:19.329+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:19.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:19 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:19.517+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:19.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:19 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:19.636+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:19.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:19 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:19.636+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:20.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:19 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:19.915+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:20.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:19 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:19.994+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:20.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:19 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:19.994+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:20.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:20 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:20.331+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:20.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:20 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:20.331+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:20.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:20 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:20.556+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:20.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:20 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:20.556+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:20.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:20 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:20.478+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:20.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:20 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:20.478+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.208847+0000 front 2024-05-10T04:43:54.208826+0000 (oldest deadline 2024-05-10T04:44:20.108792+0000) 2024-05-10T04:44:20.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:20 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:20.468+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:20.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:20 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:20.685+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:20.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:20 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:20.685+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:21.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:20 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:20.922+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:21.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:20 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:20.994+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:21.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:20 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:20.994+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:21.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:21 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:21.366+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:21.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:21 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:21.366+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:21.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:21 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:21.594+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:21.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:21 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:21.594+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:21.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:21 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:21.521+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:21.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:21 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:21.521+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.208847+0000 front 2024-05-10T04:43:54.208826+0000 (oldest deadline 2024-05-10T04:44:20.108792+0000) 2024-05-10T04:44:21.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:21 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:21.455+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:21.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:21 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:21.679+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:21.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:21 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:21.679+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:22.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:21 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:21.894+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:22.161 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:22 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:22.007+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:22.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:22 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:22.007+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:22.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:22 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:22.584+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:22.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:22 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:22.584+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:22.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:22 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:22.518+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.207631+0000 front 2024-05-10T04:43:47.207721+0000 (oldest deadline 2024-05-10T04:44:13.107598+0000) 2024-05-10T04:44:22.655 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:22 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:22.518+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.208847+0000 front 2024-05-10T04:43:54.208826+0000 (oldest deadline 2024-05-10T04:44:20.108792+0000) 2024-05-10T04:44:22.656 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:22 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5[61001]: 2024-05-10T04:44:22.518+0000 7f86c18a0700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.609592+0000 front 2024-05-10T04:44:00.609740+0000 (oldest deadline 2024-05-10T04:44:22.309885+0000) 2024-05-10T04:44:22.656 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:22 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:22.350+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:22.656 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:22 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:22.350+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:22.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:22 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:22.409+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:22.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:22 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:22.630+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:22.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:22 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:22.630+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:23.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:22 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:22.924+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:23.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:23 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:23.039+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:23.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:23 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:23.039+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:23.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:23 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:23.039+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:23.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:23 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:23.604+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:23.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:23 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:23.604+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:23.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:23 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:23.374+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:23.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:23 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:23.374+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:23.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:23 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:23.374+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:23.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:23 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:23.409+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:23.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:23 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:23.657+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:23.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:23 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:23.657+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:24.154 INFO:journalctl@ceph.osd.5.smithi181.stdout:May 10 04:44:23 smithi181 bash[87284]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-5 2024-05-10T04:44:24.162 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:23 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:23.893+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:24.162 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:23 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:23.893+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.615526+0000 front 2024-05-10T04:44:00.615608+0000 (oldest deadline 2024-05-10T04:44:23.515339+0000) 2024-05-10T04:44:24.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:24 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:24.078+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:24.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:24 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:24.078+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:24.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:24 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:24.078+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:24.601 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:24 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:24.334+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:24.601 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:24 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:24.334+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:24.601 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:24 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:24.334+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:24.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:24 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:24.440+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:24.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:24 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:24.637+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:24.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:24 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:24.637+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:24.858 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:24 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:24.600+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:24.858 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:24 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:24.601+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:24.858 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:24 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:24.601+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:25.067 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.5.service' 2024-05-10T04:44:25.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:25.127 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-10T04:44:25.127 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-10T04:44:25.127 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.6 2024-05-10T04:44:25.395 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:24 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:24.922+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:25.395 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:24 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:24.922+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.615526+0000 front 2024-05-10T04:44:00.615608+0000 (oldest deadline 2024-05-10T04:44:23.515339+0000) 2024-05-10T04:44:25.395 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:25 smithi150 systemd[1]: Stopping Ceph osd.6 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:25.395 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:25.128+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:25.395 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:25.128+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:25.395 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:25.128+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:25.651 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:25 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:25.398+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:25.651 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:25 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:25.398+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:25.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:25 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:25.620+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:25.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:25 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:25.620+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:25.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:25 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:25.620+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:25.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:25 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:25.372+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:25.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:25 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:25.372+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:25.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:25 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:25.372+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:25.661 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:25.393+0000 7f06f564b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:25.661 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:25.393+0000 7f06f564b700 -1 osd.6 94 *** Got signal Terminated *** 2024-05-10T04:44:25.661 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:25.393+0000 7f06f564b700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:26.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:25 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:25.651+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:26.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:25 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:25.651+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:26.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:25 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:25.651+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:26.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:25.942+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:26.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:25 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:25.942+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.615526+0000 front 2024-05-10T04:44:00.615608+0000 (oldest deadline 2024-05-10T04:44:23.515339+0000) 2024-05-10T04:44:26.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:26 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:26.154+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:26.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:26 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:26.154+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:26.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:26 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:26.154+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:26.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:26 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:26.433+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:26.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:26 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:26.433+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:26.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:26 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:26.649+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:26.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:26 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:26.649+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:26.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:26 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:26.649+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:26.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:26 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:26.410+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:26.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:26 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:26.410+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:26.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:26 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:26.410+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:26.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:26 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:26.597+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:26.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:26 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:26.597+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:26.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:26 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:26.597+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:27.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:26 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:26.938+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:27.411 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:26 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:26.938+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.615526+0000 front 2024-05-10T04:44:00.615608+0000 (oldest deadline 2024-05-10T04:44:23.515339+0000) 2024-05-10T04:44:27.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:27 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:27.173+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:27.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:27 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:27.173+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:27.412 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:27 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:27.173+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:27.673 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:27 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:27.400+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:27.673 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:27 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:27.400+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:27.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:27 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:27.430+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:27.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:27 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:27.430+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:27.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:27 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:27.430+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:27.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:27 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:27.596+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:27.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:27 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:27.596+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:27.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:27 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:27.596+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:28.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:27 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:27.672+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:28.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:27 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:27.672+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:28.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:27 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:27.672+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:28.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:27 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:27.909+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:28.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:27 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:27.909+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.615526+0000 front 2024-05-10T04:44:00.615608+0000 (oldest deadline 2024-05-10T04:44:23.515339+0000) 2024-05-10T04:44:28.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:28 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:28.151+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:28.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:28 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:28.151+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:28.162 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:28 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:28.151+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:28.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:28 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:28.444+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:28.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:28 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:28.444+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:28.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:28 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:28.682+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:28.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:28 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:28.682+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:28.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:28 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:28.682+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:28.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:28 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:28.574+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:28.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:28 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:28.574+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:28.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:28 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:28.574+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:28.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:28 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:28.442+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:28.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:28 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:28.442+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:28.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:28 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:28.442+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:29.200 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:28 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:28.944+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:29.200 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:28 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:28.944+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.615526+0000 front 2024-05-10T04:44:00.615608+0000 (oldest deadline 2024-05-10T04:44:23.515339+0000) 2024-05-10T04:44:29.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:29.199+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:29.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:29.199+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:29.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:29.199+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:29.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:29.634+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:29.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:29.634+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:29.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:29.634+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:29.836 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:29.429+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:29.836 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:29.429+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:29.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:29 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:29.564+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:29.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:29 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:29.564+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:29.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:29 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:29.564+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:29.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:29 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:29.425+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:29.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:29 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:29.425+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:29.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:29 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:29.425+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:30.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:29.908+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:50.113758+0000 front 2024-05-10T04:43:50.113499+0000 (oldest deadline 2024-05-10T04:44:15.413521+0000) 2024-05-10T04:44:30.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:29 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6[76714]: 2024-05-10T04:44:29.908+0000 7f06f1c03700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.615526+0000 front 2024-05-10T04:44:00.615608+0000 (oldest deadline 2024-05-10T04:44:23.515339+0000) 2024-05-10T04:44:30.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:30 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:30.213+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:30.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:30 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:30.213+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:30.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:30 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:30.213+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:30.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:30 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:30.433+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:30.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:30 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:30.433+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:30.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:30 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:30.643+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:30.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:30 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:30.643+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:30.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:30 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:30.643+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:30.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:30 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:30.643+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:30.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:30 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:30.435+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:30.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:30 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:30.435+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:30.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:30 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:30.435+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:30.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:30 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:30.516+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:30.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:30 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:30.516+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:30.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:30 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:30.516+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:31.161 INFO:journalctl@ceph.osd.6.smithi150.stdout:May 10 04:44:30 smithi150 bash[117666]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-6 2024-05-10T04:44:31.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:31 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:31.400+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:31.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:31 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:31.400+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:31.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:31 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:31.400+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:31.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:31 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:31.535+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:31.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:31 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:31.535+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:31.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:31 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:31.535+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:31.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:31.225+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:31.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:31.225+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:31.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:31 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:31.225+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:31.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:31.433+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:31.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:31.433+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:31.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:31.621+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:31.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:31.621+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:31.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:31.621+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:31.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:31 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:31.621+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:32.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.6.service' 2024-05-10T04:44:32.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:32.105 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-10T04:44:32.105 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-10T04:44:32.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.7 2024-05-10T04:44:32.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:32.235+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:32.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:32.235+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:32.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:32.235+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:32.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:32 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:32.235+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:32.437 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:32 smithi161 systemd[1]: Stopping Ceph osd.7 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:32.437 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:32.346+0000 7fda05ad2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:32.437 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:32.346+0000 7fda05ad2700 -1 osd.7 94 *** Got signal Terminated *** 2024-05-10T04:44:32.438 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:32.346+0000 7fda05ad2700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:32.438 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:32.437+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:32.438 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:32.437+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:32.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:32.437+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069072+0000 front 2024-05-10T04:44:05.068971+0000 (oldest deadline 2024-05-10T04:44:31.469642+0000) 2024-05-10T04:44:32.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:32.651+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:32.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:32.651+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:32.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:32.651+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:32.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:32 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:32.651+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:32.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:32.427+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:32.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:32.427+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:32.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:32.427+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:32.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:32.513+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:32.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:32.513+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:32.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:32 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:32.513+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:33.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:33 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:33.279+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:33.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:33 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:33.279+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:33.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:33 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:33.279+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:33.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:33 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:33.279+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:33.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:33.647+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:33.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:33.647+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:33.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:33.647+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:33.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:33.647+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:33.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:33.404+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:33.836 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:33.404+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:33.836 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:33 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:33.404+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069072+0000 front 2024-05-10T04:44:05.068971+0000 (oldest deadline 2024-05-10T04:44:31.469642+0000) 2024-05-10T04:44:33.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:33.457+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:33.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:33.457+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:33.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:33.457+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:33.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:33.504+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:33.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:33.504+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:33.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:33.504+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:34.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:34.264+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:34.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:34.264+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:34.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:34.264+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:34.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:34 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:34.264+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:34.689 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:34 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:34.435+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:34.689 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:34 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:34.435+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:34.689 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:34 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:34.435+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069072+0000 front 2024-05-10T04:44:05.068971+0000 (oldest deadline 2024-05-10T04:44:31.469642+0000) 2024-05-10T04:44:34.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:34.426+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:34.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:34.426+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:34.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:34.426+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:34.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:34.486+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:34.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:34.486+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:34.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:34 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:34.486+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:35.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:34 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:34.688+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:35.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:34 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:34.688+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:35.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:34 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:34.688+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:35.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:34 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:34.688+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:35.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:35 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:35.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:35.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:35 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:35.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:35.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:35 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:35.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:35.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:35 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:35.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:35.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:35 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:35.406+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:35.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:35 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:35.406+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:35.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:35 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:35.406+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069072+0000 front 2024-05-10T04:44:05.068971+0000 (oldest deadline 2024-05-10T04:44:31.469642+0000) 2024-05-10T04:44:35.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:35 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:35.644+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:35.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:35 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:35.644+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:35.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:35 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:35.644+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:35.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:35 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:35.644+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:35.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:35 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:35.411+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:35.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:35 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:35.411+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:35.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:35 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:35.411+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:35.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:35 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:35.530+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:35.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:35 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:35.530+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:35.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:35 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:35.530+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:36.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:36 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:36.369+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:36.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:36 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:36.369+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:36.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:36 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:36.369+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:36.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:36 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:36.482+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:36.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:36 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:36.482+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:36.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:36 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:36.482+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:36.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:36 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:36.301+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:36.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:36 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:36.301+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:36.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:36 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:36.301+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:36.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:36 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:36.301+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:36.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:36 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:36.451+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.366895+0000 front 2024-05-10T04:43:53.366861+0000 (oldest deadline 2024-05-10T04:44:18.667058+0000) 2024-05-10T04:44:36.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:36 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:36.451+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.768389+0000 front 2024-05-10T04:43:59.768590+0000 (oldest deadline 2024-05-10T04:44:25.068448+0000) 2024-05-10T04:44:36.835 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:36 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7[67900]: 2024-05-10T04:44:36.451+0000 7fd9fe082700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069072+0000 front 2024-05-10T04:44:05.068971+0000 (oldest deadline 2024-05-10T04:44:31.469642+0000) 2024-05-10T04:44:36.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:36 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:36.631+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:36.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:36 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:36.631+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:36.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:36 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:36.631+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:36.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:36 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:36.631+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:37.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:37.345+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:37.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:37.345+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:37.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:37.345+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:37.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:37.345+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.336794+0000 front 2024-05-10T04:44:14.336866+0000 (oldest deadline 2024-05-10T04:44:36.636704+0000) 2024-05-10T04:44:37.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:37.439+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:37.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:37.439+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:37.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:37.439+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:37.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:37 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:37.303+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:37.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:37 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:37.303+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:37.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:37 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:37.303+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:37.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:37 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:37.303+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:37.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:37 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:37.303+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:38.085 INFO:journalctl@ceph.osd.7.smithi161.stdout:May 10 04:44:37 smithi161 bash[87976]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-7 2024-05-10T04:44:38.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:37 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:37.620+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:38.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:37 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:37.620+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:38.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:37 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:37.620+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:38.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:37 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:37.620+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:38.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:38 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:38.350+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:38.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:38 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:38.350+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:38.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:38 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:38.350+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:38.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:38 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:38.350+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.336794+0000 front 2024-05-10T04:44:14.336866+0000 (oldest deadline 2024-05-10T04:44:36.636704+0000) 2024-05-10T04:44:38.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:38 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:38.466+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:38.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:38 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:38.466+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:38.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:38 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:38.466+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:38.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:38 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:38.309+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:38.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:38 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:38.309+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:38.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:38 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:38.309+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:38.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:38 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:38.309+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:38.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:38 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:38.309+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:38.876 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:38.598+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:38.876 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:38.598+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:38.876 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:38.598+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:38.876 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:38.598+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:38.876 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:38 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:38.598+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:39.034 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.7.service' 2024-05-10T04:44:39.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:39.075 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-10T04:44:39.075 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-10T04:44:39.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.8 2024-05-10T04:44:39.369 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 systemd[1]: Stopping Ceph osd.8 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:39.369 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:39.323+0000 7f09d90b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:39.369 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:39.323+0000 7f09d90b1700 -1 osd.8 94 *** Got signal Terminated *** 2024-05-10T04:44:39.369 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:39.323+0000 7f09d90b1700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:39.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:39.368+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:39.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:39.368+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:39.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:39.368+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:39.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:39.368+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.336794+0000 front 2024-05-10T04:44:14.336866+0000 (oldest deadline 2024-05-10T04:44:36.636704+0000) 2024-05-10T04:44:39.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:39.434+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:39.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:39.434+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:39.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:39 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:39.434+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:39.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:39 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:39.266+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:39.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:39 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:39.266+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:39.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:39 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:39.266+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:39.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:39 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:39.266+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:39.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:39 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:39.266+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:40.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:39 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:39.588+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:40.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:39 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:39.588+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:40.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:39 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:39.588+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:40.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:39 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:39.588+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:40.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:39 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:39.588+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:40.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:40 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:40.222+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:40.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:40 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:40.222+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:40.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:40 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:40.222+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:40.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:40 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:40.222+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:40.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:40 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:40.222+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:40.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:40.410+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:40.904 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:40.410+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:40.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:40.410+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:40.905 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:40.410+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.336794+0000 front 2024-05-10T04:44:14.336866+0000 (oldest deadline 2024-05-10T04:44:36.636704+0000) 2024-05-10T04:44:40.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:40.406+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:40.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:40.406+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:40.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:40.406+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:40.905 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:40 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:40.406+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:41.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:40 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:40.609+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:41.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:40 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:40.609+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:41.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:40 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:40.609+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:41.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:40 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:40.609+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:41.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:40 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:40.609+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:41.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:41.378+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:41.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:41.378+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:41.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:41.378+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:41.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:41.378+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.336794+0000 front 2024-05-10T04:44:14.336866+0000 (oldest deadline 2024-05-10T04:44:36.636704+0000) 2024-05-10T04:44:41.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:41.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:41.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:41.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:41.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:41.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:41.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:41.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:41.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:41.240+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:41.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:41.240+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:41.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:41.240+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:41.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:41.240+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:41.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:41 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:41.240+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:42.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:41 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:41.584+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:42.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:41 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:41.584+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:42.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:41 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:41.584+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:42.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:41 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:41.584+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:42.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:41 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:41.584+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:42.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:42.392+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:42.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:42.392+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:42.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:42.392+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:42.655 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:42.392+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.336794+0000 front 2024-05-10T04:44:14.336866+0000 (oldest deadline 2024-05-10T04:44:36.636704+0000) 2024-05-10T04:44:42.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:42.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:42.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:42.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:42.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:42.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:42.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:42 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:42.425+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:42.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:42.198+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:42.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:42.198+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:42.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:42.198+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:42.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:42.198+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:42.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:42 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:42.198+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:42.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:42 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:42.539+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:42.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:42 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:42.539+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:42.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:42 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:42.539+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:42.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:42 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:42.539+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:42.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:42 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:42.539+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:43.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:43.386+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:45.232191+0000 front 2024-05-10T04:43:45.232165+0000 (oldest deadline 2024-05-10T04:44:08.132104+0000) 2024-05-10T04:44:43.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:43.386+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:54.533931+0000 front 2024-05-10T04:43:54.533738+0000 (oldest deadline 2024-05-10T04:44:19.833710+0000) 2024-05-10T04:44:43.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:43.386+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.834035+0000 front 2024-05-10T04:43:59.834209+0000 (oldest deadline 2024-05-10T04:44:22.734165+0000) 2024-05-10T04:44:43.654 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8[68989]: 2024-05-10T04:44:43.386+0000 7f09d1661700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.336794+0000 front 2024-05-10T04:44:14.336866+0000 (oldest deadline 2024-05-10T04:44:36.636704+0000) 2024-05-10T04:44:43.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:43.447+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:43.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:43.447+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:43.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:43.447+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:43.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:43 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:43.447+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:43.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:43.247+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:43.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:43.247+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:43.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:43.247+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:43.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:43.247+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:43.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:43 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:43.247+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:43.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:43 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:43.564+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:43.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:43 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:43.564+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:43.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:43 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:43.564+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:43.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:43 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:43.564+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:43.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:43 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:43.564+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:44.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:44.249+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:44.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:44.249+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:44.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:44.249+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:44.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:44.249+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:44.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:44 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:44.249+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:44.871 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:44 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:44.421+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:44.871 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:44 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:44.421+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:44.871 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:44 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:44.421+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:44.871 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:44 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:44.421+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:45.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:44 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:44.602+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:45.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:44 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:44.602+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:45.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:44 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:44.602+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:45.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:44 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:44.602+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:45.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:44 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:44.602+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:45.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:44 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:44.602+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:45.154 INFO:journalctl@ceph.osd.8.smithi181.stdout:May 10 04:44:44 smithi181 bash[87538]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-8 2024-05-10T04:44:45.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:45 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:45.383+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:45.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:45 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:45.383+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:45.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:45 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:45.383+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:45.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:45 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:45.383+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:45.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:45 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:45.383+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:45.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:45.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:45.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:45.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:45.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:45.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:45.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:45.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:45.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:45 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:45.294+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:45.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:45 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:45.577+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:45.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:45 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:45.577+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:45.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:45 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:45.577+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:45.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:45 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:45.577+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:45.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:45 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:45.577+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:45.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:45 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:45.577+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:46.187 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.8.service' 2024-05-10T04:44:46.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:46.227 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-10T04:44:46.227 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-10T04:44:46.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.9 2024-05-10T04:44:46.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:46 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:46.352+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:46.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:46 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:46.352+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:46.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:46 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:46.352+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:46.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:46 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:46.352+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:46.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:46 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:46.352+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:46.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 systemd[1]: Stopping Ceph osd.9 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.288+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.288+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.288+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.288+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.288+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.458+0000 7f022425f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.458+0000 7f022425f700 -1 osd.9 94 *** Got signal Terminated *** 2024-05-10T04:44:46.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:46 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:46.458+0000 7f022425f700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:47.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:46 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:46.613+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:47.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:46 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:46.613+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:47.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:46 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:46.613+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:47.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:46 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:46.613+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:47.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:46 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:46.613+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:47.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:46 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:46.613+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:47.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:47 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:47.318+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:47.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:47 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:47.318+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:47.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:47 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:47.318+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:47.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:47 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:47.318+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:47.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:47 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:47.318+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:47.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:47.316+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:47.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:47.316+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:47.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:47.316+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:47.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:47.316+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:47.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:47.316+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:47.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:47 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:47.316+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:23.139431+0000 front 2024-05-10T04:44:23.139518+0000 (oldest deadline 2024-05-10T04:44:46.639217+0000) 2024-05-10T04:44:48.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:47 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:47.596+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:48.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:47 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:47.596+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:48.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:47 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:47.596+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:48.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:47 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:47.596+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:48.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:47 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:47.596+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:48.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:47 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:47.596+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:48.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:48 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:48.347+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:48.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:48 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:48.347+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:48.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:48 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:48.347+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:48.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:48 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:48.347+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:48.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:48 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:48.347+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:48.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:48 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:48.328+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:48.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:48 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:48.328+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:48.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:48 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:48.328+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:48.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:48 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:48.328+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:48.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:48 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:48.328+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:48.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:48 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:48.328+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:23.139431+0000 front 2024-05-10T04:44:23.139518+0000 (oldest deadline 2024-05-10T04:44:46.639217+0000) 2024-05-10T04:44:49.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:48 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:48.586+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:49.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:48 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:48.586+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:49.085 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:48 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:48.586+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:49.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:48 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:48.586+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:49.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:48 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:48.586+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:49.086 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:48 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:48.586+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:49.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:49 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:49.298+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:49.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:49 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:49.298+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:49.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:49 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:49.298+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:49.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:49 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:49.298+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:49.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:49 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:49.298+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:49.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:49 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:49.308+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:49.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:49 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:49.308+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:49.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:49 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:49.308+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:49.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:49 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:49.308+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:49.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:49 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:49.308+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:49.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:49 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:49.308+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:23.139431+0000 front 2024-05-10T04:44:23.139518+0000 (oldest deadline 2024-05-10T04:44:46.639217+0000) 2024-05-10T04:44:49.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:49.560+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:49.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:49.560+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:49.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:49.560+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:49.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:49.560+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:49.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:49.560+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:49.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:49 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:49.560+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:50.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:50 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:50.255+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:50.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:50 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:50.255+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:50.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:50 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:50.255+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:50.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:50 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:50.255+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:50.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:50 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:50.255+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:50.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:50 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:50.312+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:50.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:50 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:50.312+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:50.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:50 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:50.312+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:50.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:50 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:50.312+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:50.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:50 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:50.312+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:50.662 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:50 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:50.312+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:23.139431+0000 front 2024-05-10T04:44:23.139518+0000 (oldest deadline 2024-05-10T04:44:46.639217+0000) 2024-05-10T04:44:50.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:50 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:50.527+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:50.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:50 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:50.527+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:50.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:50 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:50.527+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:50.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:50 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:50.527+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:50.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:50 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:50.527+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:50.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:50 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:50.527+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:51.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:51 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:51.208+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:51.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:51 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:51.208+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:51.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:51 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:51.208+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:51.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:51 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:51.208+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:51.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:51 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:51.208+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:51.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:51 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:51.291+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:47.434174+0000 front 2024-05-10T04:43:47.434340+0000 (oldest deadline 2024-05-10T04:44:13.334355+0000) 2024-05-10T04:44:51.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:51 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:51.291+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.334768+0000 front 2024-05-10T04:43:53.334992+0000 (oldest deadline 2024-05-10T04:44:16.234827+0000) 2024-05-10T04:44:51.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:51 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:51.291+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.736073+0000 front 2024-05-10T04:43:59.736199+0000 (oldest deadline 2024-05-10T04:44:22.636079+0000) 2024-05-10T04:44:51.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:51 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:51.291+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:07.936826+0000 front 2024-05-10T04:44:07.936868+0000 (oldest deadline 2024-05-10T04:44:31.437020+0000) 2024-05-10T04:44:51.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:51 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:51.291+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:11.437650+0000 front 2024-05-10T04:44:11.437691+0000 (oldest deadline 2024-05-10T04:44:37.238218+0000) 2024-05-10T04:44:51.661 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:51 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9[85428]: 2024-05-10T04:44:51.291+0000 7f0220817700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:23.139431+0000 front 2024-05-10T04:44:23.139518+0000 (oldest deadline 2024-05-10T04:44:46.639217+0000) 2024-05-10T04:44:51.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:51 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:51.497+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:51.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:51 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:51.497+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:51.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:51 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:51.497+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:51.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:51 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:51.497+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:51.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:51 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:51.497+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:51.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:51 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:51.497+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:52.411 INFO:journalctl@ceph.osd.9.smithi150.stdout:May 10 04:44:52 smithi150 bash[117930]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-9 2024-05-10T04:44:52.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:52 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:52.187+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:52.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:52 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:52.187+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:52.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:52 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:52.187+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:52.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:52 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:52.187+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:52.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:52 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:52.187+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:52.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:52 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:52.487+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:52.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:52 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:52.487+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:52.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:52 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:52.487+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:52.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:52 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:52.487+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:52.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:52 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:52.487+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:52.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:52 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:52.487+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:53.114 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.9.service' 2024-05-10T04:44:53.155 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:44:53.155 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-10T04:44:53.155 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-10T04:44:53.155 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.10 2024-05-10T04:44:53.441 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 systemd[1]: Stopping Ceph osd.10 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:44:53.441 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.392+0000 7f22464f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:44:53.441 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.392+0000 7f22464f5700 -1 osd.10 94 *** Got signal Terminated *** 2024-05-10T04:44:53.441 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.392+0000 7f22464f5700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:44:53.441 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.440+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:53.442 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.440+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:53.442 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.440+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:53.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:53 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:53.171+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:53.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:53 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:53.171+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:53.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:53 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:53.171+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:53.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:53 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:53.171+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:53.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:53 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:53.171+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:53.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.440+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:53.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.440+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:53.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:53 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:53.440+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:54.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:54 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:54.200+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:54.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:54 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:54.200+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:54.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:54 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:54.200+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:54.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:54 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:54.200+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:54.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:54 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:54.200+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:54.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:54 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:54.200+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:44:54.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:54 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:54.456+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:54.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:54 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:54.456+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:54.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:54 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:54.456+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:54.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:54 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:54.456+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:54.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:54 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:54.456+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:54.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:54 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:54.456+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:54.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:54 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:54.456+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:29.071958+0000 front 2024-05-10T04:44:29.071910+0000 (oldest deadline 2024-05-10T04:44:53.771737+0000) 2024-05-10T04:44:55.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:55 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:55.195+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:55.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:55 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:55.195+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:55.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:55 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:55.195+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:55.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:55 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:55.195+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:55.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:55 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:55.195+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:55.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:55 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:55.195+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:44:55.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:55.417+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:55.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:55.417+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:55.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:55.417+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:55.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:55.417+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:55.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:55.417+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:55.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:55.417+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:55.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:55 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:55.417+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:29.071958+0000 front 2024-05-10T04:44:29.071910+0000 (oldest deadline 2024-05-10T04:44:53.771737+0000) 2024-05-10T04:44:56.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:56.197+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:56.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:56.197+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:56.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:56.197+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:56.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:56.197+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:56.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:56.197+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:56.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:56 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:56.197+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:44:56.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:56 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:56.387+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:56.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:56 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:56.387+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:56.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:56 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:56.387+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:56.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:56 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:56.387+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:56.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:56 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:56.387+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:56.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:56 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:56.387+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:56.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:56 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:56.387+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:29.071958+0000 front 2024-05-10T04:44:29.071910+0000 (oldest deadline 2024-05-10T04:44:53.771737+0000) 2024-05-10T04:44:57.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:57 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:57.226+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:57.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:57 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:57.226+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:57.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:57 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:57.226+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:57.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:57 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:57.226+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:57.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:57 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:57.226+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:57.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:57 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:57.226+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:44:57.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:57 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:57.370+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:57.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:57 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:57.370+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:57.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:57 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:57.370+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:57.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:57 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:57.370+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:57.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:57 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:57.370+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:57.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:57 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:57.370+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:57.835 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:57 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:57.370+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:29.071958+0000 front 2024-05-10T04:44:29.071910+0000 (oldest deadline 2024-05-10T04:44:53.771737+0000) 2024-05-10T04:44:58.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:58 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:58.186+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:58.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:58 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:58.186+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:58.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:58 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:58.186+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:58.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:58 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:58.186+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:58.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:58 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:58.186+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:58.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:58 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:58.186+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:44:58.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:58.352+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:44.665751+0000 front 2024-05-10T04:43:44.665829+0000 (oldest deadline 2024-05-10T04:44:10.565815+0000) 2024-05-10T04:44:58.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:58.352+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:52.767267+0000 front 2024-05-10T04:43:52.767386+0000 (oldest deadline 2024-05-10T04:44:16.867553+0000) 2024-05-10T04:44:58.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:58.352+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:44:00.968468+0000 front 2024-05-10T04:44:00.968405+0000 (oldest deadline 2024-05-10T04:44:25.068468+0000) 2024-05-10T04:44:58.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:58.352+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6814 osd.3 since back 2024-05-10T04:44:05.069114+0000 front 2024-05-10T04:44:05.069298+0000 (oldest deadline 2024-05-10T04:44:29.768997+0000) 2024-05-10T04:44:58.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:58.352+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:15.069912+0000 front 2024-05-10T04:44:15.069852+0000 (oldest deadline 2024-05-10T04:44:37.970100+0000) 2024-05-10T04:44:58.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:58.352+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:21.471281+0000 front 2024-05-10T04:44:21.471269+0000 (oldest deadline 2024-05-10T04:44:44.370991+0000) 2024-05-10T04:44:58.836 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10[76682]: 2024-05-10T04:44:58.352+0000 7f2242aad700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:29.071958+0000 front 2024-05-10T04:44:29.071910+0000 (oldest deadline 2024-05-10T04:44:53.771737+0000) 2024-05-10T04:44:59.335 INFO:journalctl@ceph.osd.10.smithi161.stdout:May 10 04:44:58 smithi161 bash[88231]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-10 2024-05-10T04:44:59.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:59 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:59.205+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:44:59.654 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:59 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:59.205+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:44:59.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:59 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:59.205+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:44:59.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:59 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:59.205+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:44:59.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:59 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:59.205+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:44:59.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:59 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:59.205+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:44:59.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:44:59 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:44:59.205+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:44:33.802772+0000 front 2024-05-10T04:44:33.802736+0000 (oldest deadline 2024-05-10T04:44:58.502717+0000) 2024-05-10T04:45:00.181 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.10.service' 2024-05-10T04:45:00.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:00.221 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-10T04:45:00.221 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-10T04:45:00.221 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.11 2024-05-10T04:45:00.463 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.160+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:45:00.464 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.160+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:45:00.464 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.160+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:45:00.464 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.160+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:45:00.464 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.160+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:45:00.464 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.160+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:45:00.464 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.160+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:44:33.802772+0000 front 2024-05-10T04:44:33.802736+0000 (oldest deadline 2024-05-10T04:44:58.502717+0000) 2024-05-10T04:45:00.464 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 systemd[1]: Stopping Ceph osd.11 for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:00.904 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.463+0000 7ff5ca799700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:00.958 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.463+0000 7ff5ca799700 -1 osd.11 94 *** Got signal Terminated *** 2024-05-10T04:45:00.959 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:00 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:00.463+0000 7ff5ca799700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-10T04:45:01.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:01.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:45:01.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:01.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:45:01.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:01.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:45:01.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:01.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:45:01.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:01.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:45:01.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:01.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:45:01.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:01 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:01.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:44:33.802772+0000 front 2024-05-10T04:44:33.802736+0000 (oldest deadline 2024-05-10T04:44:58.502717+0000) 2024-05-10T04:45:02.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:02.164+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:45:02.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:02.164+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:45:02.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:02.164+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:45:02.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:02.164+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:45:02.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:02.164+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:45:02.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:02.164+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:45:02.655 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:02 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:02.164+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:44:33.802772+0000 front 2024-05-10T04:44:33.802736+0000 (oldest deadline 2024-05-10T04:44:58.502717+0000) 2024-05-10T04:45:03.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:03.123+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:45:03.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:03.123+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:45:03.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:03.123+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:45:03.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:03.123+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:45:03.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:03.123+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:45:03.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:03.123+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:45:03.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:03 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:03.123+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:44:33.802772+0000 front 2024-05-10T04:44:33.802736+0000 (oldest deadline 2024-05-10T04:44:58.502717+0000) 2024-05-10T04:45:04.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:04.104+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:45:04.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:04.104+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:45:04.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:04.104+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:45:04.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:04.104+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:45:04.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:04.104+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:45:04.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:04.104+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:45:04.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:04 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:04.104+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:44:33.802772+0000 front 2024-05-10T04:44:33.802736+0000 (oldest deadline 2024-05-10T04:44:58.502717+0000) 2024-05-10T04:45:05.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6806 osd.0 since back 2024-05-10T04:43:42.594013+0000 front 2024-05-10T04:43:42.594062+0000 (oldest deadline 2024-05-10T04:44:08.494062+0000) 2024-05-10T04:45:05.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6804 osd.1 since back 2024-05-10T04:43:53.195226+0000 front 2024-05-10T04:43:53.195292+0000 (oldest deadline 2024-05-10T04:44:19.095183+0000) 2024-05-10T04:45:05.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2024-05-10T04:43:59.095741+0000 front 2024-05-10T04:43:59.095847+0000 (oldest deadline 2024-05-10T04:44:23.796100+0000) 2024-05-10T04:45:05.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6812 osd.4 since back 2024-05-10T04:44:14.500149+0000 front 2024-05-10T04:44:14.500240+0000 (oldest deadline 2024-05-10T04:44:40.400228+0000) 2024-05-10T04:45:05.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2024-05-10T04:44:20.400668+0000 front 2024-05-10T04:44:20.400794+0000 (oldest deadline 2024-05-10T04:44:44.500676+0000) 2024-05-10T04:45:05.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.150:6822 osd.6 since back 2024-05-10T04:44:30.302325+0000 front 2024-05-10T04:44:30.302300+0000 (oldest deadline 2024-05-10T04:44:53.802306+0000) 2024-05-10T04:45:05.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.161:6820 osd.7 since back 2024-05-10T04:44:33.802772+0000 front 2024-05-10T04:44:33.802736+0000 (oldest deadline 2024-05-10T04:44:58.502717+0000) 2024-05-10T04:45:05.405 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:05 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11[76960]: 2024-05-10T04:45:05.152+0000 7ff5c2d49700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2024-05-10T04:44:38.503287+0000 front 2024-05-10T04:44:38.503235+0000 (oldest deadline 2024-05-10T04:45:04.903817+0000) 2024-05-10T04:45:06.404 INFO:journalctl@ceph.osd.11.smithi181.stdout:May 10 04:45:06 smithi181 bash[87801]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-osd-11 2024-05-10T04:45:07.300 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@osd.11.service' 2024-05-10T04:45:07.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:07.340 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-10T04:45:07.341 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-10T04:45:07.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.a 2024-05-10T04:45:07.661 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:45:07 smithi150 systemd[1]: Stopping Ceph mds.a for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:07.661 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:45:07 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-a[93152]: 2024-05-10T04:45:07.572+0000 7fc2a5240700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:07.662 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:45:07 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-a[93152]: 2024-05-10T04:45:07.572+0000 7fc2a5240700 -1 mds.a *** got signal Terminated *** 2024-05-10T04:45:09.198 INFO:journalctl@ceph.mds.a.smithi150.stdout:May 10 04:45:08 smithi150 bash[118185]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-a 2024-05-10T04:45:09.204 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.a.service' 2024-05-10T04:45:09.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:09.241 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-10T04:45:09.241 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-10T04:45:09.241 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.d 2024-05-10T04:45:09.498 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:45:09 smithi150 systemd[1]: Stopping Ceph mds.d for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:09.911 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:45:09 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-d[92245]: 2024-05-10T04:45:09.497+0000 7fbf7acce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:09.911 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:45:09 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-d[92245]: 2024-05-10T04:45:09.497+0000 7fbf7acce700 -1 mds.d *** got signal Terminated *** 2024-05-10T04:45:13.079 INFO:journalctl@ceph.mds.d.smithi150.stdout:May 10 04:45:12 smithi150 bash[118324]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-d 2024-05-10T04:45:13.085 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.d.service' 2024-05-10T04:45:13.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:13.119 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-10T04:45:13.119 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-10T04:45:13.120 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.g 2024-05-10T04:45:13.372 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:45:13 smithi150 systemd[1]: Stopping Ceph mds.g for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:13.661 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:45:13 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-g[92765]: 2024-05-10T04:45:13.371+0000 7f87f8ef1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:13.661 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:45:13 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-g[92765]: 2024-05-10T04:45:13.371+0000 7f87f8ef1700 -1 mds.g *** got signal Terminated *** 2024-05-10T04:45:17.209 INFO:journalctl@ceph.mds.g.smithi150.stdout:May 10 04:45:16 smithi150 bash[118461]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-g 2024-05-10T04:45:17.214 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.g.service' 2024-05-10T04:45:17.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:17.248 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-10T04:45:17.249 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-10T04:45:17.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.j 2024-05-10T04:45:17.505 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:45:17 smithi150 systemd[1]: Stopping Ceph mds.j for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:17.911 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:45:17 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-j[91855]: 2024-05-10T04:45:17.503+0000 7efe2f171700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:17.911 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:45:17 smithi150 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-j[91855]: 2024-05-10T04:45:17.503+0000 7efe2f171700 -1 mds.j *** got signal Terminated *** 2024-05-10T04:45:19.411 INFO:journalctl@ceph.mds.j.smithi150.stdout:May 10 04:45:19 smithi150 bash[118600]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-j 2024-05-10T04:45:19.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.j.service' 2024-05-10T04:45:19.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:19.504 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-10T04:45:19.504 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-10T04:45:19.504 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.b 2024-05-10T04:45:19.835 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:45:19 smithi161 systemd[1]: Stopping Ceph mds.b for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:19.835 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:45:19 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: 2024-05-10T04:45:19.739+0000 7fb9840f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:19.835 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:45:19 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b[83718]: 2024-05-10T04:45:19.739+0000 7fb9840f3700 -1 mds.b *** got signal Terminated *** 2024-05-10T04:45:21.243 INFO:journalctl@ceph.mds.b.smithi161.stdout:May 10 04:45:20 smithi161 bash[88494]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-b 2024-05-10T04:45:21.249 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.b.service' 2024-05-10T04:45:21.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:21.285 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-10T04:45:21.285 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-10T04:45:21.285 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.e 2024-05-10T04:45:21.540 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:45:21 smithi161 systemd[1]: Stopping Ceph mds.e for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:21.835 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:45:21 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-e[83027]: 2024-05-10T04:45:21.538+0000 7f436a710700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:21.835 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:45:21 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-e[83027]: 2024-05-10T04:45:21.538+0000 7f436a710700 -1 mds.e *** got signal Terminated *** 2024-05-10T04:45:25.281 INFO:journalctl@ceph.mds.e.smithi161.stdout:May 10 04:45:25 smithi161 bash[88633]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-e 2024-05-10T04:45:25.286 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.e.service' 2024-05-10T04:45:25.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:25.321 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-10T04:45:25.321 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-10T04:45:25.321 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.h 2024-05-10T04:45:25.585 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:45:25 smithi161 systemd[1]: Stopping Ceph mds.h for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:26.085 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:45:25 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-h[83368]: 2024-05-10T04:45:25.587+0000 7fce1bf65700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:26.085 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:45:25 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-h[83368]: 2024-05-10T04:45:25.587+0000 7fce1bf65700 -1 mds.h *** got signal Terminated *** 2024-05-10T04:45:29.322 INFO:journalctl@ceph.mds.h.smithi161.stdout:May 10 04:45:29 smithi161 bash[88775]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-h 2024-05-10T04:45:29.328 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.h.service' 2024-05-10T04:45:29.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:29.361 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-10T04:45:29.362 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-10T04:45:29.362 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.k 2024-05-10T04:45:29.585 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:45:29 smithi161 systemd[1]: Stopping Ceph mds.k for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:30.085 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:45:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-k[82687]: 2024-05-10T04:45:29.628+0000 7fa725b08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:30.085 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:45:29 smithi161 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-k[82687]: 2024-05-10T04:45:29.628+0000 7fa725b08700 -1 mds.k *** got signal Terminated *** 2024-05-10T04:45:33.320 INFO:journalctl@ceph.mds.k.smithi161.stdout:May 10 04:45:33 smithi161 bash[88914]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-k 2024-05-10T04:45:33.326 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.k.service' 2024-05-10T04:45:33.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:33.359 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-10T04:45:33.360 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-10T04:45:33.360 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.c 2024-05-10T04:45:33.654 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:45:33 smithi181 systemd[1]: Stopping Ceph mds.c for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:33.655 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:45:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-c[83022]: 2024-05-10T04:45:33.632+0000 7f65b2d1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:33.655 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:45:33 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-c[83022]: 2024-05-10T04:45:33.632+0000 7f65b2d1c700 -1 mds.c *** got signal Terminated *** 2024-05-10T04:45:37.292 INFO:journalctl@ceph.mds.c.smithi181.stdout:May 10 04:45:36 smithi181 bash[88055]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-c 2024-05-10T04:45:37.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.c.service' 2024-05-10T04:45:37.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:37.335 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-10T04:45:37.335 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-10T04:45:37.335 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.f 2024-05-10T04:45:37.639 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:45:37 smithi181 systemd[1]: Stopping Ceph mds.f for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:37.904 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:45:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-f[82683]: 2024-05-10T04:45:37.636+0000 7f7a6811b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:37.904 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:45:37 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-f[82683]: 2024-05-10T04:45:37.636+0000 7f7a6811b700 -1 mds.f *** got signal Terminated *** 2024-05-10T04:45:41.340 INFO:journalctl@ceph.mds.f.smithi181.stdout:May 10 04:45:41 smithi181 bash[88193]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-f 2024-05-10T04:45:41.345 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.f.service' 2024-05-10T04:45:41.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:41.379 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-10T04:45:41.379 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-10T04:45:41.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.i 2024-05-10T04:45:41.640 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:45:41 smithi181 systemd[1]: Stopping Ceph mds.i for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:41.904 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:45:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-i[83363]: 2024-05-10T04:45:41.638+0000 7fb40cacd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:41.904 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:45:41 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-i[83363]: 2024-05-10T04:45:41.638+0000 7fb40cacd700 -1 mds.i *** got signal Terminated *** 2024-05-10T04:45:45.336 INFO:journalctl@ceph.mds.i.smithi181.stdout:May 10 04:45:45 smithi181 bash[88331]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-i 2024-05-10T04:45:45.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.i.service' 2024-05-10T04:45:45.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:45.378 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-10T04:45:45.379 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-10T04:45:45.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.l 2024-05-10T04:45:45.649 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:45:45 smithi181 systemd[1]: Stopping Ceph mds.l for 28928ad0-0e86-11ef-bc98-c7b262605968... 2024-05-10T04:45:45.904 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:45:45 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-l[82333]: 2024-05-10T04:45:45.647+0000 7f288f407700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-10T04:45:45.904 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:45:45 smithi181 ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-l[82333]: 2024-05-10T04:45:45.647+0000 7f288f407700 -1 mds.l *** got signal Terminated *** 2024-05-10T04:45:49.386 INFO:journalctl@ceph.mds.l.smithi181.stdout:May 10 04:45:49 smithi181 bash[88479]: ceph-28928ad0-0e86-11ef-bc98-c7b262605968-mds-l 2024-05-10T04:45:49.392 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-28928ad0-0e86-11ef-bc98-c7b262605968@mds.l.service' 2024-05-10T04:45:49.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-10T04:45:49.426 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-10T04:45:49.426 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T04:45:49.615 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:45:50.992 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T04:45:51.178 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:45:52.575 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-10T04:45:52.763 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:45:54.149 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:45:54.185 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:45:54.222 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-10T04:45:54.252 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-10T04:45:54.257 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi150/crash 2024-05-10T04:45:54.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash -- . 2024-05-10T04:45:54.302 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T04:45:54.303 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-05-10T04:45:54.305 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi161/crash 2024-05-10T04:45:54.306 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash -- . 2024-05-10T04:45:54.352 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T04:45:54.353 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-05-10T04:45:54.354 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi181/crash 2024-05-10T04:45:54.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash -- . 2024-05-10T04:45:54.396 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-10T04:45:54.396 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-05-10T04:45:54.397 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-10T04:45:54.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-10T04:45:54.435 INFO:teuthology.orchestra.run.smithi150.stderr:grep: /var/log/ceph/28928ad0-0e86-11ef-bc98-c7b262605968/ceph.log: No such file or directory 2024-05-10T04:45:54.438 INFO:tasks.cephadm:Compressing logs... 2024-05-10T04:45:54.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:45:54.481 DEBUG:teuthology.orchestra.run.smithi161:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:45:54.484 DEBUG:teuthology.orchestra.run.smithi181:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:45:54.509 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T04:45:54.510 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T04:45:54.512 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-10T04:45:58.676 INFO:tasks.cephadm:Archiving logs... 2024-05-10T04:45:58.677 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi150/log 2024-05-10T04:45:58.678 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T04:45:59.150 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi161/log 2024-05-10T04:45:59.151 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T04:45:59.678 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi181/log 2024-05-10T04:45:59.679 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-10T04:46:00.224 INFO:tasks.cephadm:Removing cluster... 2024-05-10T04:46:00.224 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 --force 2024-05-10T04:46:00.409 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:46:00.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 --force 2024-05-10T04:46:00.828 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:46:01.065 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 28928ad0-0e86-11ef-bc98-c7b262605968 --force 2024-05-10T04:46:01.253 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 28928ad0-0e86-11ef-bc98-c7b262605968 2024-05-10T04:46:01.499 INFO:tasks.cephadm:Removing cephadm ... 2024-05-10T04:46:01.500 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T04:46:01.521 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T04:46:01.539 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-10T04:46:01.556 INFO:tasks.cephadm:Teardown complete 2024-05-10T04:46:01.557 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-10T04:46:01.569 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-10T04:46:01.569 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T04:46:01.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T04:46:01.581 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-10T04:46:01.691 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-10T04:46:01.702 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-10T04:46:01.702 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:46:01.705 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:46:01.708 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-10T04:46:01.721 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2024-05-10T04:46:01.722 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2024-05-10T04:46:01.724 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2024-05-10T04:46:01.725 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:46:01.725 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-05-10T04:46:01.726 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 8 +2858us[+2858us] +/- 45ms 2024-05-10T04:46:01.726 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -1839us[-1828us] +/- 59ms 2024-05-10T04:46:01.726 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -802us[ -802us] +/- 61ms 2024-05-10T04:46:01.726 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:46:01.727 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:46:01.727 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-10T04:46:01.727 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 7 377 4 +2598us[+2801us] +/- 45ms 2024-05-10T04:46:01.727 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 8 -2095us[-1892us] +/- 59ms 2024-05-10T04:46:01.727 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -1091us[-1091us] +/- 61ms 2024-05-10T04:46:01.727 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:46:01.729 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-10T04:46:01.729 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-05-10T04:46:01.729 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 7 +2578us[+2858us] +/- 45ms 2024-05-10T04:46:01.729 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -2134us[-1854us] +/- 59ms 2024-05-10T04:46:01.729 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -1089us[ -808us] +/- 61ms 2024-05-10T04:46:01.729 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-10T04:46:01.729 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-10T04:46:01.741 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-10T04:46:01.741 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-10T04:46:01.752 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T04:46:01.850 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T04:46:01.915 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-10T04:46:01.982 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:46:02.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:46:02.016 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:46:02.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:46:02.047 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-10T04:46:02.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:46:02.077 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-10T04:46:02.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-10T04:46:02.145 INFO:teuthology.task.internal:Duration was 1748.457997 seconds 2024-05-10T04:46:02.146 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-10T04:46:02.157 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-10T04:46:02.157 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T04:46:02.160 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T04:46:02.162 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-10T04:46:02.207 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:46:02.207 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:46:02.208 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-10T04:46:02.675 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-10T04:46:02.675 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-05-10T04:46:02.675 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T04:46:02.699 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-05-10T04:46:02.700 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T04:46:02.723 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-05-10T04:46:02.723 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-10T04:46:02.747 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-10T04:46:02.747 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:46:02.751 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:46:02.765 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-10T04:46:02.840 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-10T04:46:02.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T04:46:02.842 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T04:46:02.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-10T04:46:02.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-10T04:46:02.960 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-10T04:46:02.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T04:46:02.988 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T04:46:03.015 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-10T04:46:03.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-10T04:46:03.055 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:46:03.059 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:46:03.062 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:46:03.082 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-05-10T04:46:03.086 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-05-10T04:46:03.109 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-05-10T04:46:03.131 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:46:03.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:46:03.153 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:46:03.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:46:03.169 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-10T04:46:03.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-10T04:46:03.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-10T04:46:03.199 INFO:teuthology.task.internal:Transferring archived files... 2024-05-10T04:46:03.200 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi150 2024-05-10T04:46:03.200 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T04:46:03.244 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi161 2024-05-10T04:46:03.244 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T04:46:03.287 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-10_02:33:39-fs-wip-pdonnell-testing-20240509.171335-reef-debug-distro-default-smithi/7700547/remote/smithi181 2024-05-10T04:46:03.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-10T04:46:03.330 INFO:teuthology.task.internal:Removing archive directory... 2024-05-10T04:46:03.330 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T04:46:03.333 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T04:46:03.335 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-10T04:46:03.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-10T04:46:03.394 INFO:teuthology.task.internal:Not uploading archives. 2024-05-10T04:46:03.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-10T04:46:03.409 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-10T04:46:03.409 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T04:46:03.412 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T04:46:03.414 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-10T04:46:03.426 INFO:teuthology.orchestra.run.smithi150.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 04:46 /home/ubuntu/cephtest 2024-05-10T04:46:03.428 INFO:teuthology.orchestra.run.smithi161.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 04:46 /home/ubuntu/cephtest 2024-05-10T04:46:03.437 INFO:teuthology.orchestra.run.smithi181.stdout: 262164 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 10 04:46 /home/ubuntu/cephtest 2024-05-10T04:46:03.438 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-10T04:46:03.450 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-10T04:46:03.480 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} duration: 1748.4579973220825 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-10T04:46:03.480 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-10T04:46:03.554 INFO:teuthology.run:pass